Exploring the spin-orbital ground state of Ba3CuSb2O9

Motivated by the absence of both spin freezing and a cooperative Jahn-Teller effect at the lowest measured temperatures, we study the ground state of Ba3CuSb2O9. We solve a general spin-orbital model on both the honeycomb and the decorated honeycomb lattice, revealing rich phase diagrams. The spin-orbital model on the honeycomb lattice contains an SU(4) point, where previous studies have shown the existence of a spin-orbital liquid with algebraically decaying correlations. For realistic parameters on the decorated honeycomb lattice, we find a phase that consists of clusters of nearest-neighbor spin singlets, which can be understood in terms of dimer coverings of an emergent square lattice. While the experimental situation is complicated by structural disorder, we show qualitative agreement between our theory and a range of experiments.