PHYSICAL REVIEW B, vol.105, no.4, 2022 (SCI-Expanded)
The Shastry-Sutherland (SS) model as a canonical example of frustrated magnetism has been extensively studied. The conventional wisdom has been that the transition from the plaquette valence bond order to the Ned order is direct and potentially realizes a deconfined quantum critical point beyond the Ginzburg-Landau paradigm. This scenario, however, was challenged recently by improved numerics from density matrix renormalization group which offers evidence for a narrow gapless spin liquid between the two phases. Prompted by this controversy and to shed light on this intricate parameter regime from a fresh perspective, we report high-resolution functional renormalization group analysis of the generalized SS model. The flows of over 50 million running couplings provide a detailed picture for the evolution of spin correlations as the frequency/energy scale is dialed from the ultraviolet to the infrared to yield the zero-temperature phase diagram. The singlet dimer phase emerges as a fixed point, the Ned order is characterized by divergence in the vertex function, while the transition into and out of the plaquette order is accompanied by pronounced peaks in the plaquette susceptibility. The plaquette order is suppressed before the onset of the Ned order, lending evidence for a finite spin liquid region for J(1)/J(2) is an element of (0.77, 0.82), where the flow is continuous without any indication of divergence.