In enumerative geometry, Virasoro constraints were first conjectured in Gromov-Witten theory with many new recent developments in the sheaf theoretic context. In this paper, we rephrase the sheaf theoretic Virasoro constraints in terms of primary states coming from a natural conformal vector in Joyce's vertex algebra. This shows that Virasoro constraints are preserved under wall-crossing. As an application, we prove the conjectural Virasoro constraints for moduli spaces of torsion-free sheaves on any curve and on surfaces with only (p , p) cohomology classes by reducing the statements to the rank 1 case. [ABSTRACT FROM AUTHOR]