Symmetric anisotropic interaction can be ferromagnetic and antiferromagnetic at the same time but for different crystallographic axes. We show that inducing competition of anisotropic interactions of orthogonal irreducible representations represents a general route to obtain new exotic magnetic states. We demonstrate it here by observing the emergence of a continuously tunable 12-layer spatial spin modulation when distorting the square lattice planes in the quasi-2D antiferromagnetic Sr2IrO4 under in situ shear strain. This translation-symmetry-breaking phase is a result of an unusual strain activated anisotropic interaction which is at the 4th order and competing with the inherent quadratic anisotropic interaction. Such a mechanism of competing anisotropy is distinct from that among the ferromagnetic, antiferromagnetic, and/or the Dzyaloshinskii-Moriya interactions, and it could be widely applicable and highly controllable in low dimensional magnets.