We demonstrate an anomalous spin-orbit torque induced by the broken magnetic symmetry in the antiferromagnet IrMn. We study the magnetic structure of three phases of IrMn thin films using neutron diffraction technique. The magnetic mirror symmetry M' is broken laterally in both L10-IrMn and L12-IrMn3 but not {\gamma}-IrMn3. We observe an out-of-plane damping-like spin-orbit torque in both L10-IrMn/permalloy and L12-IrMn3/permalloy bilayers but not in {\gamma}-IrMn3/permalloy. This is consistent with both the symmetry analysis on the effects of a broken M' on spin-orbit torque and the theoretical predictions of the spin Hall effect and the Rashba-Edelstein effect. In addition, the measured spin-orbit torque efficiencies are 0.61+-0.01, 1.01+-0.03 and 0.80+-0.01 for the L10, L12 and {\gamma} phases, respectively. Our work highlights the critical roles of the magnetic asymmetry in spin-orbit torque generation.
