The kagome metals AV$_3$Sb$_5$ (A=K,Rb,Cs) present an ideal sandbox to study the interrelation between multiple coexisting correlated phases such as charge order and superconductivity. So far, no consensus on the microscopic nature of these states has been reached as the proposals struggle to explain all their exotic physical properties. Among these, field-switchable electric magneto-chiral anisotropy (eMChA) in CsV$_3$Sb$_5$ provides intriguing evidence for a rewindable electronic chirality, yet the other family members have not been likewise investigated. Here, we present a comparative study of magneto-chiral transport between CsV$_3$Sb$_5$ and KV$_3$Sb$_5$. Despite their similar electronic structure, KV$_3$Sb$_5$ displays negligible eMChA, if any, and with no field switchability. This is in stark contrast to the non-saturating eMChA in CsV$_3$Sb$_5$ even in high fields up to 35 T. In light of their similar band structures, the stark difference in eMChA suggests its origin in the correlated states. Clearly, the V kagome nets alone are not sufficient to describe the physics and the interactions with their environment are crucial in determining the nature of their low-temperature state.
