Hazardous chemicals in food are an important cause of food safety problems. Mass spectrometry is an effective tool for the qualitative and quantitative analysis of these substances. In this paper, the fragmentation mechanisms for several chemical hazardous substances, including pesticides, veterinary drugs, mycotoxins, and other chemical pollutants classified by structural analogs, are reviewed. For each class of compounds, we summarize the characteristic fragments and neutral loss generated by cleavage in the mass spectrometry analysis. We also summarize the mechanisms applied to screen and discover new structural analogs in food. This review can help researchers analyze and confirm the structure of compounds and provide a theoretical basis for the discovery of new structural analogs in food.