We study the Sard problem for the endpoint map in some well-known classes of Carnot groups. Our first main result deals with step 2 Carnot groups, where we provide lower bounds (depending only on the algebra of the group) on the codimension of the abnormal set; it turns out that our bound is always at least 3, which improves the result proved in arXiv:1503.03610 and settles a question emerged in arXiv:1709.02854. In our second main result we characterize the abnormal set in filiform groups and show that it is either a horizontal line, or a 3-dimensional algebraic variety.