Abstract Background Sarcopenia and diabetes are both prevalent health problems worldwide. However, little is known about the relationship between prediabetes and the prevalence and severity of sarcopenia. Therefore, the current study aimed to explore the association between glucose status and the components of sarcopenia, including low muscle mass (LMM), low muscle strength (LMS) and low gait speed (LGS) in US adults. Methods Data from the 1999 to 2002 National Health and Nutrition Examination Survey (NHANES) were analyzed. A total of 4002 participants aged ≥ 50 years with available information on glucose status (NGR: 1939 cases; prediabetes: 1172 cases; diabetes: 891 cases) and sarcopenia were included in this study. Sarcopenia was defined according to the Foundation for National Institute of Health criteria. Muscle mass, muscle strength and gait speed were used to evaluate sarcopenia and its severity. Weighed multivariable logistic regression were used to explore the association between glucose status and the components of sarcopenia. The hypothetical population attributable fraction (PAF) for the glucose status was also calculated. Results The mean age of the cohort was 63.01 ± 9.89 years, with 49.4% being male. Multiple logistic regression analysis suggested that diabetes was an independent risk factor for sarcopenia (OR = 5.470, 95% CI 1.551–19.296) and showed a marginal association with severe sarcopenia (OR = 10.693, 95% CI 0.955–119.73) compared to NGR in men, but not in women. Additionally, prediabetes was independently associated with severe sarcopenia (OR = 3.647, 95% CI 1.532–8.697), LMS (OR = 1.472, 95% CI 1.018–2.127) and LGS (OR = 1.673, 95% CI 1.054–2.655) in the entire cohort. When stratifying by gender, we further observed that prediabetes was significantly associated with LMS in men (OR = 1.897, 95% CI 1.019–3.543) and related to LMM (OR = 3.174, 95% CI 1.287–7.829) and LGS (OR = 2.075, 95% CI 1.155–3.727) in women. HbA1c was positively associated with the prevalence of sarcopenia in men (OR = 1.993, 95% CI 1.511–2.629). PAF showed that diabetes accounted for 16.3% of observed sarcopenia cases. Maintaining NGR in the entire population could have prevented 38.5% of sarcopenia cases and 50.9% of severe sarcopenia cases. Conclusions Prediabetes and diabetes were independently associated with the prevalence and severity of sarcopenia in US population. Slowing down the progression of hyperglycemia could have prevented a significant proportion of sarcopenia cases.