Objectives Although claims data are widely used in medical research, their ability to identify persons' health-related conditions has not been fully justified. We assessed the validity of claims-based algorithms (CBAs) for identifying people with common chronic conditions in a large population using annual health screening results as the gold standard. Study Design and Setting Using a longitudinal claims database (n = 523,267) combined with annual health screening results, we defined the people with hypertension, diabetes, and/or dyslipidemia by applying health screening results as their gold standard and compared them against various CBAs. Results By using diagnostic and medication code-based CBAs, sensitivity and specificity were 74.5% (95% confidence interval [CI], 74.2%–74.8%) and 98.2% (98.2%–98.3%) for hypertension, 78.6% (77.3%–79.8%) and 99.6% (99.5%–99.6%) for diabetes, and 34.5% (34.2%–34.7%) and 97.2% (97.2%–97.3%) for dyslipidemia, respectively. Sensitivity did not decrease substantially for hypertension (65.2% [95% CI, 64.9%–65.5%]) and diabetes (73.0% [71.7%–74.2%]) when we used the same CBAs without limiting to primary care settings. Conclusion We used regularly collected data to obtain CBA association measures, which are applicable to a wide range of populations. Our framework can be a basis of the validity assessment of CBAs for identifying persons' health-related conditions with regularly collected data.