Background Difficult children are ones whose behavior deviates from the norm, which manifests as restlessness, violence, and difficulty in separating from the mother. Such problematic behaviors usually exhaust their parents during child rearing. This study aimed to identify individual and environmental factors that influence children’s problematic behavior, which could be helpful in supporting parents’ child rearing. Methods Records of children’s problematic behaviors and their individual or environmental information were collected from 8691 children at their 5-year-old health checks. Problematic behaviors were divided into three categories; anxious behaviors, developmental behaviors, and personal habits. Individual factors included sex, parental age, birth order, birth weight, and birth abnormalities. The environmental factors were mother’s smoking during pregnancy or currently, partner’s cooperation in child rearing, having someone to consult about child rearing, and television viewing time. Using logistic regression, we identified the association between such behaviors and aggravating factors. Results Problematic behavior was identified in 2.2%, 11.5%, and 16.1% of cases, respectively, with regard to anxious behaviors, developmental behaviors, and personal habits. The individual factors (including birth order and birth abnormality), and the environmental factors (including mothers currently smoking, lack of someone to consult about child rearing, and long television-watching time) were associated with the odd ratio of increased risk for some problematic behaviors. Conclusion Behaviors in difficult children are not influenced by individual factors but by several environmental factors. To reduce the parental child rearing burden, health providers should be aware of these aggravating factors.