The results investigating the relationship between vitamin D levels and gestational diabetes mellitus (GDM) are inconsistent. Thus, we focused on evaluating the association of vitamin D deficiency with GDM by conducting a meta-analysis of observed studies. A systematic literature search was conducted via PubMed, MEDLINE, and Cochrane library to identify eligible studies before August 2015. The meta-analysis of 20 studies including 9209 participants showed that women with vitamin D deficiency experienced a significantly increased risk for developing GDM (odds ratio (OR) = 1.53; 95% confidence intervals (CI), 1.33, 1.75) with a little heterogeneity (I2 = 16.20%, p = 0.252). A noteworthy decrease of 4.93 nmol/L (95% CI, −6.73, −3.14) in serum 25(OH)D was demonstrated in the participants with GDM, and moderate heterogeneity was observed (I2 = 61.40%, p = 0.001). Subgroup analysis with study design showed that there were obvious heterogeneities in nested case–control studies (I2 > 52.5%, p < 0.07). Sensitivity analysis showed that exclusion of any single study did not materially alter the overall combined effect. In summary, the evidence from this meta-analysis indicates a consistent association between vitamin D deficiency and an increased risk of GDM. However, well-designed randomized controlled trials are needed to elicit the clear effect of vitamin D supplementation on prevention of GDM.