Abstract Background There is sufficient evidence that women with atrial fibrillation (AF) have a greater symptom burden than men with AF and are more likely to experience recurrence after catheter ablation. However, the mechanisms underlying these sex differences are unclear. Methods We prospectively enrolled 125 consecutive patients, including 40 non-AF patients and 85 AF patients, who underwent high-density voltage mapping during sinus rhythm and AF patients who underwent first ablation. Results Overall, 37 (44%) female patients with AF and 24 (60%) female non-AF patients with a mean age of 61.7 ± 11.6 years and 53.6 ± 16.7 years, respectively, were enrolled in this study. The results showed that the atrial voltage of female AF patients was significantly lower than that of male AF patients (1.11 ± 0.58 mV vs. 1.53 ± 0.65 mV; P = 0.003), while there were no significant sex differences in non-AF patients (3.02 ± 0.86 mV vs. 3.21 ± 0.84 mV; P = 0.498). Multiple linear regression analysis revealed that female sex (− 0.29, 95% confidence interval [CI] − 0.64 to − 0.13, P = 0.004) and AF type (− 0.32, 95% CI − 0.69 to − 0.13, P = 0.004) were the only factors independently associated with voltage. Compared with men, women in the paroxysmal AF group had a 3.5-fold greater incidence of recurrence (adjusted hazard ratio 4.49; 95% CI 1.101–18.332, P = 0.036). Both globally and regionally, the results showed that sex-related differences in voltage values occurred prominently in paroxysmal AF patients but not in nonparoxysmal AF patients. Conclusion Sex-related differences in atrial substrates and arrhythmia-free survival were found in paroxysmal AF patients, suggesting the existence of sex-related pathophysiological factors.