Context Assessments of the duration of concussion recovery have primarily been limited to sport-related concussions and male contact sports. Furthermore, whereas durations of symptoms and return-to-activity (RTA) protocols encompass total recovery, the trajectory of each duration has not been examined separately. Objective To identify individual (eg, demographics, medical history), initial concussion injury (eg, symptoms), and external (eg, site) factors associated with symptom duration and RTA-protocol duration after concussion. Design Cohort study. Setting Three US military service academies. Patients or Other Participants A total of 10 604 cadets at participating US military service academies enrolled in the study and completed a baseline evaluation and up to 5 postinjury evaluations. A total of 726 cadets (451 men, 275 women) sustained concussions during the study period. Main Outcome Measure(s) Number of days from injury (1) until the participant became asymptomatic and (2) to complete the RTA protocol. Results Varsity athlete cadets took less time than nonvarsity cadets to become asymptomatic (hazard ratio [HR] = 1.75, 95% confidence interval = 1.38, 2.23). Cadets who reported less symptom severity on the Sport Concussion Assessment Tool, third edition (SCAT3), within 48 hours of concussion had 1.45 to 3.77 times shorter symptom-recovery durations than those with more symptom severity. Similar to symptom duration, varsity status was associated with a shorter RTA-protocol duration (HR = 1.74, 95% confidence interval = 1.34, 2.25), and less symptom severity on the SCAT3 was associated with a shorter RTA-protocol duration (HR range = 1.31 to 1.47). The academy that the cadet attended was associated with the RTA-protocol duration (P < .05). Conclusions The initial total number of symptoms reported and varsity athlete status were strongly associated with symptom and RTA-protocol durations. These findings suggested that external (varsity status and academy) and injury (symptom burden) factors influenced the time until RTA.