We convert a GeV laser-plasma electron accelerator into a compact femtosecond-pulsed $\gamma$-ray source by inserting a $100 \mu$m-thick glass plate $\sim3$ cm after the accelerator exit. With near-unity reliability, and requiring only crude alignment, this glass plasma mirror retro-reflected spent drive laser pulses (photon energy $\hbar\omega_L = 1.17$ eV) with $>50\%$ efficiency back onto trailing electrons (peak Lorentz factor $1000 < \gamma_e < 4400$), creating an optical undulator that generated $\sim10^8 \gamma$-ray photons with sub-mrad divergence, estimated peak brilliance $\sim10^{21}$ photons/s/mm$^2$/mrad$^2$/$0.1\%$ bandwidth and negligible bremsstrahlung background. The $\gamma$-ray photon energy $E_\gamma = 4\gamma_e^2 \hbar\omega_L$, inferred from the measured $\gamma_e$ on each shot, peaked from 5 to 85 MeV, spanning a range otherwise available with comparable brilliance only from large-scale GeV-linac-based high-intensity $\gamma$-ray sources.
