We present a measurement of the differential shapes of exclusive $B\to D^* \ell \bar{\nu}_\ell$ ($B = B^-, \bar{B}^0 $ and $\ell = e, \mu$) decays with hadronic tag-side reconstruction for the full Belle data set of $711\,\mathrm{fb}^{-1}$ integrated luminosity. We extract the Caprini-Lellouch-Neubert (CLN) and Boyd-Grinstein-Lebed (BGL) form factor parameters and use an external input for the absolute branching fractions to determine the Cabibbo-Kobayashi-Maskawa matrix element and find $|V_{cb}|_\mathrm{CLN} = (40.1\pm0.9)\times 10^{-3}$ and $|V_{cb}|_\mathrm{BGL} = (40.6\pm 0.9)\times 10^{-3}$ with the zero-recoil lattice QCD point $\mathcal{F}(1) = 0.906 \pm 0.013$. We also perform a study of the impact of preliminary beyond zero-recoil lattice QCD calculations on the $|V_{cb}|$ determinations. Additionally, we present the lepton flavor universality ratio $R_{e\mu} = \mathcal{B}(B \to D^* e \bar{\nu}_e) / \mathcal{B}(B \to D^* \mu \bar{\nu}_\mu) = 0.990 \pm 0.021 \pm 0.023$, the electron and muon forward-backward asymmetry and their difference $\Delta A_{FB}=0.022\pm0.026\pm 0.007$, and the electron and muon $D^*$ longitudinal polarization fraction and their difference $\Delta F_L^{D^*} = 0.034 \pm 0.024 \pm 0.007$. The uncertainties quoted correspond to the statistical and systematic uncertainties, respectively.