Two-dimensional transition metal dichalcogenides (TMD) offer a unique platform for creating van-der-Waals heterojunctions with fascinating physical properties and promising applications in optoelectronics and valleytronics. Because of their typical type-II band alignment, photoexcited electrons and holes can separate via interfacial charge transfer. To understand the nature and the dynamics of this charge transfer is of utmost importance for the design and efficiency of potential devices. However, systematic studies concerning the influence of the stacking angle on the charge transfer remain sparse. Here, we apply time- and polarization resolved second-harmonic imaging microscopy to investigate the charge-transfer dynamics for three MoS$_2$/WSe$_2$ heterostructures with different stacking angles at a previously unattainable time-resolution of $\approx$ 6 fs. For 1.70 eV excitation energy, electron transfer from WSe$_2$ to MoS$_2$ is found to depend considerably on the stacking angle with the fastest transfer time observed to be as short as 12 fs. At 1.85 eV excitation energy, ultrafast hole transfer from MoS$_2$ to hybridized states at the $\Gamma$-point or to the K-points of WSe$_2$ has to be considered. Surprisingly, the corresponding decay dynamics show only a minor stacking-angle dependence indicating that radiative recombination of indirect $\Gamma$-K excitons becomes the dominant decay route for all samples.
