Three key metrics for readout systems in quantum processors are measurement speed, fidelity, and footprint. Fast high-fidelity readout enables midcircuit measurements, a necessary feature for many dynamic algorithms and quantum error correction, while a small footprint facilitates the design of scalable, highly connected architectures with the associated increase in computing performance. Here, we present two complementary demonstrations of fast high-fidelity single-shot readout of spins in silicon quantum dots using a compact, dispersive charge sensor: a radio-frequency single-electron box. The sensor, despite requiring fewer electrodes than conventional detectors, performs at the state of the art achieving spin readout fidelity of 99.2% in less than 6 μs fitted from a physical model. We demonstrate that low-loss high-impedance resonators, highly coupled to the sensing dot, in conjunction with Josephson parametric amplification are instrumental in achieving optimal performance. We quantify the benefit of Pauli spin blockade over spin-dependent tunneling to a reservoir, as the spin-to-charge conversion mechanism in these readout schemes. Our results place dispersive charge sensing at the forefront of readout methodologies for scalable semiconductor spin-based quantum processors.