Introducing conductive nanofillers into a dielectric polymer is the most common way to tailor percolative composites with high dielectric constant. However, it usually contributes to large increase of dielectric loss, thereby restricting the improvement of practical energy storage capability. In this study, by regulating the conjugation degree of graphene derivatives and their interfacial interactions with polymer matrix, high dielectric constants were achieved for graphene/polymer composites and the loss was suppressed at a relatively low level. Electrochemical exfoliated graphene with high conjugation degree modified with a carboxyl-riched perylene derivative via π-π stacking interaction was used as a conductive nanofiller. Due to the high conjugation degree of graphene and carboxyl groups-induced high interfacial interactions with PVDF matrix, a high dielectric constant (e = 480) was obtained at a low filler content of 0.74 vol% and the dielectric loss was suppressed as low as 0.27. The enhanced dielectric properties could be reasonably explained by a proposed model. This research provides a new and promising route for researchers to construct graphene/polymer composites with good dielectric properties for applications in modern electronics and electric power systems.