It is commonly known that mineral fillers significantly affect the asphalt mixture's performance. Superior flexible pavement performance can be ensured by gaining a deeper understanding of the function of filler. This research investigates the influence of three different fillers: granite dust, cement, and hydrated lime, at Australian asphalt mixtures. The testing program includes Marshall testing, moisture damage resistance, indirect tensile strength (ITS), and indirect tensile stiffness modulus (ITSM) tests of asphalt mixtures. Analysis of variance (ANOVA) was used to statistically assess the results obtained, besides damage analysis. The results indicate that using natural granite dust yields the highest resistance to moisture, while cement produces the highest stability, ITS, and ITSM. Unexpectedly, using hydrated lime filler decreases the stability/stiffness and moisture resistance of asphalt mixtures. ANOVA tests indicate that the type of filler affects ITS, TSR, and ITSM results (i.e., the p-value