In the present work, the porous MgO nanosheet-modified activated carbon fiber felt (MgO@ACFF) was prepared for fluoride removal. The MgO@ACFF was characterized by XRD, SEM, TEM, EDS, TG, and BET. The fluoride adsorption performance of MgO@ACFF also has been investigated. The adsorption rate of the MgO@ACFF toward fluoride is fast; more than 90% of the fluoride ions can be adsorbed within 100 min, and the adsorption kinetics of MgO@ACFF can be fitted in a pseudo-second-order model. The adsorption isotherm of MgO@ACFF fitted well in the Freundlich model. Additionally, the fluoride adsorption capacity of MgO@ACFF is larger than 212.2 mg/g at neutral. In a wide pH range of 2–10, the MgO@ACFF can efficiently remove fluoride from water, which is meaningful for practical usage. The effect of co-existing anions on the fluoride removal efficiency of the MgO@ACFF also has been studied. Furthermore, the fluoride adsorption mechanism of the MgO@ACFF was studied by the FTIR and XPS, and the results reveal a hydroxyl and carbonate co-exchange mechanism. The column test of the MgO@ACFF also has been investigated; 505-bed volumes of 5 mg/L fluoride solution can be treated with effluent under 1.0 mg/L. It is believed that the MgO@ACFF is a potential candidate for a fluoride adsorbent.