Water resources are the material basis for human social development. They play an essential role in human production and life. They are vital for the healthy growth of ecosystem services. The quantitative assessment of the water conservation (WC) function is of great significance to the management of regional ecosystems and the construction of ecological civilization. Gannan is an influential water supply area in the upper Yellow River Basin. Understanding the current status and future changes of its WC function can provide scientific support for the stable use and sustainable development of water resources in the upper Yellow River Basin; its ecological status is highly important. The present study quantitatively assesses the WC function of Gannan based on the InVEST model. It aims to explore the regional ecological compensation mechanism, enhance regional ecological regulation capacity and increase water supply by studying the trend in changes in the WC function and the influence mechanism. The results show that: (1) The overall trend in WC capacity in Gannan increased during the period 2000–2020. (2) The spatial distribution of WC in Gannan varies from year to year, but there is some similarity, showing a high degree in the southwest and a low degree in the northeast. (3) Climatic factors are the dominant factors underlying the spatial and temporal distribution of WC in Gannan. (4) The important areas of WC are located in the southwest and southeast of Gannan.