Abstract Deciphering biogeographic patterns of microorganisms is important for evaluating the maintenance of microbial diversity with respect to the ecosystem functions they drives. However, ecological processes shaping distribution patterns of microorganisms across large spatial‐scale watersheds remain largely unknown. Using Illumina sequencing and multiple statistical methods, we characterized distribution patterns and maintenance diversity of microorganisms (i.e., archaea, bacteria, and fungi) in soils and sediments along the Yangtze River. Distinct microbial distribution patterns were found between soils and sediments, and microbial community similarity significantly decreased with increasing geographical distance. Physicochemical properties showed a larger effect on microbial community composition than geospatial and climatic factors. Archaea and fungi displayed stronger species replacements and weaker environmental constraints in soils than that in sediments, but opposite for bacteria. Archaea, bacteria, and fungi in soils showed broader environmental breadths and stronger phylogenetic signals compared to those in sediments, suggesting stronger environmental adaptation. Stochasticity dominated community assemblies of archaea and fungi in soils and sediments, whereas determinism dominated bacterial community assembly. Our results have therefore highlighted distinct microbial distribution patterns and diversity maintenance mechanisms between soils and sediments, and emphasized important roles of species replacement, environmental adaptability, and ecological assembly processes on microbial landscape. Our findings are helpful in predicting loss of microbial diversity in the Yangtze River Basin, and might assist the establishment of environmental policies for protecting fragile watersheds.