Here, we document microbial communities in rapid gravity filtration units, specifically serial rapid sand filters (RSFs), termed prefilters (PFs) and after- filters (AFs), fed with anoxic groundwaters low in organic carbon to prepare potable waters. A comprehensive 16S rRNA-based amplicon sequencing survey revealed a core RSF microbiome comprising few bacterial taxa (29-30 genera) dominated by Nitrospirae, Proteobacteria and Acidobacteria, with a strikingly high abundance (75-87±18%) across five examined waterworks in Denmark. Lineages within the Nitrospira genus consistently comprised the second most and most abundant fraction in PFs (27±23%) and AFs (45.2±23%), respectively, and were far more abundant than typical proteobacterial ammonium-oxidizing bacteria, suggesting a physiology beyond nitrite oxidation for Nitrospira. Within the core taxa, sequences closely related to types with ability to oxidize ammonium, nitrite, iron, manganese and methane as primary growth substrate were identified and dominated in both PFs (73.6±6%) and AFs (61.4±21%), suggesting their functional importance. Surprisingly, operational taxonomic unit richness correlated strongly and positively with sampling location in the drinking water treatment plant (from PFs to AFs), and a weaker negative correlation held for evenness. Significant spatial heterogeneity in microbial community composition was detected in both PFs and AFs, and was higher in the AFs. This is the first comprehensive documentation of microbial community diversity in RSFs treating oligotrophic groundwaters. We have identified patterns of local spatial heterogeneity and dispersal, documented surprising energy-diversity relationships, observed a large and diverse Nitrospira fraction and established a core RSF microbiome.