Arbuscular mycorrhizal fungi (AMF) colonization in roots of putative non-mycotrophic species has been known for decades, but our knowledge of AMF community structure in non-mycotrophic plants is limited. Here, we compared AMF species composition and diversity in roots of co-occurring mycotrophic and putative non-mycotrophic herbs in two wetlands. A SSU-ITS-LSU fragment in AMF rDNA was amplified, cloned and sequenced, and used to characterize the AMF community in the roots of 16 putative non-mycotrophic and 18 mycotrophic herbs. The results showed that AMF hyphae and vesicles, but not arbuscules, were commonly present in putative non-mycotrophic plants. A total of 971 AMF sequences were obtained, and these were finally assigned to 28 operational taxonomic units (OTUs). At both sites, AMF taxon richness and Hill number based on Shannon's index in the putative non-mycotrophic herbs were similar to those for mycotrophic plants, but AMF community composition between mycotrophic and non-mycotrophic plants was significantly different. Ten AMF OTUs were uniquely detected in the putative non-mycotrophic species, and two were identified as the AMF indicators in non-mycotrophic plants. These results implied that non-mycotrophic plants may harbor a potential source of AMF diversity previously ignored which should be included in our understanding of diversity, distribution pattern, and ecological significance of root-colonizing AMF. As the first direct comparison of AMF diversity and species composition between mycotrophic and putative non-mycotrophic species in wetlands, our study has important implications for the understanding of AMF distribution patterns.