The incidence of the autoimmune disease type 1 diabetes is increasing, likely caused by environmental factors. A gluten-free diet has previously been shown to ameliorate autoimmune diabetes in non-obese diabetic (NOD) mice and humans. Although the exact mechanisms are not understood, interventions influencing the intestinal microbiota early in life affect the risk of type 1 diabetes. Here, we characterize how NOD mice that are fed a gluten-free (GF) diet differ from NOD mice that are fed a gluten-containing standard (STD) diet in terms of their microbiota composition by 16S rRNA gene amplicon sequencing and pancreatic immune environment by real-time quantitative PCR at the prediabetic stage at 6 and 13 weeks of age. Gut microbiota analysis revealed highly distinct microbiota compositions in both the cecum and the colon of GF-fed mice compared with STD-fed mice. The microbiotas of the GF-fed mice were characterized by an increased Firmicutes/Bacteroidetes ratio, an increased abundance of short chain fatty acid (particularly butyrate)-producing bacteria, and a reduced abundance of Lactobacilli compared with STD mice. We found that the insulitis score in the GF mice was significantly reduced compared with the STD mice and that the markers for regulatory T cells and T helper 2 cells were upregulated in the pancreas of the GF mice. In conclusion, a GF diet during pre- and early post-natal life induces shifts in the cecal and colonic microbiota compatible with a less inflammatory environment, providing a likely mechanism for the protective effect of a GF diet in humans.