Ectomycorrhiza (ECM) is a symbiotic relation between plant and fungi that is essential for nutrient uptake of many stand forming trees. There are two conflicting views about the evolution of ECM in fungi suggesting (1) relatively few transitions to ECM followed by reversals to non-ECM, or (2) many independent origins of ECM and no reversals. In this study, we compare these, and other, hypotheses and test the impact of different models on inference. We assembled a dataset of five marker gene sequences (nuc58, nucLSU, nucSSU, rpb1, and rpb2) and 2,174 fungal taxa covering the three subphyla: Agaricomycotina, Mucoromycotina and Pezizomycotina. The fit of different models, including models with variable rates in clades or through time, to the pattern of ECM fungal taxa was tested in a Bayesian framework, and using AIC and simulations. We find that models implementing variable rates are a better fit than models without rate shift, and that the conclusion about the relative rate between ECM and non-ECM depend largely on whether rate shifts are allowed or not. We conclude that standard constant-rate ancestral state reconstruction models are not adequate for the analysis of the evolution of ECM fungi, and may give contradictory results to more extensive analyses.