How and why diverse sexual systems evolve are fascinating evolutionary questions, but few empirical studies have dealt with these questions in animals. Pedunculate (gooseneck) barnacles show such diversity, including simultaneous hermaphroditism, coexistence of dwarf males and hermaphrodites (androdioecy), and coexistence of dwarf males and females (dioecy). Here, we report the first phylogenetically controlled test of the hypothesis that the ultimate cause of the diverse sexual systems and presence of dwarf males in this group is limited mating opportunities for non-dwarf individuals, owing to mating in small groups. Within the pedunculate barnacle phylogeny, dwarf males and females have evolved repeatedly. Females are more likely to evolve in androdioecious than hermaphroditic populations, suggesting that evolution of dwarf males has preceded that of females in pedunculates. Both dwarf males and females are associated with a higher proportion of solitary individuals in the population, corroborating the hypothesis that limited mating opportunities have favoured evolution of these diverse sexual systems, which have puzzled biologists since Darwin.