Magnesium ion channels and transporters regulate the cellular concentrations of Mg(2+), which must be tightly controlled as imbalances have been associated with diseases such as osteoporosis, diabetes, and high blood pressure in humans. The channels and transporters allow the "native" Mg(2+) to be transported against a high background concentration of its major competitor, Ca(2+). Their selectivity filters (the narrowest part of the open pore) control metal ion selectivity. As the structures of Mg(2+) channels in an open conformation with bound Mg(2+) have not yet been solved, the key determinants of Mg(2+)/Ca(2+) selectivity in Mg(2+) ion channels remain elusive. Here, using density functional theory combined with continuum dielectric methods, we evaluated how the competition between Mg(2+) and Ca(2+) in model selectivity filters depends on the degree of metal hydration, which correlates with the pore size/rigidity as well as the composition and solvent accessibility of the selectivity filter. The key determinant of the selectivity for Mg(2+) over Ca(2+) in the Mg(2+) channel selectivity filter is a pore that is sufficiently large to accommodate hexahydrated Mg ions. In such wide pores, the hexahydrated metal ions interact indirectly with the protein ligands, hence metal desolvation and ligand-ligand steric repulsion become less important than Mg(2+)-water-protein interactions. These wide pores are Mg(2+)-selective because compared to Ca(2+) or Na(+) and K(+) monocations, Mg(2+) better polarizes the bound water molecules resulting in stronger Mg(2+)-water-protein interactions. Although both tetrameric and pentameric filters with pores that can accommodate hexahydrated metal ions could select Mg(2+) over Ca(2+), a bilayered pentameric filter lined with a ring of amides and a ring of carboxylates seems to best discriminate the "native" Mg(2+) from its key rival, Ca(2+). Our results are consistent with available experimental data and help to elucidate the selectivity filters in the Mg(2+)-selective TRPM6 and CorA channels.