Until very recently, helium had remained the last naturally occurring element that was known not to form stable solid compounds. Here we propose and demonstrate that there is a general driving force for helium to react with ionic compounds that contain an unequal number of cations and anions. The corresponding reaction products are stabilized not by local chemical bonds but by long-range Coulomb interactions that are significantly modified by the insertion of helium atoms, especially under high pressure. This mechanism also explains the recently discovered reactivity of He and Na under pressure. Our work reveals that helium has the propensity to react with a broad range of ionic compounds at pressures as low as 30 GPa. Since most of the Earth's minerals contain unequal numbers of positively and negatively charged atoms, our work suggests that large quantities of He might be stored in the Earth's lower mantle.