The rapidly growing interest in simulating condensed-phase materials using quantum chemistry methods calls for a library of high-quality Gaussian basis sets suitable for periodic calculations. Unfortunately, most standard Gaussian basis sets commonly used in molecular simulation show significant linear dependencies when used in close-packed solids, leading to severe numerical issues that hamper the convergence to the complete basis set (CBS) limit, especially in correlated calculations. In this work, we revisit Dunning's strategy for construction of correlation-consistent basis sets and examine the relationship between accuracy and numerical stability in periodic settings. We find that limiting the number of primitive functions avoids the appearance of problematic small exponents while still providing smooth convergence to the CBS limit. As an example, we generate double-, triple-, and quadruple-ζ correlation-consistent Gaussian basis sets for periodic calculations with Goedecker-Teter-Hutter (GTH) pseudopotentials. Our basis sets cover the main-group elements from the first three rows of the periodic table. Especially for atoms on the left side of the periodic table, our basis sets are less diffuse than those used in molecular calculations. We verify the fast and reliable convergence to the CBS limit in both Hartree-Fock and post-Hartree-Fock (MP2) calculations, using a diverse test set of 19 semiconductors and insulators.