This review discusses the progress and ongoing efforts in optical refrigeration. Optical refrigeration is a process in which phonons are removed from a solid by anti-Stokes fluorescence. The review first summarizes the history of optical refrigeration, noting the success in cooling rare-earth-doped solids to cryogenic temperatures. It then examines in detail a four-level model of rare-earth-based optical refrigeration. This model elucidates the essential roles that the various material parameters, such as the spacing of the energy levels and the radiative quantum efficiency, play in the process of optical refrigeration. The review then describes the experimental techniques for cryogenic optical refrigeration of rare-earth-doped solids employing non-resonant and resonant optical cavities. It then examines the work on laser cooling of semiconductors, emphasizing the differences between optical refrigeration of semiconductors and rare-earth-doped solids and the new challenges and advantages of semiconductors. It then describes the significant experimental results including the observed optical refrigeration of CdS nanostructures. The review concludes by discussing the engineering challenges to the development of practical optical refrigerators, and the potential advantages and uses of these refrigerators.