The creation of the pool of follicles available for selection and ovulation is a multi-faceted, tightly regulated process that spans the period from embryonic development through to the first reproductive cycle of the organism. In mice, this development can occur in mere weeks, but in humans, it is sustained for years. Embryonic germ cell development involves the migration of primordial germs cells to the genital ridge, and the mitotic division of germ cell nuclei without complete cytokinesis to form a multi-nucleated syncytia, or germ cell nest. Through combined actions of germ cell apoptosis and somatic cell migration, the germ cell nuclei are packaged, with surrounding granulosa cells, into primordial follicles to form the initial follicle pool. Though often dismissed as quiescent and possibly uninteresting, this initial follicle pool is actually quite dynamic. In a very strictly controlled mechanism, a large portion of the initial primordial follicles formed is lost by atresia before cycling even begins. Remaining follicles can undergo alternate fates of continued dormancy or selection leading to follicular growth and differentiation. Together, the processes involved in the fate decisions of atresia, sustained dormancy, or activation carve out the follicle pool of puberty, the pool of available oocytes from which all future reproductive cycles of the female can choose. The formation of the initial and pubertal follicle pools can be predictably affected by exogenous treatment with hormones or molecules such as activin, demonstrating the ways the ovary controls the quality and quantity of germ cells maintained. Here, we review the biological processes involved in the formation of the initial follicle pool and the follicle pool of puberty, address the alternate models for regulating germ cell number and outline how the ovary quality-controls the germ cells produced.