The mammalian brain consists of distinct parts that fulfil different functions. Finlay and Darlington have argued that evolution of the mammalian brain is constrained by developmental programs, suggesting that different brain parts are not free to respond individually to selection and evolve independent of other parts or overall brain size. However, comparisons among mammals with matched brain weights often reveal greater differences in brain part size, arguing against strong developmental constraints. Here we test these hypotheses using a quantitative genetic approach involving over 10,000 mice. We identify independent loci for size variation in seven key parts of the brain, and observe that brain parts show low or no phenotypic correlation, as is predicted by a mosaic scenario. We also demonstrate that variation in brain size is independently regulated from body size. The allometric relations seen at higher phylogenetic levels are thus unlikely to be the product of strong developmental constraints.