Despite a general trend for larger mammals to have larger brains, humans are the primates with the largest brain and number of neurons, but not the largest body mass. Why are great apes, the largest primates, not also those endowed with the largest brains? Recently, we showed that the energetic cost of the brain is a linear function of its numbers of neurons. Here we show that metabolic limitations that result from the number of hours available for feeding and the low caloric yield of raw foods impose a tradeoff between body size and number of brain neurons, which explains the small brain size of great apes compared with their large body size. This limitation was probably overcome in Homo erectus with the shift to a cooked diet. Absent the requirement to spend most available hours of the day feeding, the combination of newly freed time and a large number of brain neurons affordable on a cooked diet may thus have been a major positive driving force to the rapid increased in brain size in human evolution.