Most studies of animal cognition focus on group performance and neglect individual differences and the correlational structure of cognitive abilities. Moreover, no previous studies have compared the correlational structure of cognitive abilities in nonhuman animals and humans. We compared the structure of individual differences of 106 chimpanzees and 105 two-year-old human children using 15 cognitive tasks that posed problems about the physical or social world. We found a similar factor of spatial cognition for the two species. But whereas the chimpanzees had only a single factor in addition to spatial cognition, the children had two distinct additional factors: one for physical cognition and one for social cognition. These findings, in combination with previous research, support the proposal that humans share many cognitive skills with nonhuman apes, especially for dealing with the physical world, but in addition have evolved some specialized skills of social cognition.