A harsh early family environment is related to mental and physical health in adulthood. An important question is why family environment in childhood is associated with these outcomes so long after its initial occurrence. We describe a program of research that evaluates a model linking these variables to each other. Specifically, we hypothesize that low social competence and negative emotional states may mediate relations between a harsh early family environment and physiological/neuroendocrine responses to stress, as well as long-term health outcomes. We report evidence that the model characterizes self-rated health, cortisol responses to stress, and, in males only, elevated cardiovascular responses to stress. We discuss how the social context of early life (such as SES) may affect the family environment in ways that precipitate adverse health consequences. Perspectives on comorbidities in physical and mental health are discussed.