Recent evidence has shown that early exposure to stressful stimuli in the environment can impact the monoaminergic neurotransmitter systems, the structural development of the brain, and alter gene expression. In this article, we review data from animal studies, nonhuman primate studies, and human studies, which illustrate that environmental factors can influence brain chemistry and behavior. One important early social interaction is the mother-infant dyad. The nature of the infant-mother relationship appears to be important to exacerbating a potential vulnerability to affective disorders. Severe neglect, neglectful and/or emotionless parenting appears to have specific and long lasting biochemical and behavioral consequences. The exact interplay of genetic and environmental factors in affective disorders is unknown; however, we propose a model in which exposure to early adverse stimuli in an affective disorder susceptible or nonsusceptible individual during the first 3 years of life can alter gene expression leading to an increase of affective disorders.