This study explores the relation between the level of current symptoms of depression and anxiety and recalled childhood gender atypical behavior (GAB), and quality of relationships with parents among men and women who reported same-sex sexual attraction or engaged in same-sex sexual behavior and men and women who did not. Matched pairs, 79 men (n = 158) and 148 women (n = 296), with equal levels of GAB were created of Finnish participants with either same-sex sexual attraction or behavior and participants without. The measures used were retrospective questionnaires. Ratings of maternal and paternal over-control and coldness differed as a function of same-sex sexual attraction or behavior. Childhood GAB was correlated with negative ratings of parental relationships. Both same-sex sexual attraction or behavior and a history of childhood GAB affected the reported levels of current depression and anxiety. Only gender typical participants with no same-sex sexual attraction or behavior reported significantly lower levels of symptoms. The findings suggest that childhood GAB is related to later distress both among hetero- and homosexual individuals. The elevated level of psychological distress among homosexual individuals, reported in several studies, might--to some extent--be caused by their generally higher levels of childhood GAB as opposed to a homosexual orientation per se.