Most gene-environmental studies have focused on breast cancers generally, the preponderance of which occur after age 50. Young-onset breast cancers (YOBC) tend to be aggressive and may be etiologically different. The goal of this analysis was to assess interactions between an established 77-SNP polygenic risk score (PRS) and non-genetic risk factors for YOBC. We constructed the PRS using a family-based study of 1,291 women diagnosed with breast cancer before age 50 and their parents and unaffected sisters. We used conditional logistic regression to analyze interactions between the PRS and 14 established risk factors. In further analyses we assessed the same interactions, but for invasive cancer, estrogen receptor (ER) positive cancer and with broader inclusion of racial/ethnic groups. Results showed a decreased association between the PRS and YOBC risk for women who had ever used hormonal birth control (odds ratio [OR] = 2.20 versus 3.89) and a stronger association between the PRS and YOBC risk in pre-menopausal women (OR = 2.46 versus 1.23). Restricting the analysis to ER+ cancers or invasive cancers or using samples from all ethnic groups produced similar results. In conclusion, the PRS may interact with hormonal birth control use and with menopausal status on risk of YOBC.