Heterosexually transmitted HIV remains of critical concern in the United States and around the world, especially among vulnerable and disadvantaged women, complicated by socioeconomic circumstances, gender power, addiction, and experiences of abuse, among other conditions. Effective woman-initiated HIV prevention options, such as the female condom (FC), are needed that women can use in different sexual relationship contexts. We conducted a behavioral and attitudinal survey with 461 primarily African American and Latina (especially Puerto Rican) women in Hartford, Connecticut, to measure factors on the individual, partner relationship, peer, and community levels influencing their initial and continued use of FC (using the prototype FC1) for disease prevention. We used multivariate analyses and structural equation modeling to assess effects of multiple level factors on FC use and unprotected sex with primary, casual, and paying partners. Initial, recent, and continued FC use was associated with factors on the individual level (education, marital status, drug use, child abuse experiences, HIV status), partner level (number of sex partners, paying sex partner, relationship power), and peer level (more or influential peers saying positive things about FC). Community level factors of availability and support were consistently poor across all sectors, which limited overall FC use. Patterns differed between African American and Latina women in stages and contexts of FC use and unprotected sex. FC can make a valuable contribution to reducing heterosexually transmitted HIV among women in many circumstances. The greatest barrier to increased FC use is the lack of a supportive community environment for its promotion and use.