The prevalence of sexual violence is increasingly being studied in India. Yet the determinants of sexual violence, irrespective of physical violence, remain largely unexplored. Here the authors identify the determinants of sexual violence, and additionally, explore how the presence of physical violence modifies these determinants. A cross-sectional analysis is conducted using baseline data from a longitudinal study involving young married women attending reproductive health clinics in Southern India. A multivariable logistic regression analysis is conducted to first identify determinants of sexual violence and then repeated after stratifying elements based on presence or absence of physical violence identified from participants' reports. 36% and 50% of the participants report experiencing sexual and physical violence, respectively. After adjusting for other covariates, women's partners' characteristics are found most significantly associated with their odds of experiencing sexual violence. These characteristics include husbands' primary education, employment as drivers, alcohol consumption, and having multiple sex partners. Women's contribution to household income also increases their odds of experiencing sexual violence by almost twofold; however, if they are solely responsible for "all" household income, the relationship is found to be protective. Physical violence modifies the determinants of sexual violence, and among women not experiencing physical violence, husbands' primary education and employment as drivers increase women's odds of experiencing sexual violence nearly threefold, and women who contribute "all" the household income (n = 62) do not experience sexual violence. These relationships are not significant among women experiencing physical violence. Study findings improve the understanding of the determinants of sexual violence. Future research is needed to examine the risk factors for different types of GBV independently and to tease apart the differences in risk factors depending on women's experiences. The significance of male partners' characteristics warrants in-depth research, and in order to promote gender-equitable norms, future interventions need to focus on male behaviors and men's day-to-day survival challenges, all of which likely influence conflicts in marital relationships.