Men and women exhibit different neural, genital, and subjective arousal responses to visual sexual stimuli. The source of these sex differences is unknown. We hypothesized that men and women look differently at sexual stimuli, resulting in different responses. We used eye tracking to measure looking by 15 male and 30 female (15 normal cycling (NC) and 15 oral contracepting (OC)) heterosexual adults viewing sexually explicit photos. NC Women were tested during their menstrual, periovulatory, and luteal phases while Men and OC Women were tested at equivalent intervals, producing three test sessions per individual. Men, NC, and OC Women differed in the relative amounts of first looks towards, percent time looking at, and probability of looking at, defined regions of the pictures. Men spent more time, and had a higher probability of, looking at female faces. NC Women had more first looks towards, spent more time, and had a higher probability of, looking at genitals. OC Women spent more time, and had a higher probability of, looking at contextual regions of pictures, those featuring clothing or background. Groups did not differ in looking at the female body. Menstrual cycle phase did not affect women's looking patterns. However, differences between OC and NC groups suggest hormonal influences on attention to sexual stimuli that were unexplained by subject characteristic differences. Our finding that men and women attend to different aspects of the same visual sexual stimuli could reflect pre-existing cognitive biases that possibly contribute to sex differences in neural, subjective, and physiological arousal.