Sexually antagonistic alleles are selected discordantly between the sexes. Experimental evidence indicates that sexually antagonistic fitness variation is abundant in the genome of Drosophila melanogaster. Theory predicts that the X chromosome will be enriched with this type of variation. To test this prediction in D. melanogaster, we sampled, and cytogenetically cloned, 20 X chromosomes and compared their fitness variation to genome-wide levels. At the juvenile stage, in which gender roles are most similar, the X chromosome made no detectable contribution to genome-wide fitness variation. At the adult stage, in which gender roles diverge, the X chromosome was estimated to harbour 45% of the genome-wide fitness variation and 97% of the genome-wide sexually antagonistic variation. This genomic structure has important implications for the process of sexual selection because X-linked sexually antagonistic variation contributes to negative intersexual heritability for fitness, i.e. high-fitness males (females) produce, on average, low-fitness daughters (sons).