The evolution of sexual dimorphism involves an interaction between sex-specific selection and a breakdown of genetic constraints that arise because the two sexes share a genome. We examined genetic constraints and the effect of sex-specific selection on a suite of sexually dimorphic display traits in Drosophila serrata. Sexual dimorphism varied among nine natural populations covering a substantial portion of the species range. Quantitative genetic analyses showed that intersexual genetic correlations were high because of autosomal genetic variance but that the inclusion of X-linked effects reduced genetic correlations substantially, indicating that sex linkage may be an important mechanism by which intersexual genetic constraints are reduced in this species. We then explored the potential for both natural and sexual selection to influence these traits, using a 12-generation laboratory experiment in which we altered the opportunities for each process as flies adapted to a novel environment. Sexual dimorphism evolved, with natural selection reducing sexual dimorphism, whereas sexual selection tended to increase it overall. To this extent, our results are consistent with the hypothesis that sexual selection favors evolutionary divergence of the sexes. However, sex-specific responses to natural and sexual selection contrasted with the classic model because sexual selection affected females rather than males.