Many species groups, including mammals and many insects, determine sex using heteromorphic sex chromosomes. Diptera flies, which include the model Drosophila melanogaster, generally have XY sex chromosomes and a conserved karyotype consisting of six chromosomal arms (five large rods and a small dot), but superficially similar karyotypes may conceal the true extent of sex chromosome variation. Here, we use whole-genome analysis in 37 fly species belonging to 22 different families of Diptera and uncover tremendous hidden diversity in sex chromosome karyotypes among flies. We identify over a dozen different sex chromosome configurations, and the small dot chromosome is repeatedly used as the sex chromosome, which presumably reflects the ancestral karyotype of higher Diptera. However, we identify species with undifferentiated sex chromosomes, others in which a different chromosome replaced the dot as a sex chromosome or in which up to three chromosomal elements became incorporated into the sex chromosomes, and others yet with female heterogamety (ZW sex chromosomes). Transcriptome analysis shows that dosage compensation has evolved multiple times in flies, consistently through up-regulation of the single X in males. However, X chromosomes generally show a deficiency of genes with male-biased expression, possibly reflecting sex-specific selective pressures. These species thus provide a rich resource to study sex chromosome biology in a comparative manner and show that similar selective forces have shaped the unique evolution of sex chromosomes in diverse fly taxa.