This review outlines our recent studies on the spectral organization of butterfly compound eyes, with emphasis on the Japanese yellow swallowtail butterfly, Papilio xuthus, which is the most extensively studied species. Papilio has color vision when searching for nectar among flowers, and their compound eyes are furnished with six distinct classes of spectral receptors (UV, violet, blue, green, red, broadband). The compound eyes consist of many ommatidia, each containing nine photoreceptor cells. How are the six classes of spectral receptors arranged in the ommatidia? By studying their electrophysiology, histology, and molecular biology, it was found that the Papilio ommatidia can be divided into three types according to the combination of spectral receptors they contain. Different types of ommatidia are distributed randomly over the retina. Histologically, the heterogeneity appeared to be related to red or yellow pigmentation around the rhabdom. A subset of red-pigmented ommatidia contains 3-hydroxyretinol in the distal portion, fluorescing under UV epi-illumination. The red, yellow and fluorescing pigments all play crucial roles in determining the spectral sensitivities of receptors. Spectral heterogeneity and random array of ommatidia have also been found in other lepidopteran species. Similarities and differences between species are also discussed.