Millions of Internet users around the world challenged science by asking why a certain photo of a dress led different observers to have surprisingly different judgments about the color of the dress. The reason this particular photo produces so diverse a variety of judgments presumably is that the photo allows a variety of interpretations about the illumination of the dress. The most obvious explanation from color science should be that observers have different implicit assumptions about the illumination in the photo. We show that the perceived color of the dress is negatively correlated with the assumed illumination along the daylight locus. Moreover, by manipulating the observers' assumptions prior to seeing the photo, we can steer how observers will see the colors of the dress. These findings confirm the idea that the perceived colors of the dress depend on the assumptions about the illumination. The phenomenon illustrates the power of unconscious inferences and implicit assumptions in perception.