Clathrin-coated pits and caveolae are two of the most recognizable features of the plasma membrane of mammalian cells. While our understanding of the machinery regulating and driving clathrin-coated pit-mediated endocytosis has progressed dramatically, including the elucidation of the structure of individual components and partial in vitro reconstitution, the role of caveolae as alternative endocytic carriers still remains elusive 50 years after their discovery. However, recent work has started to provide new insights into endocytosis by caveolae and into apparently related pathways involving lipid raft domains. These pathways, distinguished by their exquisite sensitivity to cholesterol-sequestering agents, can involve caveolae but also exist in cells devoid of caveolins and caveolae. This review examines the current evidence for the involvement of rafts and caveolae in endocytosis and the molecular players involved in their regulation.