Caveolae have long been implicated in endocytosis. Recent data question this link, and in the absence of specific cargoes the potential cellular function of caveolar endocytosis remains unclear. Here we develop new tools, including doubly genome-edited cell lines, to assay the subcellular dynamics of caveolae using tagged proteins expressed at endogenous levels. We find that around 5% of the cellular pool of caveolae is present on dynamic endosomes, and is delivered to endosomes in a clathrin-independent manner. Furthermore, we show that caveolae are indeed likely to bud directly from the plasma membrane. Using a genetically encoded tag for electron microscopy and ratiometric light microscopy, we go on to show that bulk membrane proteins are depleted within caveolae. Although caveolae are likely to account for only a small proportion of total endocytosis, cells lacking caveolae show fundamentally altered patterns of membrane traffic when loaded with excess glycosphingolipid. Altogether, these observations support the hypothesis that caveolar endocytosis is specialized for transport of membrane lipid.