The establishment and maintenance of cell polarity is important to a wide range of biological processes ranging from chemotaxis to embryogenesis. An essential feature of cell polarity is the asymmetric organization of proteins and lipids in the plasma membrane. In this article, we discuss how polarity regulators such as small GTP-binding proteins and phospholipids spatially and kinetically control vesicular trafficking and membrane organization. Conversely, we discuss how membrane trafficking contributes to cell polarization through delivery of polarity determinants and regulators to the plasma membrane.