Centromere-associated protein-E (CENP-E) is a kinesin motor localizing at kinetochores. Although its mitotic functions have been well studied, it has been challenging to investigate direct consequences of CENP-E removal using conventional methods because CENP-E depletion resulted in mitotic arrest. In this study, we harnessed an auxin-inducible degron system to achieve acute degradation of CENP-E. We revealed a kinetochore-independent role for CENP-E that removes pericentriolar material 1 (PCM1) from centrosomes in late S/early G2 phase. After acute loss of CENP-E, centrosomal Polo-like kinase 1 (Plk1) localization is abrogated through accumulation of PCM1, resulting in aberrant phosphorylation and destabilization of centrosomes, which triggers shortened astral microtubules and oblique cell divisions. Furthermore, we also observed centrosome and cell division defects in cells from a microcephaly patient with mutations in CENPE. Orientation of cell division is deregulated in some microcephalic patients, and our unanticipated findings provide additional insights into how microcephaly can result from centrosomal defects.