Cancer stem cells are cancer cells characterized with tumor initiating capacity. β1,4-N-acetylgalactosaminyltransferase III (B4GALNT3) synthesizes GalNAcβ1-4GlcNAc (LacdiNAc) which contributes to self-renewal of mouse embryonic stem cells. We previously showed that B4GALNT3 overexpression enhances colon cancer cell malignant phenotypes in vitro and in vivo. However, the role of B4GALNT3 in cancer stemness remains unclear. We found that B4GALNT3 expression was positively correlated with advanced stages and poor survival in colorectal cancer patients. Knockdown of B4GALNT3 using small interfering (si) RNAs in colon cancer cell lines (HCT116, SW480, HCT15, and HT29 cells) decreased sphere formation and the expression of stem cell markers, OCT4 and NANOG. The expression of B4GALNT3 was upregulated in colonospheres. Interestingly, we found that B4GALNT3 primarily modified N-glycans of EGFR with LacdiNAc by Wisteria floribunda agglutinin (WFA) pull down assays. B4GALNT3 knockdown suppressed EGF-induced phosphorylation of EGFR and its downstream signaling molecules. Furthermore, EGF-induced degradation of EGFR was facilitated. In addition, EGF-induced migration and invasion were significantly suppressed by B4GALNT3 knockdown. Taken together, these data suggest B4GALNT3 regulates cancer stemness and the invasive properties of colon cancer cells through modifying EGFR glycosylation and signaling. Our results provide novel insights into the role of LacdiNAc in colorectal cancer development.