Tumors have evolved mechanisms to escape anti-tumor immunosurveillance. They limit humoral and cellular immune activities in the stroma and render tumors resistant to immunotherapy. Sensitizing tumor cells to immune attack is an important strategy to revert immunosuppression. However, the underlying mechanisms of immune escape are still poorly understood. Here we discover Indoleamine-2,3-dioxygenase-1 (IDO1)+ Paneth cells in the stem cell niche of intestinal crypts and tumors, which promoted immune escape of colorectal cancer (CRC). Ido1 expression in Paneth cells was strictly Stat1 dependent. Loss of IDO1+ Paneth cells in murine intestinal adenomas with tumor cell-specific Stat1 deletion had profound effects on the intratumoral immune cell composition. Patient samples and TCGA expression data suggested corresponding cells in human colorectal tumors. Thus, our data uncovered an immune escape mechanism of CRC and identify IDO1+ Paneth cells as a target for immunotherapy.