At the front line of the body's immunological defense system, the gastrointestinal tract faces a large number of food-derived antigens, allergens, and nutrients, as well as commensal and pathogenic microorganisms. To maintain intestinal homeostasis, the gut immune system regulates two opposite immunological reactions: immune activation and quiescence. With their versatile immunological features, intraepithelial lymphocytes (IELs) play an important role in this regulation. IELs are mainly composed of T cells, but these T cells are immunologically distinct from peripheral T cells. Not only do IELs differ immunologically from peripheral T cells but they are also comprised of heterogeneous populations showing different phenotypes and immunological functions, as well as trafficking and developmental pathways. Though IELs in the small and large intestine share common features, they have also developed differences as they adjust to the two different environments. This review seeks to shed light on the immunological diversity of small and large intestinal IELs.