Exosomes are extracellular vesicles that contain diverse components such as genetic materials, proteins, and lipids. Owing to their distinct derivation and tissue specificity, exosomes act as double-edged swords during the development of neoplasms. On the one hand, tumor-derived exosomes can modulate the immune system during tumorigenesis by regulating inflammatory cell infiltration and oxidative stress and by promoting epithelial-to-mesenchymal transition and immune-induced tumor dormancy. On the other hand, components of specific immune cell-derived exosomes may contribute to the efficacy of antitumor immunotherapy. In this review, we demonstrate the pivotal role of exosomes in the triangular relationship in the tumor microenvironment between the tumor, inflammation, and immunity, which may provide potential strategies for tumor immunotherapy at genetic and cellular levels.