Metastasis is the primary cause of death in colorectal cancer (CRC) patients. Emerging evidence has shown that CRC stem cells (CRCSCs) play a significant role in metastatic dissemination and tumor recurrence. However, strategies for targeting CRCSCs are limited because CRCSCs are resistant to therapeutic interventions and because the tumor microenvironment (TME) provides a supportive niche. Moreover, growing evidence highlights the critical role of CRCSCs in immune adaptation and modulation of the TME. CRCSCs escape immune surveillance by avoiding recognition by the innate immune system and shaping the TME through exosomes, cytokines, and chemokines to generate an immunosuppressive niche that facilitates cancer progression. In this review, we summarize studies investigating the immunomodulatory properties of CRCSCs and their underlying mechanisms in order to improve the efficacy of treatment strategies against advanced CRC.