In this study, Chinese yam polysaccharide (CYP) was isolated from yam by hydroextraction and alcoholic precipitation. Subsequently, the chlorosulfate-pyridine (CSA-Pyr) method was used to obtain the sulfated Chinese yam polysaccharide derivative (S-CYP) to evaluate its immunomodulatory activity in RAW 264.7 cells and to investigate its mechanism of action. The results revealed that the sulfated modification altered the physicochemical properties of CYP but had no impact on the main chain structure. S-CYP demonstrated excellent immunomodulatory activity by increasing the viability of RAW 264.7 macrophage cells and stimulating the production of reactive oxygen species (ROS), nitric oxide (NO), tumor necrosis factor-α (TNF-α) and interleukin (IL)-6. Moreover, signal transduction experiments showed that S-CYP induced the activation of mitogen-activated protein kinase (MAPK) and nuclear factor-κB (NF-κB) pathways through toll-like receptor 4 (TLR4), dramatically increasing p-ERK, p-JNK and p-p38 proteins. Meanwhile, immunofluorescence results showed that S-CYP could significantly promote the entry of NF-κB p65 into the nucleus, which is essential for triggering the NF-κB pathway. Furthermore, blocking antibody experiments revealed that specific inhibitors of TLR4, MAPKs, and NF-κB suppressed the generation of TNF-α and IL-6 in RAW 264.7 cells. These findings suggested that both CYP and S-CYP could be used as immunomodulatory agents and may have potential application prospects in the food and pharmaceutical industries.