The paper addresses coupling of magnetic nanoparticles (MNPs) with the polymer matrix of temperature-sensitive microgels and their response to magnetic fields. Therefore, CoFe2O4@CA (CA = citric acid) NPs are embedded within N-isopropylacrylamid (NIPAM) based microgels. The volume phase transition (VPT) of the magnetic microgels and the respective pure microgels is studied by dynamic light scattering and electrophoretic mobility measurements. The interaction between MNPs and microgel network is studied via magnetometry and AC-susceptometry using a superconducting quantum interference device (SQUID). The data show a significant change of the magnetic properties by crossing the VPT temperature (VPTT). The change is related to the increased confinement of the MNP due to the shrinking of the microgels. Modifying the microgel with hydrophobic allyl mercaptan (AM) affects the swelling ability and the magnetic response, i.e. the coupling of MNPs with the polymer matrix. Modeling the AC-susceptibility data results in an effective size distribution. This distribution represents the varying degree of constraint in MNP rotation and motion by the microgel network. These findings help to understand the interaction between MNPs and the microgel matrix to design multi responsive systems with tunable particle matrix coupling strength for future applications.