NOD2 is essential for antimicrobial innate immunity and tissue homeostasis, but require tight regulation to avert pathology. A focal point of NOD2 signaling is RIP2, which upon polyubiquitination nucleates the NOD2:RIP2 complex, enabling signaling events leading to inflammation, yet the precise nature and the regulation of the polyubiquitins coordinating this process remain unclear. Here we show that NOD2 signaling involves conjugation of RIP2 with lysine 63 (K63), K48 and M1 polyubiquitin chains, as well as with non-canonical K27 chains. In addition, we identify MYSM1 as a proximal deubiquitinase that attenuates NOD2:RIP2 complex assembly by selectively removing the K63, K27 and M1 chains, but sparing the K48 chains. Consequently, MYSM1 deficient mice have unrestrained NOD2-mediated peritonitis, systemic inflammation and liver injury. This study provides a complete overview of the polyubiquitins in NOD2:RIP2 signaling and reveal MYSM1 as a central negative regulator restricting these polyubiquitins to prevent excessive inflammation.