The ubiquitously expressed protein tyrosine phosphatase SHP2 is required for signaling downstream of receptor tyrosine kinases (RTKs) and plays a role in regulating many cellular processes. Genetic knockdown and pharmacological inhibition of SHP2 suppresses RAS/MAPK signaling and inhibit the proliferation of RTK-driven cancer cell lines. Here, we describe the first reported fragment-to-lead campaign against SHP2, where X-ray crystallography and biophysical techniques were used to identify fragments binding to multiple sites on SHP2. Structure-guided optimization, including several computational methods, led to the discovery of two structurally distinct series of SHP2 inhibitors binding to the previously reported allosteric tunnel binding site (Tunnel Site). One of these series was advanced to a low-nanomolar lead that inhibited tumor growth when dosed orally to mice bearing HCC827 xenografts. Furthermore, a third series of SHP2 inhibitors was discovered binding to a previously unreported site, lying at the interface of the C-terminal SH2 and catalytic domains.