The chemokine CXCL12 and its G protein-coupled receptor (GPCR) CXCR4 are high-priority clinical targets because of their involvement in metastatic cancers (also implicated in autoimmune disease and cardiovascular disease). Because chemokines interact with two distinct sites to bind and activate their receptors, both the GPCRs and chemokines are potential targets for small molecule inhibition. A number of chemokines have been validated as targets for drug development, but virtually all drug discovery efforts focus on the GPCRs. However, all CXCR4 receptor antagonists with the exception of MSX-122 have failed in clinical trials due to unmanageable toxicities, emphasizing the need for alternative strategies to interfere with CXCL12/CXCR4-guided metastatic homing. Although targeting the relatively featureless surface of CXCL12 was presumed to be challenging, focusing efforts at the sulfotyrosine (sY) binding pockets proved successful for procuring initial hits. Using a hybrid structure-based in silico/NMR screening strategy, we recently identified a ligand that occludes the receptor recognition site. From this initial hit, we designed a small fragment library containing only nine tetrazole derivatives using a fragment-based and bioisostere approach to target the sY binding sites of CXCL12. Compound binding modes and affinities were studied by 2D NMR spectroscopy, X-ray crystallography, molecular docking and cell-based functional assays. Our results demonstrate that the sY binding sites are conducive to the development of high affinity inhibitors with better ligand efficiency (LE) than typical protein-protein interaction inhibitors (LE ≤ 0.24). Our novel tetrazole-based fragment 18 was identified to bind the sY21 site with a K(d) of 24 μM (LE = 0.30). Optimization of 18 yielded compound 25 which specifically inhibits CXCL12-induced migration with an improvement in potency over the initial hit 9. The fragment from this library that exhibited the highest affinity and ligand efficiency (11: K(d) = 13 μM, LE = 0.33) may serve as a starting point for development of inhibitors targeting the sY12 site.