Development of tool molecules that inhibit Jumonji demethylases allows for the investigation of cancer-associated transcription. While scaffolds such as 2,4-pyridinedicarboxylic acid (2,4-PDCA) are potent inhibitors, they exhibit limited selectivity. To discover new inhibitors for the KDM4 demethylases, enzymes overexpressed in several cancers, we docked a library of 600,000 fragments into the high-resolution structure of KDM4A. Among the most interesting chemotypes were the 5-aminosalicylates, which docked in two distinct but overlapping orientations. Docking poses informed the design of covalently linked fragment compounds, which were further derivatized. This combined approach improved affinity by ∼ 3 log-orders to yield compound 35 (Ki = 43 nM). Several hybrid inhibitors were selective for KDM4C over the related enzymes FIH, KDM2A, and KDM6B while lacking selectivity against the KDM3 and KDM5 subfamilies. Cocrystal structures corroborated the docking predictions. This study extends the use of structure-based docking from fragment discovery to fragment linking optimization, yielding novel KDM4 inhibitors.