The stress-inducible mammalian heat shock protein 70 (HSP70) and its bacterial orthologue DnaK are highly conserved nucleotide binding molecular chaperones. They represent critical regulators of cellular proteostasis, especially during conditions of enhanced stress. Cancer cells rely on HSP70 for survival, and this chaperone represents an attractive new therapeutic target. We have used a structure-activity approach and biophysical methods to characterize a class of inhibitors that bind to a unique allosteric site within the C-terminus of HSP70 and DnaK. Data from X-ray crystallography together with isothermal titration calorimetry, mutagenesis, and cell-based assays indicate that these inhibitors bind to a previously unappreciated allosteric pocket formed within the non-ATP-bound protein state. Moreover, binding of inhibitor alters the local protein conformation, resulting in reduced chaperone-client interactions and impairment of proteostasis. Our findings thereby provide a new chemical scaffold and target platform for both HSP70 and DnaK; these will be important tools with which to interrogate chaperone function and to aid ongoing efforts to optimize potency and efficacy in developing modulators of these chaperones for therapeutic use.