The many protein processing reactions of the ATP-hydrolyzing Hsp70s are regulated by J cochaperones, which contain J domains that stimulate Hsp70 ATPase activity and accessory domains that present protein substrates to Hsp70s. We report the structure of a J domain complexed with a J responsive portion of a mammalian Hsp70. The J domain activates ATPase activity by directing the linker that connects the Hsp70 nucleotide binding domain (NBD) and substrate binding domain (SBD) toward a hydrophobic patch on the NBD surface. Binding of the J domain to Hsp70 displaces the SBD from the NBD, which may allow the SBD flexibility to capture diverse substrates. Unlike prokaryotic Hsp70, the SBD and NBD of the mammalian chaperone interact in the ADP state. Thus, although both nucleotides and J cochaperones modulate Hsp70 NBD:linker and NBD:SBD interactions, the intrinsic persistence of those interactions differs in different Hsp70s and this may optimize their activities for different cellular roles.