Defining the components of an HIV immunogen that could induce effective CD8+ T cell responses is critical to vaccine development. We addressed this question by investigating the viral targets of CD8+ T cells that potently inhibit HIV replication in vitro, as this is highly predictive of virus control in vivo. We observed broad and potent ex vivo CD8+ T cell-mediated viral inhibitory activity against a panel of HIV isolates among viremic controllers (VC, viral loads <5000 copies/ml), in contrast to unselected HIV-infected HIV Vaccine trials Network (HVTN) participants. Viral inhibition of clade-matched HIV isolates was strongly correlated with the frequency of CD8+ T cells targeting vulnerable regions within Gag, Pol, Nef and Vif that had been identified in an independent study of nearly 1000 chronically infected individuals. These vulnerable and so-called "beneficial" regions were of low entropy overall, yet several were not predicted by stringent conservation algorithms. Consistent with this, stronger inhibition of clade-matched than mismatched viruses was observed in the majority of subjects, indicating better targeting of clade-specific than conserved epitopes. The magnitude of CD8+ T cell responses to beneficial regions, together with viral entropy and HLA class I genotype, explained up to 59% of the variation in viral inhibitory activity, with magnitude of the T cell response making the strongest unique contribution. However, beneficial regions were infrequently targeted by CD8+ T cells elicited by vaccines encoding full-length HIV proteins, when the latter were administered to healthy volunteers and HIV-positive ART-treated subjects, suggesting that immunodominance hierarchies undermine effective anti-HIV CD8+ T cell responses. Taken together, our data support HIV immunogen design that is based on systematic selection of empirically defined vulnerable regions within the viral proteome, with exclusion of immunodominant decoy epitopes that are irrelevant for HIV control.