The mechanisms underlying HIV-1 control by protective HLA class I alleles are not fully understood and could involve selection of escape mutations in functionally important Gag epitopes resulting in fitness costs. This study was undertaken to investigate, at the population level, the impact of HLA-mediated immune pressure in Gag on viral fitness and its influence on HIV-1 pathogenesis. Replication capacities of 406 recombinant viruses encoding plasma-derived Gag-protease from patients chronically infected with HIV-1 subtype C were assayed in an HIV-1-inducible green fluorescent protein reporter cell line. Viral replication capacities varied significantly with respect to the specific HLA-B alleles expressed by the patient, and protective HLA-B alleles, most notably HLA-B81, were associated with lower replication capacities. HLA-associated mutations at low-entropy sites, especially the HLA-B81-associated 186S mutation in the TL9 epitope, were associated with lower replication capacities. Most mutations linked to alterations in replication capacity in the conserved p24 region decreased replication capacity, while most in the highly variable p17 region increased replication capacity. Replication capacity also correlated positively with baseline viral load and negatively with baseline CD4 count but did not correlate with the subsequent rate of CD4 decline. In conclusion, there is evidence that protective HLA alleles, in particular HLA-B81, significantly influence Gag-protease function by driving sequence changes in Gag and that conserved regions of Gag should be included in a vaccine aiming to drive HIV-1 toward a less fit state. However, the long-term clinical benefit of immune-driven fitness costs is uncertain given the lack of correlation with longitudinal markers of disease progression.