The eradication of HIV-1 from infected individuals remains the ultimate goal of all anti-HIV therapeutics. Although highly active antiretroviral therapy (HAART) has led to a profound decrease in morbidity and mortality in infected people by suppressing HIV replication, the virus continues to evolve slowly during therapy even when patients achieve below detectable levels of HIV in plasma. HIV-1 persists in latently infected memory CD4+ T cells and there is minimal decay of HIV in this compartment despite prolonged HAART. Various other reservoirs and sanctuary sites harboring HIV are also established in vivo during antiretroviral therapy. Collectively these sites represent a major impediment to the eradication of HIV-1. This review presents a detailed overview of various reservoir sites in vivo, and discusses their impact on the success and failure of HAART for HIV patients. In addition, it addresses the effect of sub-optimal drug concentrations on reservoir establishment and outlines future therapeutic strategies to counteract these reservoirs and sanctuaries.