Oropharyngeal squamous cell carcinoma (OPSCC) incidence is increasing at a nearly epidemic rate, largely driven by the human papillomavirus (HPV). Despite the generally favorable clinical outcomes of patients with HPV driven (HPV+) OPSCC, a significant subset of HPV tumors associated with tobacco exposure have diminished treatment response and worse survival. The tumor immune microenvironment (TIME) has been shown to be a critical driver of treatment response and oncologic outcomes in OPSCC generally and HPV+ OPSCC more specifically. However, the impact of tobacco exposure on the TIME in OPSCC patients remains unclear. We analyzed the relationship between TIME, tobacco exposure and clinical outcomes in OPSCC patients (n = 143) with extensive tobacco exposure (median pack-years = 40). P16 overexpression, a surrogate marker of HPV association, was a strong predictor of relapse-free (RFS) and overall survival (OS) (p < 0.001, p < 0.001 respectively) regardless of tobacco exposure and associated strongly with differential infiltration of the tumor by both CD3 and CD8 lymphocytes measured via immunohistochemistry (p < 001, p < 0.001 respectively). CD3 and CD8 infiltration was a strong predictor of RFS and OS and associated strongly with disease stage (AJCC 8th Edition Staging Manual). Tobacco exposure correlated significantly (p < 0.001) with decreased CD8 infiltration in p16+ OPSCC tumors. Our findings demonstrate that the HPV+ OPSCC clinical outcomes are strongly correlated with the TIME, which is potentially modulated by tobacco exposure. Immunomodulatory strategies targeting this disease in smokers must take into consideration the potential modifying effects of tobacco exposure on treatment effectiveness and clinical outcomes.