Predictive utility of tumor-infiltrating lymphocytes (TILs) in HER2-positive breast cancer patients receiving neoadjuvant chemotherapy (NAC) with concurrent trastuzumab remains unclear. We examined TILs grades of pretreatment cancer tissue specimens and residual tumors after NAC with trastuzumab and determined the predictive utility of the TILs grade in pathological complete response (pCR) and the prognostic power of TILs in HER2-positive breast cancer. This cohort study included 128 HER2-positive breast cancer who received NAC with trastuzumab. TILs grading of the tumor stroma in pretreatment biopsy specimens and residual tumors after NAC with trastuzumab was categorized as low, intermediate, and high based on the criteria of the International Working Group. In current study, the pCR rate was 64.8%, and the Relapse-free survival (RFS) was significantly worse in the non-pCR group than in the pCR group. The pCR rate correlated with the TILs grade in pretreatment tumors. In 45 non-pCR patients, TILs grade was higher in the residual tumors than in the pretreatment tumors. The RFS was significantly better in residual tumors with high TILs grade than those with low TILs grade (p = 0.033). In conclusion, assessment of the TILs grade in residual tumors after NAC with trastuzumab might be necessary to determine patients with good prognosis among those who do not achieve pCR.