The detection and identification of epidermal growth factor receptor 2 (HER2)-positive breast cancer cells is crucial for the clinic therapy of breast cancer. For the aim of the detection, a novel surface-enhanced Raman scattering (SERS) probe for distinguishing breast cancers at different HER2 statuses is reported in this paper. In such a probe, anti-HER2 antibody-conjugated silver nanoparticles have been synthesized for specific targeting of HER2-positive breast cancer cells. More importantly, different from the previously reported SERS probe for targeting cancer cells, p-mercaptobenzoic acid is utilized as both the Raman reporter and the conjugation agent for attaching antibody molecules, which leads to a much simplified structure. For investigating the ability of such a probe to distinguish breast cancer cells, SKBR3 and MCF7 cells were chosen as two model systems, which are HER2-positive- and HER2-negative-expressing cells, respectively. The experimental results reveal that SKBR3 cells exhibit much stronger SERS signals than MCF7 cells, indicating that the probe could be utilized to distinguish breast cancer cells at different HER2 statuses. This kind of SERS probe holds a potential for a direct detection of living breast cancer cells with the advantages of easy fabrication, high SERS sensitivity, and biocompatibility.