The development of safe and effective vaccines to prevent SARS-CoV-2 infections remains an urgent priority worldwide. We have used a recombinant vesicular stomatitis virus (rVSV)-based prime-boost immunization strategy to develop an effective COVID-19 vaccine candidate. We have constructed VSV genomes carrying exogenous genes resulting in the production of avirulent rVSV carrying the full-length spike protein (SF), the S1 subunit, or the receptor-binding domain (RBD) plus envelope (E) protein of SARS-CoV-2. Adding the honeybee melittin signal peptide (msp) to the N-terminus enhanced the protein expression, and adding the VSV G protein transmembrane domain and the cytoplasmic tail (Gtc) enhanced protein incorporation into pseudotype VSV. All rVSVs expressed three different forms of SARS-CoV-2 spike proteins, but chimeras with VSV-Gtc demonstrated the highest rVSV-associated expression. In immunized mice, rVSV with chimeric S protein-Gtc derivatives induced the highest level of potent neutralizing antibodies and T cell responses, and rVSV harboring the full-length msp-SF-Gtc proved to be the superior immunogen. More importantly, rVSV-msp-SF-Gtc vaccinated animals were completely protected from a subsequent SARS-CoV-2 challenge. Overall, we have developed an efficient strategy to induce a protective response in SARS-CoV-2 challenged immunized mice. Vaccination with our rVSV-based vector may be an effective solution in the global fight against COVID-19.