One year in the coronavirus disease 2019 (COVID-19) pandemic, the first vaccines are being rolled out under emergency use authorizations. It is of great concern that newly emerging variants of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) can escape antibody-mediated protection induced by previous infection or vaccination through mutations in the spike protein. The glutamate (E) to Lysine (K) substitution at position 484 (E484K) in the receptor binding domain (RBD) of the spike protein is present in the rapidly spreading variants of concern belonging to the B.1.351 and P.1 lineages. We performed in vitro microneutralization assays with both the USA-WA1/2020 virus and a recombinant (r)SARS-CoV-2 virus that is identical to USA-WA1/2020 except for the E484K mutation introduced in the spike RBD. We selected 34 sera from study participants based on their SARS-CoV-2 spike ELISA antibody titer (negative [N=4] versus weak [N=8], moderate [N=11] or strong positive [N=11]). In addition, we included sera from five individuals who received two doses of the Pfizer SARS-CoV-2 vaccine BNT162b2. Serum neutralization efficiency was lower against the E484K rSARS-CoV-2 (vaccination samples: 3.4 fold; convalescent low IgG: 2.4 fold, moderate IgG: 4.2 fold and high IgG: 2.6 fold) compared to USA-WA1/2020. For some of the convalescent donor sera with low or moderate IgG against the SARS-CoV-2 spike, the drop in neutralization efficiency resulted in neutralization ID50 values similar to negative control samples, with low or even absence of neutralization of the E484K rSARS-CoV-2. However, human sera with high neutralization titers against the USA-WA1/2020 strain were still able to neutralize the E484K rSARS-CoV-2. Therefore, it is important to aim for the highest titers possible induced by vaccination to enhance protection against newly emerging SARS-CoV-2 variants. Two vaccine doses may be needed for induction of high antibody titers against SARS-CoV-2. Postponing the second vaccination is suggested by some public health authorities in order to provide more individuals with a primer vaccination. Our data suggests that this may leave vaccinees less protected against newly emerging variants.