Simultaneous electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) has become a widely used application in spite of EEG perturbations due to electromagnetic interference in the MR environment. The most prominent and disturbing artifacts in the EEG are caused by the alternating magnetic fields (gradients) of the MR scanner. Different methods for gradient artifact correction have been developed. Here we propose an approach for the systematic evaluation and comparison of these gradient artifact correction methods. Exemplarily, we evaluate different algorithms all based on artifact template subtraction--the currently most established means of gradient artifact removal. We introduce indices for the degree of gradient artifact reduction and physiological signal preservation. The combination of both indices was used as a measure for the overall performance of gradient artifact removal and was shown to be useful in identifying problems during artifact removal. We demonstrate that the evaluation as proposed here allows to reveal frequency-band specific performance differences among the algorithms. This emphasizes the importance of carefully selecting the artifact correction method appropriate for the respective case.