Magnetic Particle Imaging (MPI) is a promising tracer imaging modality that employs a kidney-safe contrast agent and does not use ionizing radiation. MPI already shows high contrast and sensitivity in small animal imaging, with great potential for many clinical applications, including angiography, cancer detection, inflammation imaging, and treatment monitoring. Currently, almost all clinically relevant imaging techniques can be modeled as systems with linearity and shift invariance (LSI), characteristics crucial for quantification and diagnostic utility. In theory, MPI has been proven to be LSI. However, in practice, high-pass filters designed to remove unavoidable direct feedthrough interference also remove information crucial to ensuring LSI in MPI scans. In this work, we present a complete theoretical and experimental description of the image artifacts from filtering. We then propose and validate a robust algorithm to completely restore the lost information for the x-space MPI method. We provide the theoretical, simulated, and experimental proof that our algorithm indeed restores the LSI properties of MPI.