A modified structure of single-polarization single-mode (SPSM) photonic crystal fiber (PCF) with different background materials is presented and analyzed by using the full-vector finite-element method. Simulation results confirmed that the proposed PCF can realize low-loss SPSM on three wavebands with the same structure and different background materials. The wavebands are 1.46-1.60 μm for silica-based fiber, 1.97-2.3 μm for lead silicate glass fiber, and 3.16-3.58 μm for chalcogenide glass fiber. For three PCFs with different background materials, only the slow-axis mode exists and the confinement loss is less than 100 dB/m in the SPSM wavebands.