Low-inertia vortex formation and pattern selection are examined for axisymmetric Taylor-Couette flow with spatially modulated cylinders. The forcing is arbitrary but remains periodic. The modulation amplitude is assumed to be small, and a regular perturbation expansion is used to determine the flow field at small to moderately large Taylor numbers (below the critical threshold). It is found that the presence of a weak modulation leads unambiguously to the emergence of steady Taylor-vortex flow even at vanishingly small Taylor number. This situation is closely reminiscent of the effect of end plates, and the consequent onset of imperfect bifurcation. The vortex structure is found to have the same periodicity as the forcing when only one of the cylinders is modulated, or when the modulations are commensurate. For incommensurate modulations, the vortex pattern is quasiperiodic, with regions of almost purely azimuthal flow. When the counter-rotation speed of the outer cylinder increases, the original vortices are gradually replaced by new ones that end up spanning the entire gap width, and in turn break up into two vortices resulting in two rows of vortices commensurate with each cylinder modulation. It is also shown that, for any modulation amplitude, the forcing wave number that generates the most intense vortex flow for a given Taylor number varies monotonically with Ta, but always reaches the critical value predicted by linear stability analysis for straight cylinders, regardless of which cylinder is modulated.