Disease-associated trinucleotide repeats form secondary DNA structures that interfere with replication and repair. Replication has been implicated as a mechanism that can cause repeat expansions and contractions. However, because structure-forming repeats are also replication barriers, it has been unclear whether the instability occurs due to slippage during normal replication progression through the repeat, slippage or misalignment at a replication stall caused by the repeat, or during subsequent replication of the repeat by a restarted fork that has altered properties. In this study, we have specifically addressed the fidelity of a restarted fork as it replicates through a CAG/CTG repeat tract and its effect on repeat instability. To do this, we used a well-characterized site-specific replication fork barrier (RFB) system in fission yeast that creates an inducible and highly efficient stall that is known to restart by recombination-dependent replication (RDR), in combination with long CAG repeat tracts inserted at various distances and orientations with respect to the RFB. We find that replication by the restarted fork exhibits low fidelity through repeat sequences placed 2-7 kb from the RFB, exhibiting elevated levels of Rad52- and Rad8ScRad5/HsHLTF-dependent instability. CAG expansions and contractions are not elevated to the same degree when the tract is just in front or behind the barrier, suggesting that the long-traveling Polδ-Polδ restarted fork, rather than fork reversal or initial D-loop synthesis through the repeat during stalling and restart, is the greatest source of repeat instability. The switch in replication direction that occurs due to replication from a converging fork while the stalled fork is held at the barrier is also a significant contributor to the repeat instability profile. Our results shed light on a long-standing question of how fork stalling and RDR contribute to expansions and contractions of structure-forming trinucleotide repeats, and reveal that tolerance to replication stress by fork restart comes at the cost of increased instability of repetitive sequences.