Homolog pairing and crossing over during meiosis I prophase is required for accurate chromosome segregation to form euploid gametes. The repair of Spo11-induced double-strand breaks (DSB) using a homologous chromosome template is a major driver of pairing in many species, including fungi, plants, and mammals. Inappropriate pairing and crossing over at ectopic loci can lead to chromosome rearrangements and aneuploidy. How (or if) inappropriate ectopic interactions are disrupted in favor of allelic interactions is not clear. Here we used an in vivo "collision" assay in budding yeast to test the contributions of cohesion and the organization and motion of chromosomes in the nucleus on promoting or antagonizing interactions between allelic and ectopic loci at interstitial chromosome sites. We found that deletion of the cohesin subunit Rec8, but not other chromosome axis proteins (e.g. Red1, Hop1, or Mek1), caused an increase in homolog-nonspecific chromosome interaction, even in the absence of Spo11. This effect was partially suppressed by expression of the mitotic cohesin paralog Scc1/Mdc1, implicating Rec8's role in cohesion rather than axis integrity in preventing nonspecific chromosome interactions. Disruption of telomere-led motion by treating cells with the actin polymerization inhibitor Latrunculin B (Lat B) elevated nonspecific collisions in rec8Δ spo11Δ. Next, using a visual homolog-pairing assay, we found that the delay in homolog pairing in mutants defective for telomere-led chromosome motion (ndj1Δ or csm4Δ) is enhanced in Lat B-treated cells, implicating actin in more than one process promoting homolog juxtaposition. We suggest that multiple, independent contributions of actin, cohesin, and telomere function are integrated to promote stable homolog-specific interactions and to destabilize weak nonspecific interactions by modulating the elastic spring-like properties of chromosomes.