Multiplication is thought to be primarily solved via direct retrieval from memory. Two of the main factors known to influence the retrieval of multiplication facts are problem size and interference. Because these factors are often intertwined, we sought to investigate the unique influences of problem size and interference on both performance and neural responses during multiplication fact retrieval in healthy adults. Behavioral results showed that both problem size and interference explained separate unique portions of RT variance, but with significantly stronger contribution from problem size, which contrasts with previous work in children. Whole-brain fMRI results relying on a paradigm that isolated multiplication fact retrieval from response selection showed highly overlapping brain areas parametrically modulated by both problem size and interference in a large network of frontal, parietal, and subcortical brain areas. Subsequent analysis within these regions revealed problem size to be the stronger and more consistent "unique" modulating factor in overlapping regions as well as those that appeared to respond only to problem size or interference at the whole-brain level, thus underscoring the need to look beyond anatomical overlap using arbitrary thresholds. Additional unique contributions of interference (beyond problem size) were identified in right angular gyrus and subcortical regions associated with procedural processing. Together, our results suggest that problem size, relative to interference, tends to be the more dominant factor in driving behavioral and neural responses during multiplication fact retrieval in adults. Nevertheless, unique contributions of both factors demonstrate the importance of considering the overlapping and unique contributions of each in explaining the cognitive and neural bases of mental multiplication.