Event-related brain potentials (ERPs) were used to examine the electrophysiological effect of working memory (WM) load on involuntary attention caused by a task-irrelevant sound in an auditory-visual distraction paradigm. The different WM loads were manipulated by requiring subjects to remember the order of either three digits (low-load condition) or seven digits (high-load condition), and the irrelevant auditory stimuli consisted of repetitive standard sounds (80%) and environmental novel sounds (20%). We found that the difference waves (novel-minus-standard) showed significant MMN and Novelty-P3 components in the two WM load conditions. The amplitude of MMN increased with increasing the WM load, which indicated a more engaged change detection process under high-load condition. Then, the amplitude of Novelty-P3 was attenuated under high-load condition, which indicated a much reduced involuntary orienting of attention to novel sounds when increasing the WM load. These results indicated the top-down control of involuntary attention might be mainly active at the early change detection stage and the control of the later involuntary orienting of attention might be passive.