This study investigates the fate of information that was previously stored in visual working memory but that is no longer needed. Previous research has found inconsistent results, with some showing effective release of irrelevant information and others showing proactive interference. Using change detection tasks of colors or shapes, we show that participants tend to falsely classify a changed item as "no change" if it matches one of the memory items on the preceding trial. The interference is spatially specific: Memory for the preceding trial interferes more if it matches the feature value and the location of a test item than if it does not. Interference results from retaining information in visual working memory, since it is absent when items on the preceding trials are passively viewed, or are attended but not memorized. We conclude that people cannot fully eliminate unwanted visual information from current working memory tasks.