The Stroop task has a long-standing history in psychological research and diagnostics, and many variants have emerged. Computerized versions have recently gained popularity because of their applicability in brain-imaging studies. It remains unclear, however, whether computerized versions are content valid with reference to the original task. We compare the performance in the original task with two computerized versions. All three versions show high test-retest reliability and are able to elicit interference effects, but to varying degrees. However, performances in the computerized versions and in the original task do not correlate. The transition from oral to manual response and from listed to single stimulus presentation seems not only to diminish the interference effect, but also to alter its nature in such a way that it no longer looks genuinely "Stroop-like". These findings have important clinical implications on the use and interpretation of computerized Stroop tasks in children and adults.