Who's afraid of response bias?

Abstract

Response bias (or criterion) contamination is insidious in studies of consciousness: that observers report they do not see a stimulus may not mean they have absolutely no subjective experience; they may be giving such reports in relative terms in the context of other stimuli. Bias-free signal detection theoretic measures provide an excellent method for avoiding response bias confounds, and many researchers correctly adopt this approach. However, here we discuss how a fixation on avoiding criterion effects can also be misleading and detrimental to fruitful inquiry. In a recent paper, Balsdon and Azzopardi (Absolute and relative blindsight. Consciousness and Cognition 2015; 32:79-91.) claimed that contamination by response bias led to flawed findings in a previous report of "relative blindsight". We argue that their criticisms are unfounded. They mistakenly assumed that others were trying (and failing) to apply their preferred methods to remove bias, when there was no such intention. They also dismissed meaningful findings because of their dependence on criterion, but such dismissal is problematic: many real effects necessarily depend on criterion. Unfortunately, these issues are technically tedious, and we discuss how they may have confused others to misapply psychophysical metrics and to draw questionable conclusions about the nature of TMS (transcranial magnetic stimulation)-induced blindsight. We conclude by discussing the conceptual importance of criterion effects in studies of conscious awareness: we need to treat them carefully, but not to avoid them without thinking.

Figure 1.

Illustration of a relative blindsight effect. For two conditions, Performance (indexed by percent correct, d’, or some other measure of perceptual sensitivity/capacity) is equivalent, but Awareness or Confidence ratings differ.

Figure 2.

Illustration of hypothetical weight distributions of USA and Hong Kong citizens (top panel), and effects of different selections for a criterion to classify individuals as “overweight” (lower three panels). If we choose a criterion that is either too liberal (A) or too conservative (C), we will likely see no differences in the percentage of people classified as “overweight” between the USA and Hong Kong: nearly everybody or practically nobody will be classified as “overweight” in these two scenarios, respectively. But this does not mean the differences are not there; we must choose a reasonable criterion (B) in order to detect the differences. Note that Lau & Passingham (2006) encouraged subjects to use such a reasonable criterion for “awareness” when they reported a difference in “awareness” ratings across two conditions (e.g. Fig. 1B). However, their lack of finding any differences in objective performance across those two conditions (e.g. Fig. 1A) does not indicate that they chose an unreasonable criterion for performance: classifying trials as “correct” vs. “incorrect” does not depend on subjective criterion selection, and subjects did not display ceiling (100% correct) or floor (50% correct) performance. The relevant issue of whether response bias may play a role in performance measured here is discussed in the main text.