Reading ability and print exposure: item response theory analysis of the author recognition test

Abstract

In the author recognition test (ART), participants are presented with a series of names and foils and are asked to indicate which ones they recognize as authors. The test is a strong predictor of reading skill, and this predictive ability is generally explained as occurring because author knowledge is likely acquired through reading or other forms of print exposure. In this large-scale study (1,012 college student participants), we used item response theory (IRT) to analyze item (author) characteristics in order to facilitate identification of the determinants of item difficulty, provide a basis for further test development, and optimize scoring of the ART. Factor analysis suggested a potential two-factor structure of the ART, differentiating between literary and popular authors. Effective and ineffective author names were identified so as to facilitate future revisions of the ART. Analyses showed that the ART is a highly significant predictor of the time spent encoding words, as measured using eyetracking during reading. The relationship between the ART and time spent reading provided a basis for implementing a higher penalty for selecting foils, rather than the standard method of ART scoring (names selected minus foils selected). The findings provide novel support for the view that the ART is a valid indicator of reading volume. Furthermore, they show that frequency data can be used to select items of appropriate difficulty, and that frequency data from corpora based on particular time periods and types of texts may allow adaptations of the test for different populations.

Keywords: Author recognition test; Eye movements; Item response theory; Print exposure; Reading.

Figure 1

Item Characteristic Curves of effective ART items (left) and items with low discrimination (right) using 65-ART Item Parameters. Note that the 50-Item ART Parameters are very similar, but T.C Boyle and Jean M. Auel are omitted from the shorter test

Figure 2

Test Information Function for 50-item ART.

Figure 3

Relationship between word frequency and mean gaze duration as a function of author knowledge: Words were grouped by log frequency into 20 equally-sized bins. Consistent with the ICCs shown in Figure 1, only 1.3% of participants selected Kurt Vonnegut but not J. R. R. Tolkien.

Figure 4

The Relationship between Item Difficulty and Log Frequency of Author Name: Item difficulty of author name (b-parameter) and Log10 transformed frequency of author name. N = 61. Based on the 65-item version of the test in order to show items not assigned a factor. Brian Herbert, Dick Francis, Jean M. Auel, and Jane Smiley are not included due to b-parameters greater than 10.

Figure 5

Difference in author log frequency from Google Ngram Books Viewer (1999-2008; 1979-1988) by difference in logit transformed mean selection rate per author. Difference between current study and Stanovich studies (left) and difference between Acheson study and Stanovich studies (right).