"Which box should I check?": examining standard check box approaches to measuring race and ethnicity

Abstract

Objective: This study examined methodological concerns with standard approaches to measuring race and ethnicity using the federally defined race and ethnicity categories, as utilized in National Institutes of Health (NIH) funded research.

Data sources/study setting: Surveys were administered to 219 economically disadvantaged, racially and ethnically diverse participants at Boston Women Infants and Children (WIC) clinics during 2010.

Study design: We examined missingness and misclassification in responses to the closed-ended NIH measure of race and ethnicity compared with open-ended measures of self-identified race and ethnicity.

Principal findings: Rates of missingness were 26 and 43 percent for NIH race and ethnicity items, respectively, compared with 11 and 18 percent for open-ended responses. NIH race responses matched racial self-identification in only 44 percent of cases. Missingness and misclassification were disproportionately higher for self-identified Latina(o)s, African-Americans, and Cape Verdeans. Race, but not ethnicity, was more often missing for immigrant versus mainland U.S.-born respondents. Results also indicated that ethnicity for Hispanic/Latina(o)s is more complex than captured in this measure.

Conclusions: The NIH's current race and ethnicity measure demonstrated poor differentiation of race and ethnicity, restricted response options, and lack of an inclusive ethnicity question. Separating race and ethnicity and providing respondents with adequate flexibility to identify themselves both racially and ethnically may improve valid operationalization.

Keywords: Measurement of race and ethnicity; National Institutes of Health (NIH) race and ethnicity reporting; health disparities research; racial and ethnic self-identification.

Figure 1

Aggregated Self-Identified Race Responses within Each NIH Race Category Note. Self-identified race responses were aggregated into the following categories: “Black” includes those who self-identified as Black (n = 46), Black American (n = 1), or African/Black and Black/African (5), Hispanic Black (n = 4), and Cape Verdean-Black (n = 2) but excludes those who identify solely as African; “African American” includes AfAm (n = 25), AfAm/West Indian (n = 1), Caribbean/AfAm (n = 1), and AfAm/Black (n = 10) but excludes those who identified solely as African; “Cape Verdean” includes Cape Verdean (n = 23), American/Cape Verdean (n = 2); “Hispanic or Latino” includes Hispanic (n = 32), Hispanic/Latino/a (1), Latino/a (n = 7), Hispanic/Puerto Rican (n = 2), Puerto Rican (n = 5), Spanish/Puerto Rican (n = 1), Hispanic (light skin, light + dark hair, green eyes) (n = 1), and Brown/Hispanic (n = 1); “White” includes White (n = 6), Caucasian (n = 1), and Caucasian/Eastern European (n = 1). “Other groups:” Aggregate categories that had fewer than 8 participants included multiracial (e.g., Mestizo, mixed, two or more, Black/White, Triguena, Black/German/Italian, Portuguese/Irish) (n = 8), African (n = 1), and Spanish or Spanish/American (n = 6). Pie graph sections outlined in bold represent agreement between NIH race and self-identified race.

Figure 2

NIH Race Responses within Each Aggregate Self-Identified Race Category Note. Bars outlined in bold represent agreement between NIH race and self-identified race. Included above are any response categories that had more than five respondents.

Figure 3

Aggregated Self-Identified Ethnicity Responses within Each NIH Ethnicity Category Note. Self-identified ethnicity responses were aggregated into the following categories: “African American” includes those who self-identified as African American (n = 61) and Black/African American (n = 1); “ Hispanic or Latino/a” includes those who self-identified as Catholic Hispanic (1), Hispanic (14), Latin (1), Latin-American (1), and Latino/a (2); “Puerto Rican” includes those who specifically identified as Puerto Rican (n = 17); “Dominican” includes those who self-identified as Dominican (n = 13), Hispanic/ Dominican (n = 1), and American-Dominican (n = 1); “Cape Verdean” includes those who self-identified as Cape Verdean (n = 10), Cape Verdean/Portuguese (n = 2), and American/Cape Verdean (n = 1).“Caribbean” includes Caribbean (n = 2), Jamaican (n = 1), parents Trinidad and Tobago (n = 1) and West Indian (n = 1). “Pan-African” includes those who self-identified as African (n = 6), African/Black (n = 1), Yoruba/African (n = 1); “European American” includes those who self-identified as Caucasian (n = 1), English (n = 1), English/Irish (n = 1), Irish (n = 2), White/Spanish (n = 1), Spanish (n = 1); “Haitian” includes Haitian (n = 5) and African Haitian American (n = 1). “Missing” includes all those who left the item blank (n = 40). “Not aggregated:” Responses not included in aggregate categories include (n = 1 unless otherwise indicated): food, N/A, other, both, Catholic, all of them, none (n = 2), American (n = 3). Responses of Black (n = 7) and White (n = 2) were also not included in an aggregated category as it was unclear to which aggregated category they should be assigned. Mixed ethnicity (n = 5) such as African American/European American or Cape Verdean/Middle Eastern were not included in any category. Pie graph sections outlined in bold represent agreement between NIH ethnicity and self-identified ethnicity.