Standards for Instrument Migration When Implementing Paper Patient-Reported Outcome Instruments Electronically: Recommendations from a Qualitative Synthesis of Cognitive Interview and Usability Studies

Value Health. 2018 Jan;21(1):41-48. doi: 10.1016/j.jval.2017.07.002. Epub 2017 Aug 10.


Objectives: To synthesize the findings of cognitive interview and usability studies performed to assess the measurement equivalence of patient-reported outcome (PRO) instruments migrated from paper to electronic formats (ePRO), and make recommendations regarding future migration validation requirements and ePRO design best practice.

Methods: We synthesized findings from all cognitive interview and usability studies performed by a contract research organization between 2012 and 2015: 53 studies comprising 68 unique instruments and 101 instrument evaluations. We summarized study findings to make recommendations for best practice and future validation requirements.

Results: Five studies (9%) identified minor findings during cognitive interview that may possibly affect instrument measurement properties. All findings could be addressed by application of ePRO best practice, such as eliminating scrolling, ensuring appropriate font size, ensuring suitable thickness of visual analogue scale lines, and providing suitable instructions. Similarly, regarding solution usability, 49 of the 53 studies (92%) recommended no changes in display clarity, navigation, operation, and completion without help. Reported usability findings could be eliminated by following good product design such as the size, location, and responsiveness of navigation buttons.

Conclusions: With the benefit of accumulating evidence, it is possible to relax the need to routinely conduct cognitive interview and usability studies when implementing minor changes during instrument migration. Application of design best practice and selecting vendor solutions with good user interface and user experience properties that have been assessed in a representative group may enable many instrument migrations to be accepted without formal validation studies by instead conducting a structured expert screen review.

Keywords: PRO; cognitive interview; ePRO; electronic patient-reported outcomes; measurement equivalence; patient-reported outcomes.

MeSH terms

  • Adolescent
  • Adult
  • Aged
  • Aged, 80 and over
  • Benchmarking
  • Child
  • Child, Preschool
  • Cognition
  • Decision Making
  • Electronic Health Records / standards*
  • Evidence-Based Medicine
  • Female
  • Humans
  • Interviews as Topic*
  • Male
  • Middle Aged
  • Paper
  • Patient Reported Outcome Measures*
  • Qualitative Research