Best practices in reporting analyses of questionnaires as objective rating scales of variable measures

Abstract

Background: The best practice model states that the highest quality of scientific information in a discipline should be used when addressing pertinent problems. The usefulness of any measure depends on the least allowable error, which implies that best practice approaches must be used during analyses of rating scales. However, modern theories of objective measurement in advanced statistics suggest there are some shortcomings in reports of questionnaire analyses.

Aim: To highlight some common problems in questionnaire data and suggest techniques of constructing objective measures during rating scale analysis.

Discussion: Questionnaires are frequently used as rating scales of latent variables, such as knowledge, anxiety and outcomes of treatments. However, reports of the steps involved before generating the final 'measures' often fail to present known limitations and robust solutions to the problems common to questionnaire data. Most designers of questionnaires generate variable measures for either educational or clinical research purposes without providing adequate explanations of the steps taken to address inherent limitations that may worsen the error terms in the outcome measure.

Conclusion: Cursory attention is given to the problems in questionnaire analysis as most users do not convincingly justify the measurement techniques they used before they present variable estimation. Reporting the techniques used to address data complexity by engaging objective measurement parameters ensures best practice and emphasises the credibility of the outcome measure.

Implications for practice: Among researchers, using the techniques outlined here will lead to standardisation of questionnaire analysis and elimination of avoidable errors in constructing variable measures, resulting in high-quality data suitable for parametric statistics. For clinicians, these methods will simplify the interpretation of numerical measures to equivalent indicators on Wright maps, thus avoiding inconsistencies and misinterpretations of variable measure.

Keywords: data analysis; instrument design; qualitative research; research; research methods; surveys.