Background: Health agencies frequently seek to develop indicators of the quality and performance of work done by clinicians. The validity of such indicators is a subject of debate among clinicians and health managers.
Objectives: Our aim was to quantify the effects of chance and small caseload on an indicator of referral behaviour for GPs.
Methods: The study used random simulation of GP referral to physiotherapy and variance components analysis of routinely collected accident insurance data. It analysed 129 079 episodes of accident-related back pain in New Zealand which were managed by 2679 GPs. The main outcome measure was the percentage of back pain cases referred for physiotherapy and for specialist assessment and by each GP.
Results: The observed number of GPs who refer to physiotherapy at high levels is satisfactorily accounted for by chance. The variability of practice among GPs within any one area is not related to the absolute level of referral.
Conclusion: The primary care setting, in which a low caseload for any one condition is the norm, presents challenges for measuring clinical performance. An emphasis upon changing the behaviour of GPs with extremely high levels on a performance indicator cannot necessarily be expected to have an impact upon the level of the indicator across a geographic area. Indicators for quality improvement should be used across whole populations of practitioners, rather than used to focus upon extremely high referring individuals.