Inter-rater reliability and usability of CATHIS core for homeopathic intervention studies

Martin Loef; Petra Weiermayer; Katharina Gaertner; Daniel Wrzałko; Abhijit Dutta; Subhranil Saha; J A D S Lakshani; Stephan Baumgartner; Robbert van Haselen

doi:10.1016/j.ctim.2026.103338

Inter-rater reliability and usability of CATHIS core for homeopathic intervention studies

Complement Ther Med. 2026 May:97:103338. doi: 10.1016/j.ctim.2026.103338. Epub 2026 Feb 27.

Authors

Martin Loef¹, Petra Weiermayer², Katharina Gaertner³, Daniel Wrzałko⁴, Abhijit Dutta⁵, Subhranil Saha⁶, J A D S Lakshani⁷, Stephan Baumgartner⁸, Robbert van Haselen⁹

Affiliations

¹ Institute of Integrative Medicine, University of Witten/Herdecke, Germany; Society for Clinical Research, Berlin, Germany. Electronic address: martin.loef@uni-wh.de.
² Institute of Integrative Medicine, University of Witten/Herdecke, Germany.
³ Aarepraxis Bern, Läuferplatz 6, Bern 3011, Switzerland.
⁴ Institute of Complementary and Integrative Medicine, University of Bern, Switzerland.
⁵ International Cooperation Section, Ministry of Ayush, New Delhi, India.
⁶ Dept. of Repertory, D. N. De Homoeopathic Medical College and Hospital, Govt. of West Bengal, Kolkata, India.
⁷ Department of Information Technology, Faculty of Information Technology, University of Moratuwa, Sri Lanka.
⁸ Institute of Integrative Medicine, University of Witten/Herdecke, Germany; Society for Clinical Research, Berlin, Germany; Institute of Complementary and Integrative Medicine, University of Bern, Switzerland.
⁹ Society for Clinical Research, Berlin, Germany; World Integrated Medicine Forum, London, United Kingdom.

PMID: 41765150
DOI: 10.1016/j.ctim.2026.103338

Abstract

Background: The Critical Appraisal Tool for Homeopathic Intervention Studies (CATHIS) core is a streamlined appraisal tool for homeopathic intervention studies focusing on credibility, coherence, and clinical relevance. The aim of the research project was to evaluate its inter-rater reliability, feasibility, and face validity.

Methods: In a preregistered cross-sectional study, four raters independently applied CATHIS core to 28 trials (21 randomised controlled trials, 7 non-randomised studies on interventions) drawn from reviews on insomnia and hypertension; two external reviewers provided consensus ratings. Inter-rater reliability (IRR) was estimated using percent agreement, Fleiss' κ, and Gwet's AC2 (95% CIs). Feasibility was quantified as rating time and consensus time. Associations among the three domains were explored with correlation analyses and sensitivity checks.

Results: IRR varied markedly by domain. Credibility showed good agreement (Fleiss' κ=0.66, 95% CI 0.57-0.74; AC2 =0.76, 0.71-0.82). Coherence yielded only poor-to-fair agreement (κ=0.28, 0.16-0.40; AC2 =0.41, 0.30-0.51). Clinical relevance was similarly limited (κ=0.32, 0.23-0.41; AC2 =0.36, 0.28-0.44). Individual ratings required on average 65.8 min, while consensus discussions averaged 17.7 min. Correlation analyses indicated heterogeneous and partly overlapping domain signals with limited interpretability. Face-validity responses reflected moderate-to-high acceptance but difficulties in consistent application.

Conclusion: CATHIS core yielded reproducible credibility ratings but only fair and operationally fragile agreement for coherence and clinical relevance, alongside non-trivial rating burden. Taken together, the current reliability profile is insufficient for confident use in systematic reviews. Targeted refinement appears warranted before broader implementation.

Keywords: Interrater reliability; clinical relevance; homeopathy; model validity; systematic reviews.

MeSH terms

Cross-Sectional Studies
Homeopathy* / methods
Homeopathy* / standards
Humans
Hypertension / therapy
Observer Variation
Randomized Controlled Trials as Topic
Reproducibility of Results
Sleep Initiation and Maintenance Disorders / therapy