Background: Differences in performance of the Self-Controlled Case Series (SCCS) for signal detection have been reported across different databases. However, there has been limited comparative analysis of performance and it remains unknown whether combinations of databases could enable more effective signal detection.
Objectives: This study aims to compare the performance of the SCCS for signal detection across several data sources, and to determine whether combinations of databases can improve SCCS performance.
Methods: We applied the SCCS to macrolides and fluoroquinolone antibiotics, in four databases: Merative MarketScan Commercial Claims and Medicare, the Clinical Practice Research Datalink (CPRD) Aurum and the Système National des Données de Santé. We developed a reference set of 104 positive controls and 58 negative controls, using a taxonomy framework to ensure the selected drug outcome pairs are theoretically well suited to the SCCS design. The observation period lasted 2 years, with a 30-day risk-window after each dispensing. Diagnostic performance was measured using sensitivity, specificity and area under the receiver operating curve (AUC) with respect to the product labels, both for individual and combinations of databases.
Results: The sensitivity of the SCCS ranged from 0.57-0.89 across individual databases, and the specificity from 0.43-0.77 when limited to drug-outcome pairs sufficiently powered. The combination of all databases achieved the maximum sensitivity of 0.89 (0.41 specificity) for the full reference set, and a sensitivity of 1 (0.35 specificity) for drug outcome pairs with enough power. Whilst AUCs ranged from 0.66 to 0.71 across individual databases, the highest performing combination was CPRD plus MarketScan Commercial Claims (0.76 AUC).
Conclusions: Using a carefully designed reference set of drug-outcome pairs well suited to the study design, the SCCS performance varied substantially by database due to differences in population, reporting, healthcare and coding systems and prescribing patterns. Multi-database studies showed increased performance of SCCS for signal detection.
Keywords: claims data; electronic health records; pharmacoepidemiology; pharmacovigilance; real‐world data; self‐controlled case series; signal detection.
We explored the performance of a self‐controlled method, where risks of outcomes (potential adverse events) are compared between different time points in each patient, for the detection of Adverse Drug Reactions using health claims databases. We also explored the added value of a recent methodological development, active comparators, which can address some of the remaining bias. We looked at the capacity of the method at identifying known drug side effects for commonly prescribed antibiotics. The method was able to give a correct result in 57% to 89% of positive controls (known adverse drug events) depending on the databases, and 43% to 77% of negative controls (drugs known not to be associated with certain outcomes) when limited to drug‐outcome pairs with a satisfying number of occurrences. Overall, the performance of the method varied substantially by database, due to differences in population, healthcare systems and prescribing patterns in the different countries. Using combinations of databases enabled an increase in the performance of the method.
© 2026 GSK and The Author(s). Pharmacoepidemiology and Drug Safety published by John Wiley & Sons Ltd.