Skip to main page content
Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2017 Mar 22;8:15-30.
doi: 10.2147/POR.S122563. eCollection 2017.

An Evaluation of Exact Matching and Propensity Score Methods as Applied in a Comparative Effectiveness Study of Inhaled Corticosteroids in Asthma

Affiliations
Free PMC article

An Evaluation of Exact Matching and Propensity Score Methods as Applied in a Comparative Effectiveness Study of Inhaled Corticosteroids in Asthma

Anne Burden et al. Pragmat Obs Res. .
Free PMC article

Abstract

Background: Cohort matching and regression modeling are used in observational studies to control for confounding factors when estimating treatment effects. Our objective was to evaluate exact matching and propensity score methods by applying them in a 1-year pre-post historical database study to investigate asthma-related outcomes by treatment.

Methods: We drew on longitudinal medical record data in the PHARMO database for asthma patients prescribed the treatments to be compared (ciclesonide and fine-particle inhaled corticosteroid [ICS]). Propensity score methods that we evaluated were propensity score matching (PSM) using two different algorithms, the inverse probability of treatment weighting (IPTW), covariate adjustment using the propensity score, and propensity score stratification. We defined balance, using standardized differences, as differences of <10% between cohorts.

Results: Of 4064 eligible patients, 1382 (34%) were prescribed ciclesonide and 2682 (66%) fine-particle ICS. The IPTW and propensity score-based methods retained more patients (96%-100%) than exact matching (90%); exact matching selected less severe patients. Standardized differences were >10% for four variables in the exact-matched dataset and <10% for both PSM algorithms and the weighted pseudo-dataset used in the IPTW method. With all methods, ciclesonide was associated with better 1-year asthma-related outcomes, at one-third the prescribed dose, than fine-particle ICS; results varied slightly by method, but direction and statistical significance remained the same.

Conclusion: We found that each method has its particular strengths, and we recommend at least two methods be applied for each matched cohort study to evaluate the robustness of the findings. Balance diagnostics should be applied with all methods to check the balance of confounders between treatment cohorts. If exact matching is used, the calculation of a propensity score could be useful to identify variables that require balancing, thereby informing the choice of matching criteria together with clinical considerations.

Keywords: asthma; exact matching; observational; propensity score.

Conflict of interest statement

Disclosure AB and CM were employees of Research in Real-Life (RiRL), Cambridge, UK. Research in Real-Life was subcontracted by Observational and Pragmatic Research Institute Pte Ltd, Singapore, to conduct this study and has conducted paid research in respiratory disease on behalf of the following other organizations in the past 5 years: Aerocrine, AKL Ltd, Almirall, AstraZeneca, Boehringer Ingelheim, Chiesi, GlaxoSmithKline, Meda, Mundipharma, Napp, Novartis, Orion, Takeda, Teva, and Zentiva, a Sanofi company. NR has received over the past 3 years: 1) fees for speaking, organizing education, participation in advisory boards or consulting from 3M, Aerocrine, Almirall, AstraZeneca, Boehringer Ingelheim, Chiesi, Cipla, GlaxoSmithKline, MSD-Chibret, Mundipharma, Novartis, Pfizer, Sanofi, Sandoz, Teva; 2) research grants from Novartis, Boehringer Ingelheim and Pfizer. EVH is a consultant to RiRL and has received payment for writing and editorial support to Merck. The University of Groningen has received money for DSP regarding an unrestricted educational grant for research from AstraZeneca, Chiesi. Travel to conferences for the European Respiratory Society (ERS) and/or the American Thoracic Society (ATS) has been partially funded by AstraZeneca, Chiesi, GSK, Takeda. Fees for consultancies were given to the University of Groningen by AstraZeneca, Boehringer Ingelheim, Chiesi, GSK, Takeda, and TEVA. Travel and lectures in China were paid by Chiesi. RMCH and JAO are employees of the PHARMO Institute. This independent research institute performs financially supported studies for government and related health care authorities and several pharmaceutical companies. DvE and JMK are employees of Takeda. DBP has Board Membership with Aerocrine, Almirall, Amgen, AstraZeneca, Boehringer Ingelheim, Chiesi, Meda, Mundipharma, Napp, Novartis, and Teva. Consultancy: Almirall, Amgen, AstraZeneca, Boehringer Ingelheim, Chiesi, GlaxoSmithKline, Meda, Mundipharma, Napp, Novartis, Pfizer, Teva, and Zentiva; Grants/Grants Pending with UK National Health Service, British Lung Foundation, Aerocrine, AstraZeneca, Boehringer Ingelheim, Chiesi, Eli Lilly, GlaxoSmithKline, Meda, Merck, Mundipharma, Novartis, Orion, Pfizer, Respiratory Effectiveness Group, Takeda, Teva, and Zentiva; Payments for lectures/speaking: Almirall, AstraZeneca, Boehringer Ingelheim, Chiesi, Cipla, GlaxoS-mithKline, Kyorin, Meda, Merck, Mundipharma, Novartis, Pfizer, SkyePharma, Takeda, and Teva; Payment for manuscript preparation: Mundipharma and Teva; Patents (planned, pending or issued): AKL Ltd.; payment for the development of educational materials: GlaxoSmithKline, Novartis; Stock/Stock options: Shares in AKL Ltd which produces phyto-pharmaceuticals and owns 80% of Research in Real-Life Ltd, 75% of the social enterprise Optimum Patient Care Ltd and 75% of Observational and Pragmatic Research Institute Pte Ltd; received payment for travel/accommodations/meeting expenses from Aerocrine, Boehringer Ingelheim, Mundipharma, Napp, Novartis, and Teva; funding for patient enrolment or completion of research: Almirral, Chiesi, Teva, and Zentiva; peer reviewer for grant committees: Medical Research Council (2014), Efficacy and Mechanism Evaluation programme (2012), HTA (2014); and received unrestricted funding for investigator-initiated studies from Aerocrine, AKL Ltd, Almirall, Boehringer Ingelheim, Chiesi, Meda, Mundi-pharma, Napp, Novartis, Orion, Takeda, Teva, and Zentiva. The authors report no other conflicts of interest in this work.

Figures

Figure 1
Figure 1
Standardized differences between cohorts in key baseline characteristics for the unmatched dataset, exact matching, propensity score matching, and the pseudo-dataset weighted by the stabilized IPTW. Absolute standardized differences in the unmatched dataset extended to 0.375, and for the exact-matched dataset, standardized differences were outside of the ±0.1 interval defining balance for allergy prescriptions, asthma-related hospital admissions, evidence of rhinitis, and evidence of GERD. All standardized differences were within ±0.1 for the datasets matched on propensity score and the pseudo-dataset weighted by IPTW. Abbreviations: ICS, inhaled corticosteroid; GERD, gastroesophageal reflux disease; IPTW, inverse probability of treatment weighting; LABA, long-acting β2-agonist; LTRA, leukotriene receptor antagonist; NSAIDs, nonsteroidal anti-inflammatory drugs; RiRL, Research in Real-Life; SABA, short-acting β2-agonist; SAMA, short-acting muscarinic antagonist; Y/N, yes/no.
Figure 2
Figure 2
Comparison of outcomes using exact matching and propensity score methods. Notes: (A) Results for comparison of exacerbation rates using exact matching and propensity score methods. aAdjusted for propensity score and baseline exacerbations (0/≥1). bAdjusted for age group and baseline exacerbations (0/≥1). cAdjusted for evidence of GERD and baseline exacerbations (0/≥1). dAdjusted for baseline exacerbations (0/≥1). Comparison of rate ratios (95% CIs) for severe exacerbation rates estimated using a Poisson regression model. (B) Results for comparison of risk-domain asthma control using exact matching and propensity score methods. aAdjusted for propensity score and baseline RDAC status. bAdjusted for the evidence of GERD and baseline RDAC status. cAdjusted for age group, evidence of GERD, and time from first asthma prescription. dAdjusted for evidence of GERD. Odds ratios compare ciclesonide versus the fine-particle ICS cohort (the latter set at odds=1.0). Odds ratios (95% CIs) for risk-domain asthma control estimated using a logistic regression model. (C) Results for comparison of overall asthma control using exact matching and propensity score methods. aAdjusted for propensity score, baseline RDAC status, and time from first asthma prescription. bAdjusted for evidence of GERD, leukotriene receptor antagonist use, baseline average daily SABA dose (categorized) and baseline RDAC status. cAdjusted for age group, evidence of GERD, baseline average daily SABA dose (categorized) and baseline RDAC status. dAdjusted for evidence of GERD and baseline overall asthma control. eAdjusted for evidence of GERD, baseline average daily SABA dose (categorized as 0/1–100/101–200/>200 μg) and baseline RDAC status. Odds ratios compare ciclesonide versus the fine-particle ICS cohort (the latter set at odds =1.0) and were estimated using a logistic regression model. (D) Results for comparison of change in therapy using exact matching and propensity score methods. aAdjusted for evidence of rhinitis and evidence of GERD. bAdjusted for evidence of GERD. cAdjusted for evidence of rhinitis. Odds ratios compare ciclesonide versus the fine-particle ICS cohort (the latter set at odds=1.0). Odds ratios (95% CIs) for change in therapy estimated using a logistic regression model. Abbreviations: CI, confidence interval; GERD, gastroesophageal reflux disease; ICS, inhaled corticosteroid; IPTW, inverse probability of treatment weighting; PS, propensity score; PSM, propensity score matching; RDAC, risk-domain asthma control; RiRL, Research in Real-Life.
Figure 2
Figure 2
Comparison of outcomes using exact matching and propensity score methods. Notes: (A) Results for comparison of exacerbation rates using exact matching and propensity score methods. aAdjusted for propensity score and baseline exacerbations (0/≥1). bAdjusted for age group and baseline exacerbations (0/≥1). cAdjusted for evidence of GERD and baseline exacerbations (0/≥1). dAdjusted for baseline exacerbations (0/≥1). Comparison of rate ratios (95% CIs) for severe exacerbation rates estimated using a Poisson regression model. (B) Results for comparison of risk-domain asthma control using exact matching and propensity score methods. aAdjusted for propensity score and baseline RDAC status. bAdjusted for the evidence of GERD and baseline RDAC status. cAdjusted for age group, evidence of GERD, and time from first asthma prescription. dAdjusted for evidence of GERD. Odds ratios compare ciclesonide versus the fine-particle ICS cohort (the latter set at odds=1.0). Odds ratios (95% CIs) for risk-domain asthma control estimated using a logistic regression model. (C) Results for comparison of overall asthma control using exact matching and propensity score methods. aAdjusted for propensity score, baseline RDAC status, and time from first asthma prescription. bAdjusted for evidence of GERD, leukotriene receptor antagonist use, baseline average daily SABA dose (categorized) and baseline RDAC status. cAdjusted for age group, evidence of GERD, baseline average daily SABA dose (categorized) and baseline RDAC status. dAdjusted for evidence of GERD and baseline overall asthma control. eAdjusted for evidence of GERD, baseline average daily SABA dose (categorized as 0/1–100/101–200/>200 μg) and baseline RDAC status. Odds ratios compare ciclesonide versus the fine-particle ICS cohort (the latter set at odds =1.0) and were estimated using a logistic regression model. (D) Results for comparison of change in therapy using exact matching and propensity score methods. aAdjusted for evidence of rhinitis and evidence of GERD. bAdjusted for evidence of GERD. cAdjusted for evidence of rhinitis. Odds ratios compare ciclesonide versus the fine-particle ICS cohort (the latter set at odds=1.0). Odds ratios (95% CIs) for change in therapy estimated using a logistic regression model. Abbreviations: CI, confidence interval; GERD, gastroesophageal reflux disease; ICS, inhaled corticosteroid; IPTW, inverse probability of treatment weighting; PS, propensity score; PSM, propensity score matching; RDAC, risk-domain asthma control; RiRL, Research in Real-Life.

Similar articles

See all similar articles

Cited by 5 articles

References

    1. Yang D, Dalton JE. SAS Global Forum 2012. Statistics and Data Analysis. Orlando, Florida: [Accessed April 28, 2016]. A unified approach to measuring the effect size between two groups using SAS®. (Paper 335–2012). Available from: http://www.lerner.ccf.org/qhs/software/lib/stddiff.pdf.
    1. Austin PC. An introduction to propensity score methods for reducing the effects of confounding in observational studies. Multivariate Behav Res. 2011;46(3):399–424. - PMC - PubMed
    1. Concato J, Shah N, Horwitz RI. Randomized, controlled trials, observational studies, and the hierarchy of research designs. N Engl J Med. 2000;342(25):1887–1892. - PMC - PubMed
    1. Krishnan JA, Schatz M, Apter AJ. A call for action: comparative effectiveness research in asthma. J Allergy Clin Immunol. 2011;127(1):123–127. - PMC - PubMed
    1. Price D, Bateman ED, Chisholm A, et al. Complementing the randomized controlled trial evidence base. Evolution not revolution. Ann Am Thorac Soc. 2014;11(Suppl 2):S92–S98. - PubMed
Feedback