Skip to main page content
Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Comparative Study
. 2011 May 23;11:77.
doi: 10.1186/1471-2288-11-77.

Logistic Random Effects Regression Models: A Comparison of Statistical Packages for Binary and Ordinal Outcomes

Free PMC article
Comparative Study

Logistic Random Effects Regression Models: A Comparison of Statistical Packages for Binary and Ordinal Outcomes

Baoyue Li et al. BMC Med Res Methodol. .
Free PMC article


Background: Logistic random effects models are a popular tool to analyze multilevel also called hierarchical data with a binary or ordinal outcome. Here, we aim to compare different statistical software implementations of these models.

Methods: We used individual patient data from 8509 patients in 231 centers with moderate and severe Traumatic Brain Injury (TBI) enrolled in eight Randomized Controlled Trials (RCTs) and three observational studies. We fitted logistic random effects regression models with the 5-point Glasgow Outcome Scale (GOS) as outcome, both dichotomized as well as ordinal, with center and/or trial as random effects, and as covariates age, motor score, pupil reactivity or trial. We then compared the implementations of frequentist and Bayesian methods to estimate the fixed and random effects. Frequentist approaches included R (lme4), Stata (GLLAMM), SAS (GLIMMIX and NLMIXED), MLwiN ([R]IGLS) and MIXOR, Bayesian approaches included WinBUGS, MLwiN (MCMC), R package MCMCglmm and SAS experimental procedure MCMC.Three data sets (the full data set and two sub-datasets) were analysed using basically two logistic random effects models with either one random effect for the center or two random effects for center and trial. For the ordinal outcome in the full data set also a proportional odds model with a random center effect was fitted.

Results: The packages gave similar parameter estimates for both the fixed and random effects and for the binary (and ordinal) models for the main study and when based on a relatively large number of level-1 (patient level) data compared to the number of level-2 (hospital level) data. However, when based on relatively sparse data set, i.e. when the numbers of level-1 and level-2 data units were about the same, the frequentist and Bayesian approaches showed somewhat different results. The software implementations differ considerably in flexibility, computation time, and usability. There are also differences in the availability of additional tools for model evaluation, such as diagnostic plots. The experimental SAS (version 9.2) procedure MCMC appeared to be inefficient.

Conclusions: On relatively large data sets, the different software implementations of logistic random effects regression models produced similar results. Thus, for a large data set there seems to be no explicit preference (of course if there is no preference from a philosophical point of view) for either a frequentist or Bayesian approach (if based on vague priors). The choice for a particular implementation may largely depend on the desired flexibility, and the usability of the package. For small data sets the random effects variances are difficult to estimate. In the frequentist approaches the MLE of this variance was often estimated zero with a standard error that is either zero or could not be determined, while for Bayesian methods the estimates could depend on the chosen "non-informative" prior of the variance parameter. The starting value for the variance parameter may be also critical for the convergence of the Markov chain.


Figure 1
Figure 1
IMPACT study: Box plot of a sample of the random effects (for center 1 to 10). Each box represents a center with its random effects estimate and confidence interval.
Figure 2
Figure 2
IMPACT study: Histogram of the random effects in the binary model in R.

Similar articles

See all similar articles

Cited by 26 articles

See all "Cited by" articles


    1. Rasbash J. What are multilevel models and why should I use them?
    1. Molenberghs G, Verbeke G. Models for Discrete Longitudinal Data. Berlin, Springer; 2005.
    1. Peter C, Jack V, David A. Comparing hierarchical modeling with traditional logistic regression analysis among patients hospitalized with acute myocardial infarction: should we be analyzing cardiovascular outcomes data differently? American Heart Journal. 2003;145:27–35. doi: 10.1067/mhj.2003.23. - DOI - PubMed
    1. Zhou X, Perkins AJ, Hui SL. Comparisons of software packages for generalized linear multilevel models. The American Statistician. 1999;53:282–290. doi: 10.2307/2686112. - DOI
    1. Guo G, Zhao H. Multilevel modeling for binary data. Annual Review of Sociology. 2000;26:441–462. doi: 10.1146/annurev.soc.26.1.441. - DOI

Publication types

LinkOut - more resources