Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2014 Aug 1;72(2):217-229.
doi: 10.1007/s40300-014-0047-y.

Negotiating Multicollinearity with Spike-and-Slab Priors

Veronika Ročková  1 , Edward I George  2
Affiliations

Negotiating Multicollinearity with Spike-and-Slab Priors

Veronika Ročková et al. Metron. .

Abstract

In multiple regression under the normal linear model, the presence of multicollinearity is well known to lead to unreliable and unstable maximum likelihood estimates. This can be particularly troublesome for the problem of variable selection where it becomes more difficult to distinguish between subset models. Here we show how adding a spike-and-slab prior mitigates this difficulty by filtering the likelihood surface into a posterior distribution that allocates the relevant likelihood information to each of the subset model modes. For identification of promising high posterior models in this setting, we consider three EM algorithms, the fast closed form EMVS version of Rockova and George (2014) and two new versions designed for variants of the spike-and-slab formulation. For a multimodal posterior under multicollinearity, we compare the regions of convergence of these three algorithms. Deterministic annealing versions of the EMVS algorithm are seen to substantially mitigate this multimodality. A single simple running example is used for illustration throughout.

PubMed Disclaimer

Figures

Figure 1
Figure 1
Likelihood surface and multimodal posterior landscapes under the point-mass spike-and-slab prior
Figure 2
Figure 2
Posterior landscapes under continuous spike and slab priors
Figure 3
Figure 3
Geometry of EM convergence regions
Figure 4
Figure 4
Geometry of convergence regions using deterministic annealing.
Figure 5
Figure 5
Geometry of convergence regions using deterministic annealing.
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

References

    1. Bar H, Booth J, Wells M. An Empirical Bayes Approach to Variable Selection and QTL Analysis. In the Proceedings of the 25th International Workshop on Statistical Modelling, Glasgow, Scotland. 2010:63–68.
    1. Figueiredo MA. Adaptive Sparseness for Supervised Learning. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2003;25:1150–1159.
    1. George EI, McCulloch RE. Variable Selection Via Gibbs Sampling. Journal of the American Statistical Association. 1993;88:881–889.
    1. George EI, McCulloch RE. Approaches for Bayesian Variable Selection. Statistica Sinica. 1997;7:339–373.
    1. George E, Rockova V, Lesaffre E. Faster spike-and-slab variable selection with dual coordinate ascent EM. Proceedings of the 28thWorkshop on Statistical Modelling. 2013;1:165–170.

LinkOut - more resources