Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Comparative Study
. 2012 Jan 2;59(1):319-30.
doi: 10.1016/j.neuroimage.2011.07.039. Epub 2011 Jul 27.

Comparing dynamic causal models using AIC, BIC and free energy

W D Penny  1
Affiliations
Comparative Study

Comparing dynamic causal models using AIC, BIC and free energy

W D Penny. Neuroimage. .

Abstract

In neuroimaging it is now becoming standard practise to fit multiple models to data and compare them using a model selection criterion. This is especially prevalent in the analysis of brain connectivity. This paper describes a simulation study which compares the relative merits of three model selection criteria (i) Akaike's Information Criterion (AIC), (ii) the Bayesian Information Criterion (BIC) and (iii) the variational Free Energy. Differences in performance are examined in the context of General Linear Models (GLMs) and Dynamic Causal Models (DCMs). We find that the Free Energy has the best model selection ability and recommend it be used for comparison of DCMs.

PubMed Disclaimer

Figures

Fig. 1
Fig. 1
Design matrix for the full GLM. The nested GLM uses an identical design matrix but with the first three columns removed. The full design matrix comprises N = 351 rows, one for each fMRI scan, and twelve columns, one for each putative experimental effect.
Fig. 2
Fig. 2
Log Bayes factor of full versus nested model, Log Bf, n, versus the signal to noise ratio, SNR, when the true model is the full GLM for FL (black), AIC (blue) and BIC (red).
Fig. 3
Fig. 3
Log Bayes factor of nested versus full model, Log Bn, f, versus the signal to noise ratio, SNR, when the true model is the nested GLM for FL (black), AIC (blue) and BIC (red).
Fig. 4
Fig. 4
Log Bayes factor of full versus nested model, Log Bf, n, versus the number of data points, N, when the true model is the full GLM for FL (black), AIC (blue), BIC (red) and AICc (green).
Fig. 5
Fig. 5
Log Bayes factor of nested versus full model, Log Bn, f, versus the number of data points, N, when the true model is the nested GLM for FL (black), AIC (blue), BIC (red) and AICc (green).
Fig. 6
Fig. 6
A nested (left) and full (right) DCM. The full DCM is identical to the nested DCM except for having an additional modulatory forward connection from region P to region A. Intrinsic connections are indicated by dotted arrows, modulatory connections by overlaid solid arrows and inputs by solid squares with an arrow.
Fig. 7
Fig. 7
Log Bayes factor of full versus nested model, Log Bf, n, versus the signal to noise ratio, SNR, when the true model is the full DCM for FL (black), AIC (blue) and BIC (red).
Fig. 8
Fig. 8
Log Bayes factor of nested versus full model, Log Bn, f, versus the signal to noise ratio, SNR, when the true model is the nested DCM for FL (black), AIC (blue) and BIC (red).
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

    1. Akaike H. Information measures and model selection. Bull. Int. Stat. Inst. 1973;50:277–290.
    1. Attias H. Proceedings of the Fifteenth Conference on Uncertainty in Artificial Intelligence. 1999. Inferring parameters and structure of latent variable models by variational Bayes.
    1. Beal M., Ghahramani Z. The variational Bayesian EM algorithms for incomplete data: with application to scoring graphical model structures. In: Bernardo J., Bayarri M., Berger J., Dawid A., editors. Vol. 7. Cambridge University Press; 2003. (Bayesian Statistics).
    1. Bernardo J.M., Smith A.F.M. Wiley; Chichester: 2000. Bayesian Theory.
    1. Bishop C.M. Springer; 2006. Pattern Recognition and Machine Learning.

Publication types