Book contents
- Frontmatter
- Contents
- Preface
- A guide to notation
- 1 Model selection: data examples and introduction
- 2 Akaike's information criterion
- 3 The Bayesian information criterion
- 4 A comparison of some selection methods
- 5 Bigger is not always better
- 6 The focussed information criterion
- 7 Frequentist and Bayesian model averaging
- 8 Lack-of-fit and goodness-of-fit tests
- 9 Model selection and averaging schemes in action
- 10 Further topics
- Overview of data examples
- References
- Author index
- Subject index
4 - A comparison of some selection methods
Published online by Cambridge University Press: 05 September 2012
- Frontmatter
- Contents
- Preface
- A guide to notation
- 1 Model selection: data examples and introduction
- 2 Akaike's information criterion
- 3 The Bayesian information criterion
- 4 A comparison of some selection methods
- 5 Bigger is not always better
- 6 The focussed information criterion
- 7 Frequentist and Bayesian model averaging
- 8 Lack-of-fit and goodness-of-fit tests
- 9 Model selection and averaging schemes in action
- 10 Further topics
- Overview of data examples
- References
- Author index
- Subject index
Summary
In this chapter we compare some information criteria with respect to consistency and efficiency, which are classical themes in model selection. The comparison is driven by a study of the ‘penalty’ applied to the maximised log-likelihood value, in a framework with increasing sample size. AIC is not strongly consistent, though it is efficient, while the opposite is true for the BIC. We also introduce Hannan and Quinn's criterion, which has properties similar to those of the BIC, while Mallows's Cp and Akaike's FPE behave like AIC.
Comparing selectors: consistency, efficiency and parsimony
If we make the assumption that there exists one true model that generated the data and that this model is one of the candidate models, we would want the model selection method to identify this true model. This is related to consistency. A model selection method is weakly consistent if, with probability tending to one as the sample size tends to infinity, the selection method is able to select the true model from the candidate models. Strong consistency is obtained when the selection of the true model happens almost surely. Often, we do not wish to make the assumption that the true model is amongst the candidate models. If instead we are willing to assume that there is a candidate model that is closest in Kullback–Leibler distance to the true model, we can state weak consistency as the property that, with probability tending to one, the model selection method picks such a closest model.
- Type
- Chapter
- Information
- Model Selection and Model Averaging , pp. 99 - 116Publisher: Cambridge University PressPrint publication year: 2008