We consider the recently proposed prior information criterion for statistical model selection (PIC; van de Schoot et al. 2012). Using simple binomial models as an example, we demonstrate that the PIC can produce puzzling outcomes. When employed to test various forms of inequality and equality constraints, the PIC can yield inconsistent selection results, in that it fails to select the correct, data-generating model even when the underlying truth lies strictly in that model, and not in the alternative model. Moreover, in certain cases, such inconsistency arises for all sample sizes, meaning that it is not merely an asymptotic property. By contrast, when applied across the same testing scenarios, the Bayes factor provides consistent model selection. We explain why the PIC exhibits inconsistent model selection by examining its analytic forms for binomial models in comparison to those of the Bayes factor. We extend the same account to exponential families, and provide an insight into general cases in which the PIC bears a relationship to the Bayes factor.
