eCite Digital Repository

Model Flexibility Analysis does not measure the persuasiveness of a fit

Citation

Evans, NJ and Howard, ZL and Heathcote, A and Brown, SD, Model Flexibility Analysis does not measure the persuasiveness of a fit, Psychological Review, 124, (3) pp. 339-345. ISSN 0033-295X (2017) [Refereed Article]

Copyright Statement

2017 American Psychological Association

DOI: doi:10.1037/rev0000057

Abstract

Recently, Veksler, Myers, and Gluck (2015) proposed model flexibility analysis as a method that "aids model evaluation by providing a metric for gauging the persuasiveness of a given fit" (p. 755) Model flexibility analysis measures the complexity of a model in terms of the proportion of all possible data patterns it can predict. We show that this measure does not provide a reliable way to gauge complexity, which prevents model flexibility analysis from fulfilling either of the 2 aims outlined by Veksler et al. (2015): absolute and relative model evaluation. We also show that model flexibility analysis can even fail to correctly quantify complexity in the most clear cut case, with nested models. We advocate for the use of well-established techniques with these characteristics, such as Bayes factors, normalized maximum likelihood, or cross-validation, and against the use of model flexibility analysis. In the discussion, we explore 2 issues relevant to the area of model evaluation: the completeness of current model selection methods and the philosophical debate of absolute versus relative model evaluation.

Item Details

Item Type:Refereed Article
Keywords:Model Selection; Flexibility; Complexity; Goodness-of-fit
Research Division:Psychology and Cognitive Sciences
Research Group:Cognitive Sciences
Research Field:Cognitive Science not elsewhere classified
Objective Division:Expanding Knowledge
Objective Group:Expanding Knowledge
Objective Field:Expanding Knowledge in Psychology and Cognitive Sciences
Author:Heathcote, A (Professor Andrew Heathcote)
ID Code:113330
Year Published:2017
Deposited By:Psychology
Deposited On:2016-12-21
Last Modified:2017-08-30
Downloads:0

Repository Staff Only: item control page