University of Tasmania
Browse

File(s) under permanent embargo

Model Flexibility Analysis does not measure the persuasiveness of a fit

journal contribution
posted on 2023-05-19, 00:30 authored by Evans, NJ, Howard, ZL, Heathcote, A, Brown, SD
Recently, Veksler, Myers, and Gluck (2015) proposed model flexibility analysis as a method that "aids model evaluation by providing a metric for gauging the persuasiveness of a given fit" (p. 755) Model flexibility analysis measures the complexity of a model in terms of the proportion of all possible data patterns it can predict. We show that this measure does not provide a reliable way to gauge complexity, which prevents model flexibility analysis from fulfilling either of the 2 aims outlined by Veksler et al. (2015): absolute and relative model evaluation. We also show that model flexibility analysis can even fail to correctly quantify complexity in the most clear cut case, with nested models. We advocate for the use of well-established techniques with these characteristics, such as Bayes factors, normalized maximum likelihood, or cross-validation, and against the use of model flexibility analysis. In the discussion, we explore 2 issues relevant to the area of model evaluation: the completeness of current model selection methods and the philosophical debate of absolute versus relative model evaluation.

History

Publication title

Psychological Review

Volume

124

Pagination

339-345

ISSN

0033-295X

Department/School

School of Psychological Sciences

Publisher

Amer Psychological Assoc

Place of publication

United States

Rights statement

© 2017 American Psychological Association

Repository Status

  • Restricted

Socio-economic Objectives

Expanding knowledge in psychology

Usage metrics

    University Of Tasmania

    Categories

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC