posted on 2023-05-20, 06:56authored byStarns, JJ, Cataldo, AM, Rotello, CM, Annis, J, Aschenbrenner, A, Broder, A, Cox, G, Criss, A, Curl, RA, Dobbins, IG, Dunn, J, Enam, T, Evans, NJ, Farrell, S, Fraundorf, SH, Gronlund, SD, Heathcote, A, Heck, DW, Hicks, JL, Huff, MJ, Kellen, D, Key, KN, Kilic, A, Klauer, KC, Kraemer, KR, Leite, FP, Lloyd, ME, Malejka, S, Mason, A, McAdoo, RM, McDonough, IM, Michael, RB, Mickes, L, Mizrack, E, Morgan, DP, Mueller, ST, Osth, A, Reynolds, A, Seale-Carlisle, TM, Singmann, H, Sloane, JF, Smith, AM, Tillman, G, van Ravenzwaaij, D, Weidemann, CT, Wells, L, White, CN, J Wilson
Scientific advances across a range of disciplines hinge on our ability to make inferences about unobservable theoretical entities based on empirical data patterns. Accurate inferences rely on both a) discovering valid, replicable data patterns, and b) accurately 25 interpreting those patterns in terms of their implications for theoretical constructs. The replication crisis in science has led to widespread efforts to improve the reliability of research findings, but comparatively little attention has been devoted to the validity of inferences based on those findings. Using an example from cognitive psychology, we demonstrate a blinded inference paradigm for assessing the quality of theoretical inferences from data. Our results 30 reveal substantial variability in expert judgements on the very same data, hinting at a possible inference crisis.
History
Publication title
Advances in Methods and Practices in Psychological Science