How modellers model: the overlooked social and human dimensions in model intercomparison studies
Albanito, F and McBey, D and Harrison, MT and Smith, P and Ehrhardt, F and Bhatia, A and Bellocchi, G and Brilli, L and Carozzi, M and Christie, K and Doltra, J and Dorich, CD and Doro, L and Grace, P and Grant, B and Leonard, J and Liebig, M and Ludemann, C and Martin, R and Meier, E and Meyer, R and De Antoni Migliorati, M and Myrgiotis, V and Recous, S and Sandor, R and Snow, V and Soussana, J-P and Smith, WN and Fitton, N, How modellers model: the overlooked social and human dimensions in model intercomparison studies, Environmental Science & Technology ISSN 0013-936X (In Press) [Refereed Article]
There is a growing realisation that the complexity of model ensemble studies depends not only the models used, but also on the experience and approach used by modellers to calibrate and validate results, which remain a source of uncertainty. Here, we applied a multi-criteria decision-making method to investigate the rationale applied by modellers in a model ensemble study where twelve process-based biogeochemical models were compared across five successive calibration stages. The modellers shared a common level of agreement about the importance of the variables used to initialise their models for calibration. However, we found inconsistency among modellers when judging the importance of input variables across the different calibration stages. The level of subjective weighting attributed by modellers to calibration data decreased sequentially as the extent and number of variables provided increased. In this context, the perceived importance attributed to variables such as fertilisation rate, irrigation regime, soil texture, pH, and initial levels of soil organic carbon and nitrogen stocks were statistically different when classified according to model types. The importance attributed to input variables such as experimental duration, gross primary production, net ecosystem exchange, varied significantly according to the length of the modeller's experience. We argue that the gradual access to input data across the five calibration stages negatively influenced the consistency of the interpretations made by the modellers, with cognitive bias in "trial-and-error" calibration routines. Our study highlights that overlooked human and social attributes is critical in the outcomes of modelling and model intercomparison studies. While complexity of the processes captured in the model algorithms and parameterisation are important, we contend that (1) the modeller's assumptions on the extent to which parameters should be altered, and (2) modeller perceptions of the importance of model parameters, are just as critical in obtaining a quality model calibration as numerical or analytical details.
model ensembles, biogeochemical models, multi-criteria decision-making, model calibration, model intercomparison, climate change, greenhouse gases, soil carbon, DNDC, DayCent, APSIM, DairyMod, Century, climate crisis, climate change, net-zero