eCite Digital Repository

Evaluating the ability of economic models of diabetes to simulate new cardiovascular outcomes trials: a report on the Ninth Mount Hood Diabetes Challenge


Si, L and Willis, MS and Asseburg, C and Nilsson, A and Tew, M and Clarke, PM and Lamotte, M and Ramos, M and Shao, H and Shi, L and Zhang, P and McEwan, P and Ye, W and Herman, WH and Kuo, S and Isaman, DJ and Schramm, W and Sailer, F and Brennan, A and Pollard, D and Smolen, HJ and Leal, J and Gray, A and Patel, R and Feenstra, T and Palmer, AJ, Evaluating the ability of economic models of diabetes to simulate new cardiovascular outcomes trials: a report on the Ninth Mount Hood Diabetes Challenge, Value in Health, 23, (9) pp. 1163-1170. ISSN 1098-3015 (2020) [Refereed Article]

Copyright Statement

Copyright 2020 ISPOR–The Professional Society for Health Economics and Outcomes Research

DOI: doi:10.1016/j.jval.2020.04.1832


Objectives: The cardiovascular outcomes challenge examined the predictive accuracy of 10 diabetes models in estimating hard outcomes in 2 recent cardiovascular outcomes trials (CVOTs) and whether recalibration can be used to improve replication.

Methods: Participating groups were asked to reproduce the results of the Empagliflozin Cardiovascular Outcome Event Trial in Type 2 Diabetes Mellitus Patients (EMPA-REG OUTCOME) and the Canagliflozin Cardiovascular Assessment Study (CANVAS) Program. Calibration was performed and additional analyses assessed model ability to replicate absolute event rates, hazard ratios (HRs), and the generalizability of calibration across CVOTs within a drug class.

Results: Ten groups submitted results. Models underestimated treatment effects (ie, HRs) using uncalibrated models for both trials. Calibration to the placebo arm of EMPA-REG OUTCOME greatly improved the prediction of event rates in the placebo, but less so in the active comparator arm. Calibrating to both arms of EMPA-REG OUTCOME individually enabled replication of the observed outcomes. Using EMPA-REG OUTCOME–calibrated models to predict CANVAS Program outcomes was an improvement over uncalibrated models but failed to capture treatment effects adequately. Applying canagliflozin HRs directly provided the best fit.

Conclusions: The Ninth Mount Hood Diabetes Challenge demonstrated that commonly used risk equations were generally unable to capture recent CVOT treatment effects but that calibration of the risk equations can improve predictive accuracy. Although calibration serves as a practical approach to improve predictive accuracy for CVOT outcomes, it does not extrapolate generally to other settings, time horizons, and comparators. New methods and/or new risk equations for capturing these CV benefits are needed.

Item Details

Item Type:Refereed Article
Keywords:cardiovascular outcomes trial, computer modeling, diabetes, Mount Hood Diabetes Challenge
Research Division:Economics
Research Group:Applied economics
Research Field:Health economics
Objective Division:Health
Objective Group:Clinical health
Objective Field:Clinical health not elsewhere classified
UTAS Author:Si, L (Mr Lei Si)
UTAS Author:Palmer, AJ (Professor Andrew Palmer)
ID Code:140460
Year Published:2020
Web of Science® Times Cited:4
Deposited By:Menzies Institute for Medical Research
Deposited On:2020-08-19
Last Modified:2021-05-05

Repository Staff Only: item control page