The predictive validity of the UMAT: A multi-institutional study

Background and Aims: This multi-centre study aimed to determine the predictive validity of the Undergraduate Medicine and Health Sciences Admission Test (UMAT) and other selection tools (ATAR/GPA, Interview) on medical student academic performance, to inform best practice in undergraduate medical student selection.

Methods: Eleven of twelve universities across Australia and New Zealand participated. The dataset consists of N=10471 medical students commencing from 2006-2012. Admissions scores (UMAT, ATAR/GPA and Interview score), sex and age were used to predict academic performance during medical school. Two transition points were identified as outcome variables using knowledge- and clinical-based assessment results as well as total year scores. These were the transition from campus-based to hospital-based training (T1), and between the penultimate to pre-internship year (T2). An overall course total was also included (T3).

Results: Hierarchical Multiple Regressions were used to analyse the predictive validity of the selection tools. ATAR/GPA was the strongest predictor of higher performance at the transition years (T1 &T2) and overall during the course (T3). UMAT Section 1 (Problem Solving and Logical Reasoning) was a significant predictor of increased performance at all three points. UMAT Section 2 (Understanding People) was a significant predictor at T1 and T2. UMAT Section 3 (Non-Verbal Reasoning) was a significant negative predictor at T2 and T3. Interview score was a predictor of higher performance in the hospital-based years, and of overall course performance.

Conclusions / Recommendations: The presentation reports the preliminary analysis of this comprehensive dataset. Additional analyses using advanced statistical modelling techniques are underway to further determine the predictive validity of selection tools on medical student academic performance.