On moving from structured oral assessments to computer-aided assessments for vocational training

Included in methods commonly used for assessing knowledge and skills associated with vocational training are structured oral assessments (SOAs) since they can be used to assess knowledge and skills to a depth rarely achieved in other forms of testing. However, SOAs require considerable preparation by the assessors, they can be restricted by time and assessor allocation, it is difficult to cover the course and they can put stress on the examinees. Computer-aided tests have merits and problems. They are efficient and straightforward to run, and they give an assessment environment which is less stressful than SOAs, but, they have a disadvantage in that students will on occasions guess answers when multiple-choice questions (MCQs) are used. Two electronic assessment methods are developed here, one using a scoring method and the other using a set of pairs of MCQs with both designed to counteract examinee guessing. A comparison of the four assessment results follows for three cohorts of students, namely, the results from an SOA, the results of the electronic assessment method using the “scoring method”, the results of the electronic assessment method using the “set-of-pairs” method and the results of an electronic assessment method using the traditional “positive-grades-only” scoring rules.