Transforming Exams: Processes and platform for e-Exams in supervised BYOD environments (final report)

This 'Transforming Exams' seed grant developed a proof of concept for a bring-your-own-device (BYOD) based e-Exams approach. This included the development of practice guides and technology components that were verified via live trials at The University of Queensland (UQ) and University of Tasmania (UTAS) using the prototype e-Exam System (student module) software. The approach built on prior work undertaken by Dr Andrew Fluck at UTAS and the exploration carried out though Professor Geoffrey Crisp's Australian Learning and Teaching Council (ALTC) e-assessment fellowship and Dr Mathew Hillier's subsequent Office for Learning and teaching (OLT) extension grant.

The approach to computerised examinations considered a systems view of high stakes assessment that took in the concerns of stakeholders, desirable pedagogical capabilities, technology affordances and available facilities. A multi stage research program surrounded the development of the approach that included consultation with academic and practice literature as well as a range of stakeholders including students, academics, exams officers, technology support and senior leadership of the host institution. Data was collected from students involved in exam trials at points before and after the exam event that included first impressions, technical information, exam experience, exam writing strategies as well as general writing strategies and use of computers in their learning outside of examinations.

The approach to computerised examinations entails making a 'whole computer' environment available in the supervised exam room context in a way that leverages the availability of bring-your-own laptops supplied by students while maintaining controlled, consistent software facilities for all candidates. The e-Exam system provides for a transition from paper-based assessment, though paper-equivalent assessment to post-paper assessments that can include multimedia, simulations, virtual experiments, spread sheets and the ability to host a range of discipline specific software tools. The software is highly configurable allowing for word processor based exams, and with further development, computer marked question based on an on-board Moodle learning management system. Access to the Internet is optional and configurable. Where reliable and robust network connectivity can be provided the e-Exam system can also serve as a restricted gateway to online hosted exams within an institutional LMS.

The e-Exam system software, based on a Linux Live USB stick leaves the hard drive of student owned equipment untouched while allowing the examining authority control over the software environment. The e-Exam system software client is wholly made up of open source elements thus eliminating the costs associated with purchasing and managing commercial software licenses.

As a part of this seed project, a series of dissemination events were hosted to gauge interest by audiences in the e-exam concept and to spread news of the work. The feedback gathered from event participants is included in this report, as are details of the events held. Events included conference presentations and seminars or briefing sessions held directly at institutions in Australia, New Zealand, Singapore, Hong Kong as well as online. As a result expressions of interest were received from over 30 institutions to trial the e-exam approach or to be kept in touch with project developments. Further connections were made with over half dozen institutions participating in an international online conference hosted by the project leader on e-Exams. These connections were formed with a view to facilitating information and technology exchange in the future.

The prototype e-exam system is available to the public for download from the project website at www.transformingexams.com as are all research outputs from the live trials, and user guides for students and technical administrators. Finally, a set of recommendations for the further development of the e-Exam approach is provided at the conclusion of this report.