An evaluation of the Workplace Based Assessment at the Launceston General Hospital

This study investigated and evaluated the Workplace Based Assessment (WBA) program undertaken by International Medical Graduates (IMGs) at the Launceston General Hospital (LGH). Commenced in 2010, 21 IMGs had participated in the program as part of their accreditation with the Australian Medical Council (AMC). In this program, each IMG was required to pass 22 assessments. This evaluation used a mixed-methods approach including analysis of assessment records and interviews with both assessors and IMGs. A total of 91 medical practitioners had been trained in WBA methods. IMGs had worked in the areas of medicine; surgery; emergency medicine; paediatric medicine; psychiatry; and obstetrics and gynaecology. Of the 21 IMGs, 13 had completed the program and 8 were still completing the WBA program at the time of this study. These 21 IMGs had been assessed on 370 separate occasions using three tools: the Mini-CEX, Cased Based Discussion (CBD) assessments and the In Training Assessment (ITA). A high level internal consistency was found for each tool: Mini-CEX (α= .964, n=197); CBD (α = .894, n=102); and the ITA (α= .926, n=71) and strong inter-rater reliability between assessors was also demonstrated: Mini-CEX [F(6,1170) = 17.234, p = .000]; CBD [F(2,612) = 19.339, p = .000]; and ITA [F(2,630) = 3.841, p = .000]. These results should help allay assessors’ concern their assessments were different from others Interviews were conducted with 4 IMGs (three of whom had completed the WBA program, while the fourth had completed 13 assessments) and 14 assessors (11 specialists, 2 general practitioners and 1 visiting medical officer) who had undertaken 29.1% of the total number of WBA assessments. The assessors recognised the benefit of spreading the assessments over a large number of medical practitioners, though would have liked to be more involved in the program. They reported confidence in the program and applauded its flexibility. The WBA was seen as a recruitment drawcard to attract IMGs to the Launceston General Hospital and commented that they were seeing a higher calibre of IMG working at the hospital since the program had commenced. Assessors reported the program provided useful insights to each IMG; their clinical competence and their ability to communicate and interact with other health professionals and patients. Assessors and IMGs felt the program reduced anxiety, stress and helped new IMGs better understand the Australian medical system. Many of those interviewed discussed the subjectivity of the program. This was, in part, due to IMGs being known to the assessors and having worked together. However, IMGs saw this familiarity as a positive aspect of the program that helped reduce anxiety, improved communication and skill development. Concerns were expressed regarding the level at which IMGs were to be assessed as it was thought that some assessors expected IMGs to function at the level of a registrar rather than at the intern level. There were also questions related to the assessment processes. Some assessors felt they should select a case or patient whom the candidate had not seen before, while others asked IMGs to present a patient they knew when undertaking a Mini-CEX. Participants felt that the current range of assessments reasonably represented the seven competencies identified by the AMC as required for registration as a doctor. Many assessors felt the current assessment tools were easy to read, simple to complete and acted as a prompt within the assessment. Nevertheless, there were some concerns regarding the wording of items for use in areas such as psychiatry. In addition, there was little enthusiasm for the adoption of tools such as the Multi-Source Feedback (3600 assessments) and Direct Observation of Procedural Skills (DOPS). Despite these concerns, comments were made which suggest support for the collection of data similar to the 3600.