An empirical test of short notice surveys in two accreditation programs

Objective To evaluate short notice surveys in accreditation programmes. Design Two trials using short notice surveys were conducted independently: a study of 20 healthcare organizations with the Australian Council on Healthcare Standards (ACHS) and a study of 7 general practices with the Australian General Practice Accreditation Limited (AGPAL). Participating organizations volunteered. ACHS and AGPAL selected 17 and 13 surveyors, respectively, and provided training for them on short notice surveys. Methods Each agency's short notice surveys were an abbreviated version of their current advanced notification surveys. Short notice surveys assessed accreditation programme criteria or indicators that corresponded to the Australian Commission on Safety and Quality in Health Care's priority issues. Fifteen (out of 45) ACHS criteria and 48 (out of 174) AGPAL indicators that aligned to the Commission's criteria were evaluated. Participating organizations were given 2 days notice prior to the short notice surveys. Ratings from the short notice surveys were compared with those from the most recent advanced notification surveys, and statistical tests were performed to detect differences and potential confounding factors. Surveyors and organizational staff completed a post-survey feedback questionnaire which was analysed thematically and by inferential statistics. Results The short notice survey approach overall produced ratings congruent with the advanced notification survey for both accreditation programmes. However, for both programmes short notice surveys assessed that more organizations would not reach the accreditation threshold as compared with the previous survey. Organizations in both programmes were judged to have achieved less successful performance against clinical standards by the short notice survey than the advanced notification survey. There was support from surveyors and organizational staff for short notice survey to be adopted. However, there were mixed views about the impact of short notice surveys and whether they validated trial participants' continuous improvement efforts. Conclusions The study demonstrated that short notice surveys are more critical in their assessment of clinical than administrative or corporate items. Short notice surveys, while broadly comparable with existing advanced notification survey practice, produced different accreditation outcomes for a significant proportion of the study organizations. The overall value and worth of short notice surveys remains to be proved.