Multi-Sites, Multi-Scenarios and Multi-Examiners using Qpercom’s OMIS System

Galway, 27 February 2016

On Friday the 26th of February, Qpercom participated successfully in the NUIG School of Medicine’s multi-site Year 4 Objective Structured Clinical Examination (OSCE). New from last year’s OSCE was the opening of the ‘fit for purpose’ clinical skills suites at Sligo General Hospital Medical Academy. Together with the St Angelas NUIG academy, 2 beautiful sites were participating in Sligo along with the ‘old reliable’ 4 Galway sites. These included 2 HSE sites within the Obstetrics and Pediatrics departments of UCHG and 2 NUIG sites at the Clinical Science Institute and the Comerford Education Centre. All in all, a total of 60 stations were rolled out across the 6 sites in Galway and Sligo in this large scale OSCE assessing 189 medical student’s clinical competencies.

hospital administrator team

To avoid students briefing the content of individual stations to their fellow examinees, some of the most critical stations used the ‘multi-scenario’ feature on Qpercom’s OMIS system with adjusted scenarios and likewise, adjusted forms. Furthermore, due to the functionality of using the ‘multi-examiners’ feature on OMIS, examiners could be easily swapped during the course of the examination. Even inexperienced examiners or Simulated Patients (SP) could join the examination process while being supervised by more experienced and higher rated examiners. In total, 80 examiners participated during this multi-site OSCE.

In data-control centers in both Galway and Sligo, team members of Qpercom, under the guidance of senior lecturer and CEO of Qpercom Dr Thomas Kropmans, kept an eye on the incoming streams of live data. This data could be retrieved, stored and analysed in real time offering a basic quality review of raw data entry, examiners analysis and summary psychometric reports comparing online results from each of the 6 participating sites. Assessment and Quality Assurance data became available for module coordinators and program directors while the exam was still ongoing. Compared to the usual paper-based approach, this is a win-win scenario ensuring all exam data is successfully captured during the exam.

Cropped view of a man lying on hospital bed in the emergency room, breathing oxygen, surrounded by doctors. The patient is wearing a hospital gown, and an African American doctor is taking his blood pressure.

OSCEs are very labour intensive during the preparation, examination and results analysis phases. However, due to the implementation of OMIS at the School of Medicine in NUIG in December 2008, 70% of the administrative costs could be saved due to the OSCE Planning and Management software. Where lower technology approaches with paper forms contain up to 30% errors, the OMIS system is error free and data cannot be lost or misplaced. Furthermore, since the implementation of OMIS, Generalizability Coefficients (reliability and validity measures) have improved over time and average results and their variations have decreased to more normal dispersion due to improved standard setting and Quality Assurance procedures within OMIS.

However, students could benefit more from the implementation of all functionalities within OMIS such as the instant Student Feedback System. Unfortunately, the School of Medicine is not currently using the Student Feedback mechanism which offers the opportunity to inform students about their performance in an individualized PDF report highlighting those aspects that went well and those competencies that need further improvement. This is achieved through personalised radar plots explaining student’s results and while feedback is the most powerful learning tool, students cannot get access to this information yet. Furthermore, Module Coordinators could benefit from an OSCE management tool storing all relevant documents, SP and training examiners for specific stations.