Recently, we published a few blogs about our video-integrated (clinical) skills assessments and interview solution, Qpercom Observe for health sciences and medicine. Health sciences and medicine have used these very specific examinations since the 1970s. They are ‘objective’, highly ‘structured’ and focussed on ‘clinical’ ‘examinations’, and popularly known by the abbreviated term OSCE (phonetically pronounced OSKEE).
The interview element of these examinations is known as Multiple-Mini-Interviews or MMIs. In educational literature, they are well-researched and proven to be reliable and valid. The objectivity is associated with the multiple examiners that see the candidates or students in different stations where they ‘show their competency in the medical field’ or in the case of interview applicants, candidates introduce themselves while discussing different ‘non-technical skills’. However, in this blog we want to emphasize the fact that these time-constrained communication exercises can be effective for other sciences, schools and colleges struggling with how to assess their students remotely in a valid and reliable manner during and continuously after the COVID-19 pandemic.
How to use time-constrained examinations to assess students
Like the OSCE and MMI formats, time-constrained scenario-based practical examinations (TSPEs) are not new and were introduced by Emily Hall in veterinary nursing. The University of London (UL) actually produced an Assessment Toolkit II focusing on time-constrained examinations. In the UL toolkit they refer to briefing papers, essays and case studies. Case studies, from an inclusive design perspective, are an attractive form of learning and assessment. According to the same toolkit II, students may be given a degree of choice over their case study and thus be in a position to consider their diverse backgrounds and experiences when choosing to work on a particular case study. In my own personal experience as a senior lecturer in Medical Informatics and Medical Education, I have used various (medical/healthcare) cases to assess interdisciplinary skills in informatics, public healths, law and/or ethics. In a one station format, using Qpercom Observe, it was even very efficient to mark these individual or group assignments. With a universal marking grid approved by all content matter experts, the time-constrained scenario-based practical examination is offering students in health informatics, public health, law and ethics the opportunity to showcase their ability (skills), knowledge and expertise (attitude) in a multi-station approach using the embedded video technology. Another approach could well be taking more content matter experts on board and have them discuss content specific aspects of the case to achieve a more ‘objective score’ based on multiple examiners approaches. This is all done remotely.
Figure 1 refers to the ‘classical style ‘station examination’ where examiners use software to mark individual student performances. With the recently embedded video integration this ‘time constrained scenario’ can now be discussed with students remotely.
The content matter experts, as remote examiners, focus on their own area of expertise and a very inclusive, valid and reliable way of examination can be achieved by not only marking the assignment but also the student’s perspectives on it.
Moreover, the authors of the Toolkit II conclude that students need to have opportunities to practice online examinations in advance as well as an awareness of self-perception of typing speeds. Having a tool like Qpercom Observe allows the organisation of formative sessions were students can discuss various drafts of their assignment with their supervisors/content matter experts or peers to get used to this new systematic consecutive station remote exam.
A student submits their assignment online, the examiner/content matter expert marks the assignment in Qpercom Observe and students join various time-constrained stations of 5-10 minutes each in which they remotely meetup with their examiners. The system takes care of all the logistics. Examiners, during discussions with the student, can review their scoresheet online in the same station and re-submit the scoresheet and discuss or send feedback to the student instantly. It can also be sent by email retrospectively.
(picture with courtesy to Diana Lazariuk of InventorSoft)
Examiner meets student in stations with access to assessment score sheet
Examiner and student can discuss the work submitted while the examiner has the assessment sheet open in the background. In the case of a multi-scenario design where the examiner has access to two different scoresheets, the scoresheet of the written work can be reopened and scores can be adjusted. An ‘oral interview form’ can be used to mark the ‘defense’ of the student like it is done in a viva.
In summary:
- A time-constrained station-based approach for assessment can be considered for subjects and fields other than medicine and healthcare sciences.
- The station-based approach works for briefing papers, (all subject) essays and (technical or legal) case studies.
- An interprofessional station setup (different professional roles), matches very well with interprofessional collaboration and identity training and assessment.
- It has been tested in-house to assess interdisciplinary/interprofessional skills in informatics, public healths, law and/or ethics.
- This assessment method is perfect for remote station-based assessments, where examiners and students cannot be in the same place, at the same time.
Would you like to discuss how your department or school can implement time-constrained station-based assessment for remote examinations?
Fill out the contact form below, we’d love to hear more about your situation.