A few words about this paper…
It was not long after we implemented our OSCE Management Information System within the School of Medicine in the National University of Ireland, Galway, that our colleagues from the School of Nursing started to use the system. Moreover, as nurses embed solid practice in their research, the purpose of this practice-based investigation was to initiate a first user acceptance test. We were very pleased with their initiative as all (n=18) nurse examiners appeared to very satisfied with the electronic system and its embedded functionalities. With a University/National cut-score in nursing of (only) 40%, it was surprising to see when the difficulty of the stations using Borderline Regression Analysis was incorporated, the cutscore rose towards 60%. Nurses are very well trained in their skill sets. Although not further researched to date, approximately thirty hours of administration time was saved. In contrast to previous paper-based approaches prior to 2014, results and feedback could be released immediately after the exam was finished, according to Pauline, Eimear and Evelyn.
AUTHORS
Pauline Meskell, Eimear Burke, Thomas J.B. Kropmans, Evelyn Byrne, Winny Setyonugroho and Kieran M. Kennedy
Highlights
- We utilise an online marking tool in administration of a Nursing and Midwifery OSCE.
- We explored users acceptance of the online marking tool.
- We present the potential for application in Nursing and Midwifery education.
- We demonstrate improved reliability and validity of the assessment process.
- The system enabled quality assurance analysis of consecutive years of student cohorts.
Summary
Background
The Objective Structured Clinical Examination (OSCE) is an established tool in the repertoire of clinical assessment methods in nurse education. The use of OSCEs facilitates the assessment of psychomotor skills as well as knowledge and attitudes. Identified benefits of OSCE assessment include development of students’ confidence in their clinical skills and preparation for clinical practice. However, a number of challenges exist with the traditional paper methodology, including documentation errors and inadequate student feedback.
Objectives
To explore electronic OSCE delivery and evaluate the benefits of using an electronic OSCE management system. To explore assessors’ perceptions of and attitudes to the computer based package.
Design
This study was conducted using electronic software in the management of a four station OSCE assessment with a cohort of first year undergraduate nursing students delivered over two consecutive years (n = 203) in one higher education institution in Ireland. A quantitative descriptive survey methodology was used to obtain the views of the assessors on the process and outcome of using the software.
Methods
OSCE documentation was converted to electronic format. Assessors were trained in the use of the OSCE management software package and laptops were procured to facilitate electronic management of the OSCE assessment. Following the OSCE assessment, assessors were invited to evaluate the experience.
Results
Electronic software facilitated the storage and analysis of overall group and individual results thereby offering considerable time savings. Submission of electronic forms was allowed only when fully completed thus removing the potential for missing data. The feedback facility allowed the student to receive timely evaluation on their performance and to benchmark their performance against the class.
Conclusions
Assessors’ satisfaction with the software was high. Analysis of assessment results can highlight issues around internal consistency being moderate and examiners variability. Regression analysis increases fairness of result calculations.
Click below for article full text…
Back to the future 2: An online OSCE Management Information System for nursing OSCEs
Click here to provide feedback on the paper.