Research: A Systematic Review of Reliability and Validity of OSCE Checklists

by | Jul 20, 2015

A paper on the reliability and validity of OSCE checklists in assessing communication skills by Winny Setyonugroho, Dr Kieran Kennedy and Dr Thomas Kropmans was accepted by Patient Education and Counselling. As part of this research, 250 assessment forms were analysed across four academic terms in order to measure the true caliber of communications skills of OSCE performance. The paper can be accessed via the following link:

Reliability and validity of OSCE checklists used to assess the communication skills of undergraduate medical students: A systematic review. Patient Education and Counselling

 

research

 

Abstract

Objectives

To explore inter-rater agreement between reviewers comparing reliability and validity of checklist forms that claim to assess the communication skills of undergraduate medical students in Objective Structured Clinical Examinations (OSCEs).

Methods

Papers explaining rubrics of OSCE checklist forms were identified from Pubmed, Embase, PsycINFO, and the ProQuest Education Databases up to 2013. Included were those studies that report empirical validity or reliability values for the communication skills assessment checklists used. Excluded were those papers that did not report reliability or validity.

Results

Papers focusing on generic communication skills, history taking, physician-patient communication, interviewing, negotiating treatment, information giving, empathy and 18 other domains (ICC −0.12–1) were identified. Regarding the validity and reliability of the communication skills checklists, agreement between reviewers was 0.45.

Conclusions

Heterogeneity in the rubrics used in the assessment of communication skills and a lack of agreement between reviewers makes comparison of student competences within and across institutions difficult.

Practice implications

Consideration should be afforded to the adoption of a standardized measurement instrument to assess communication skills in undergraduate medical education. Future research will focus upon evaluating the potential impact of the adoption of a standardised measurement instrument.

 

To discuss or request more information contact Dr Thomas Kropmans. 

 

Free eBook:

The Use of Technology in Clinical Skills Assessment
An essential read for those introducing technology into clinical skills assessment.

Technology can:

  • Reduce error rates
  • Decrease administration time
  • Increase quality standards