What is Borderline Regression Analysis?
Borderline regression analysis (BRA) is an absolute, examinee-centered standard setting method widely used to standard set OSCE exams, Yousuf, Violato, and Zuberi (2015). Candidates are awarded a “global score” for a station in a circuit, based on the examiner’s professional judgment of their ability.
Borderline Regression Method is illustrated above using item score on the Y-axis and Global Ratings on the X-axis. 0=Fail, 1=Borderline, 2=Pass, 3=Good and 4=Excellent.
For a working example, in Qpercom’s OSCE Management Information System, Observe, three different types of Borderline Regression Analysis are available:
1. Borderline Group Average
2. Borderline Regression (forecast) Method 1
3. Borderline Regression (forecast) Method 2
Borderline Group Average takes the average of all examinees that scored ‘Borderline’ (1) and is considered as unreliable in case only a few (< 10) candidates are marked as borderline. In the latter case, Method 1 or 2 are recommended as they take into account all of the Global Rating Scores and not just the Borderline scores (Dwyer et al., 2016).
The difference between Regression Method 1 and Regression Method 2 is the type of Global Rating Score (GRS) used. In Method 1, only 1 Borderline score (represented by 1) is used to mark candidates. In Method 2, scores for Borderline Fail (represented by 1) and Borderline Pass (represented by 2) are utilised. The cut-score in the latter case will be higher (6.53) than in Method 1 (5.99) as it is right in-between Borderline Pass and Borderline Fail (Homer, Pell, Fuller, & Patterson, 2016).
Standard setting procedures can be categorised as either exam-centered, in which the content of the test is reviewed by the expert judges (e.g., Angoff method), or examinee-centered, where expert decisions are based on the actual performance of the examinees, like in the BRM of Qpercom’s software.
The examiner rates the performance at each station by completing a checklist and a global rating scale. The checklist marks from all examinees at each station are then regressed on the attributed global rating scores, providing a linear equation. The global score representing borderline performance (e.g., 1 or 1 and 2 on the global performance rating scale) is substituted into the equation. This predicts the pass-fail cut-score for the checklist marks using the Forecast, method using the Intercept and 1 or 1.5 times the Slope of the Regression line.
References
Dwyer, T., Wright, S., Kulasegaram, K. M., Theodoropoulos, J., Chahal, J., Wasserstein, D., . . . OgilvieHarris, D. (2016). How to set the bar in competency-based medical education: standard setting after an Objective Structured Clinical Examination (OSCE). BMC Med Educ, 16, 1. doi:10.1186/s12909-015-0506-z
Meskell, P., Burke, E., Kropmans, T. J., Byrne, E., Setyonugroho, W., & Kennedy, K. M. (2015). Back to the future: An online OSCE Management Information System for nursing OSCEs. Nurse Educ Today, 35(11), 1091-1096. doi:10.1016/j.nedt.2015.06.010
Schoonheim-Klein, M., Muijtjens, A., Habets, L., Manogue, M., van der Vleuten, C., & van der Velden, U. (2009). Who will pass the dental OSCE? Comparison of the Angoff and the borderline regression standard setting methods. Eur J Dent Educ, 13(3), 162-171. doi:10.1111/j.16000579.2008.00568.x
Yousuf, N., Violato, C., & Zuberi, R. W. (2015). Standard Setting Methods for Pass/Fail Decisions on High-Stakes Objective Structured Clinical Examinations: A Validity Study. Teach Learn Med, 27(3), 280-291. doi:10.1080/10401334.2015.1044749