Since December 2008, Qpercom shares their expertise in advanced assessment solutions with their academic partners. Since June 2020 Qpercom is accredited as CPD provider. Due to its longest history with automated observational assessments at Qpercom’s home base, the College of Medicine, Nursing & Healthsciences, School of Medicine (Advanced Clinical Skills module, YR4MB) of the National University of Ireland Galway is leading in the way assessment data is used in educational decision making.
Recently we invited 20 Universities to join a mutual publication analysing penultimate assessments to provide public insight in assessment data analysis. A politically sensitive operation whereby finally only 8 Universities were willing to join this publication called: Cross-institutional OSCE Quality Assurance in Europe using a mutual assessment strategy; are we equipped for it? (under review June 2020). From this publication it derived that all participating universities, medical and health science educators do something different regarding their Quality Assurance.
Qpercom’s purpose is about transparent and fair decision making using cutting edge internet technologies to protect students, employees and society. Enhancing universities in using this cutting edge technology was apparently not enough. Despite a vast amount of scientific evidence being available about Quality Assurance in assessment, Universities ‘tend to do their own thing’ said Dr Thomas Kropmans, CEO of Qpercom. ‘Medical and healthcare assessors, doctors, nurses, veterinary staff are not used to utilize educational statistics to analyse the quality of their assessments and take ‘assessment error’ into account to progress students or graduates to their next level.
Qpercom processes assessment data for about 30 Universities and has access to over 1 million anonymized assessment records. Assessment Data Analysis, Artificial Intelligence and Machine Learning are currently prioritised and used to analyse these datasets to enhance and deploy decision making technologies like ‘error analysis’; ‘feedback analysis’ and ‘educational decision making’. Training our clients to dive into their own data appeared on our agenda after winning a National Tender in Norway (2019) where all higher level health sciences educational institutions chose Qpercom as their preferred remote assessment provider.
‘Dive into Data’ is our first CPD accredited (7 CPD points) remote assessment course 2*3.5 hrs spread over two days. Despite existing and well known assessment courses in Medical Education in the market, ‘Dive into Data’ is very much ‘hands on’. Hands on in this case means working with assessment data achieved through Qpercom’s Advanced Assessment Solutions. Anonymised DEMO data are used for 15 hands on assignments looking into and discussing assessment design, discrepancies in outcome, examiners variability, advantages and disadvantages of Cronbach’s Alpha, Standard Error of Measurement, Borderline Regression Analysis and how to apply these Classical Test Theory approaches on existing assessment data within your own home base. This course is for educators, OSCE organisers, examiners and administrators involved in Quality Assurance analysis. If your department is considering purchasing an advanced assessment solution like Qpercom, sign up for this course first and see what you get! (embed an assignment link)
‘Compute your Assessment’ is our second CPD accredited ( 3 CPD points) and is a 3.5 hrs course training administrators, assessors and coordinators how to set up Qpercom’s intrusive Advanced Assessment Solution in such a way that more complex assessment designs with multiple examiners, multiple scenarios or single station scenarios, observers and simulated patients can be administered as examiners. Qpercom’s assessment solutions are very flexible and allow ‘competency frameworks’ being embedded, single and multiple item score sheets being designed (mapping competencies to items, adding pictures or videos) with or without Global Rating Scales for Borderline Regression Analysis being embedded. Moreover, exam monitoring tools are available to monitor the live stream of assessment data ensuring a ‘fullhouse of data’. Our initial product training of 2 hours will get you going, compute your Assessment dives deeper into assessment design and functionality. You want to know all about Qpercom’s flexibility prior to purchasing our assessment solution, sign up for this course first and be fully trained and accredited prior to actually using the full scope of our system.
“Qpercom University” is the dream of founder CEO Dr Thomas Kropmans likewise co-authors from 8 EU Universities that collaborated in Cross-institutional OSCE Quality Assurance in 2019 knowledge and insights should be shared to enhance assessment expertise and outcome. At the moment (2020) Thomas is working with 3 Masters students in Business Analytics of the National University of Ireland in Galway and a PhD student of the same University who joined ‘Dive into Data’ firstly. A first feasibility study to analyse discrepancies between observed scores of students, the global rating scores provided by their examiners and available written feedback from same examiners is currently executed by these students analysing about 650.000 records using statistical software ‘R’ for text analysis, Tableau for data visualisation and establishing a data model for Machine Learning.
As in 2008, when we first started with Web 2.0 technologies again we embed new and cutting edge technologies to enhance educational decision making in assessment analysis. The fully anonymized outcome will be reported soon by students Gaurav, Archit, Urvika and Akram.
Through “Qpercom University” clients should have access to their visualisations and quality assurance of data being processed automatically and preferably being compared and shared online with anonymized outcomes of partnering universities. Marks, Global Ratings and Feedback need to match and Qpercom’s Advanced Assessment Solutions should learn through machine learning technologies. The seeds were planted in 2019 with the first mutual publication and some time will pass before we will harvest more. We’ll keep you informed!