Assessing clinical reasoning: stop looking for the right answer
In physiotherapy practice, clinical reasoning is required in order to understand and address health issues. However, assessing people’s clinical reasoning skills proves to be challenging, as there usually isn’t just one right answer. As part of the O-SCT project, which forms part of the Physiotherapy programme at the HAN University of Applied Sciences, we are experimenting with the online Script Concordance Test (O-SCT), which is able to take into account differences of opinion as to the best diagnosis or treatment option.
Students are trained to base their clinical reasoning as much as possible on scientific evidence. But in practice, it turns out that evidence isn’t always available or appropriate to the situation, or is inconsistent with the physiotherapist’s experience or the patient’s wishes. Students find themselves confronted with differences of opinion among physiotherapists as to the best solution, which leads to frustration. Practice supervisors often get students telling them ‘We never covered that in class’, and students, in turn, are often asked by their supervisor ‘What exactly is it you did learn during your courses?’ Educators point the finger at the practice supervisors, and vice versa. This is a long-standing problem.
Differences of opinion
The range of different opinions among professionals makes it difficult to assess students’ clinical reasoning skills in the classroom. When can the student be considered to have made the right choice of diagnosis or treatment method? It is evident that striving for consensus is fighting a losing battle. Lecturers therefore prefer not to burn their fingers on a test with close-ended questions, where there’s only one correct answer in every case. Instead they resort to solutions such as the use of case study tests or performance tests, which provide more freedom in terms of the possible answers, but which are also difficult to standardise. The Script Concordance Test is a test with close-ended questions which allows for differences of opinion, provided that these correspond to the range of opinions held across a panel of experts. The following video instruction provides more information about this.
With the support of SURFnet, I am developing, together with my colleagues from HAN University of Applied Sciences and Saxion, an online Script Concordance Test which enables students to see, immediately after taking the test, the extent to which their answer corresponds with the views of a panel of experts. The online test also makes it possible to use authentic audio/video material to illustrate health issues as they occur in practice. Collecting this material, incidentally, is quite challenging. The problem we run up against is that practicing physiotherapists have a hard time allowing other people to peek behind the scenes, while patients, in turn, are also reluctant to ‘expose themselves’ to outsiders.
When the test is ready, we will roll it out among students in the Bachelor’s programmes in Physiotherapy at HAN University of Applied Sciences and Saxion. I have also contacted the Royal Dutch Society for Physical Therapy (KNGF) to invite 600 physiotherapists in primary healthcare settings to complete the test as well. I hope that this will enable me to chart the differences among students, between students and professionals, and among professionals. The idea is to not view this range of opinions as a negative thing, but to use it to kick-start the discussion about desirable and undesirable differences in opinion when it comes to quality of the care provided to the patient. Hopefully this will enable us to understand each other better.
Innovation scheme: Digital Assessment for customised education
This is one of the 9 projects from the SURFnet ‘Digital Assessment for customised education’ innovation scheme. As part of this scheme, between 1 July 2015 and 1 July 2016 higher education institutions will be experimenting with the use of digital assessment in designing customised education, improving education quality and the fit between learning/teaching methods and the requirements of students and lecturers.