Research outlines need for improvement with online symptom checkers
Online medical symptom checker tools only give the correct diagnosis first in 37.7% of cases, and do worse when users require urgent attention, according to new research from Doctorlink, an NHS supplier of digital medical consultations.
The report explores the wide variation in efficacy across several digital symptom checker tools available in the market today. These tools analyse a user’s symptoms to triage patients, provide a diagnosis, and suggest the most appropriate treatment pathway.
When built and deployed properly, these tools can address some of the major challenges facing medical professionals today, including staff shortages and long wait times for treatment. However, these benefits can have the opposite effect if a symptom checking system calculates a diagnosis that is not appropriate or does not suggest an appropriate care pathway.
The peer-reviewed research, which set out to test symptom checker tools already available on the market, was conducted by specialist data consultancy, Methods Analytics. Researchers ran a series of 50 clinical cases through twelve publicly available symptom checkers to test their efficacy. The overall result showed significant room for improvement, with the correct diagnosis being present in the top five diagnoses in 51% of cases.
Dr Ben Littlewood-Hillsden, chief medical officer of Doctorlink, said: “With the onset of the pandemic, the adoption of digital health tools, including online symptom checkers, has grown dramatically – and that is a positive development in terms of what such tools can bring to both the patient experience and clinical capacity. The challenge for online symptom checkers, which this report clearly highlights, is the fact it is a fragmented area of the healthtech market. Whilst intelligent tools are indeed helping some of the UK’s 3,000 or so clinical centres to diagnose and triage their patients accurately and safely, others exist in the market that are unable to do so - and some even claim to use AI when they do not.
“Healthcare is a high stakes industry, with clinical teams being responsible for the lives of their patients. This is an argument in itself for why we must mandate the use of proper tools that provide consistently accurate results. These have the potential to be transformational, but they simply aren’t at the moment because many are just not sophisticated enough to be truly fit-for-purpose.”
Three major findings of the report are:
- Both diagnostic accuracy and appropriate resource recommendation varied significantly between tools: the percentage of times in which the correct diagnosis was listed first ranged from 22% to 72%. Accuracy of recommended approach to care, meanwhile, ranged from 36% to 90%. Patients have no way of knowing where the symptom tracker they are using falls on this broad spectrum.
- Current tools largely increase, rather than reduce, use of healthcare resources: additional resource use was suggested in over half of cases with primary care resources being suggested in 80.8% of conditions that could be self-managed.
- The accuracy of diagnosis fell in line with the urgency of the case: safety of disposition advice, which means identifying the right person, right clinician and right timeframe, decreased with condition urgency (being 71.8% for emergency cases and 87.3% for non-urgent cases.
Ranjan Singh, CEO and co-founder of HealthHero, Europe’s largest digital telehealth platform and parent company of Doctorlink, says: “More of us than ever before are using online symptom checkers – but we must be able to trust them, and they must offer real benefit to clinicians. At the moment, such tools are proliferating but their overall performance is significantly below what would be accepted in any other medical field – despite the fact that there are outliers that do consistently provide very accurate results. What’s important is that we continue to test these tools with rigour and transparency, because they have the potential to dramatically increase the efficiency of our healthcare system.
“For that to happen, validation and regulation must urgently be put in place. Routine audits to make sure that they remain faithful to current clinical guidelines is important to ensure that both clinicians and patients are able to trust that their results are accurate. Highly effective tools do exist, as evidenced by this data, but we risk ineffective alternatives having the converse effect on clinical resources if they are not improved.”