Traditional methods are often ill-suited to the rapidly evolving world of health care research, characterised by data volume, complexity and pace. Machine learning offers an opportunity to address challenges in all facets of health research but is often subject to bias which limits its use.
A team of digital health researchers, led by Annette O’Connor and including CRE Chief Investigator Professor Paul Glasziou, recently investigated building an evidence base to gain trust in systematic review automation technologies.
Their paper discusses the potential reasons for slow adoption of machine learning tools to assist with conducting systematic reviews. They explore how the needs of researchers need to be addressed if they are to have trust in automation tools, confidence in their precision and value, and comfortableness with their compatibility with established workflows and tasks.
The assessment of automation tools presents unique challenges for investigators and systematic reviews, including the need to clarify which metrics are of interest to the systematic review community and the unique documentation challenges for reproducible software experiments. The findings highlight the need for tool developers to have guidance in how to design tools and report tool evaluations in ways that end users can assess their validity and suggest approaches to formatting and announcing publicly available datasets suitable for assessment of automation technologies. Making these resources available will increase trust and increase the changes of uptake.
You can read the full paper here and access the IEBH SR-Accelerator tools which speeds up steps in the Systematic Review (SR) process.
Find out more about the work of CRE partner Bond University and the work of the Centre for Evidence Based Healthcare here.
O’Connor, A. M., Tsafnat, G., Thomas, J., Glasziou, P., Gilbert, S. B., & Hutton, B. (2019). A question of trust: can we build an evidence base to gain trust in systematic review automation technologies? Systematic Reviews, 8(1), 143. [143]. https://doi.org/10.1186/s13643-019-1062-0
Aug 30, 2019