22 May 2020

In the journal Radiology researchers from Radboudumc, Bernhoven Hospital and Jeroen Bosch Hospital described the evaluation of a newly developed artificial intelligence (AI) system, CAD4COVID-XRay, for the detection of COVID-19 characteristics on chest X-ray (CXR). 

The system is developed by Thirona and is available cost-free to any institution upon request.  The system was trained on 24,678 CXR images and tested on a set of 454 images from an independent center (223 patients tested positive for COVID-19, the remaining 231 tested negative).  The CXR images were also examined and scored by 6 radiologists independently.  Analysis was performed by calculation of the receiver operating characteristic curve (ROC).  In the task of identifying COVID-19 positive patients the AI system performed at a level comparable to all six radiologists and had an area under the ROC of 0.81.  The system may be useful as part of a triage process for symptomatic subjects, particularly in low-resource settings where radiological expertise is not available. 
Background Chest radiography (CXR) may play an important role in triage for COVID-19, particularly in low-resource settings. Purpose To evaluate the performance of an artificial intelligence (AI) system for detection of COVID-19 pneumonia on chest radiographs. Methods An AI system (CAD4COVID-Xray) was trained on 24,678 CXR images including 1,540 used only for validation while training. The test set consisted of a set of continuously acquired CXR images (n=454) obtained in patients suspected for COVID-19 pneumonia between March 4th and April 6th 2020 in a single center (223 RT-PCR positive subjects, 231 RT-PCR negative subjects). The radiographs were independently analyzed by six readers and by the AI system. Diagnostic performance was performed by receiver operating characteristic curve analysis. Results For the test set, the mean age of the patients was 67.3 (+/-14.4) years (56% male). Using RT-PCR test results as the reference standard, the AI system correctly classified CXR images as COVID-19 pneumonia with an AUC of 0.81. The system significantly outperforms each reader (p < 0.001 using McNemar test) at their highest possible sensitivities. At their lowest sensitivities, only one reader can significantly outperform the AI system (p=0.04). Conclusions An AI system for detection of COVID-19 on chest radiographs was comparable to six independent readers.

COVID-19 on the Chest Radiograph: A Multi-Reader Evaluation of an AI System.
Murphy K, Smits H, Knoops AJG, Korst MBJM, Samson T, Scholten ET, Schalekamp S, Schaefer-Prokop CM, Philipsen RHHM, Meijers A, Melendez J, van Ginneken B, Rutten M.


  • Want to know more about these subjects? Click on the buttons below for more news.

    Medical Imaging

Related news items

KWF grant for better selection of individuals and lung nodules in lung cancer screening

1 November 2021 The Dutch Cancer Society has awarded the consortium project ‘Multi-source data approach for Personalized Outcome Prediction in lung cancer screening’ with a grant of 1,425,000 Euro. Colin Jacobs will lead the work package on using artificial intelligence for accurate risk estimation of lung nodules. read more

Radiation boost lowers risk of prostate cancer recurrence

21 January 2021 An additional external-beam radiation dose delivered directly to the tumor can benefit the prospects of men with non-metastatic prostate cancer, without causing additional side effects. The risk of relapse within five years for these men is smaller than for men who did not receive this boost. read more

First podcast 'AI for Life' about Artificial Intelligence in Nijmegen Podcast on medical image processing with Bram van Ginneken, Ritse Mann and Eva van Rikxoort

16 July 2020 The first podcast on Artificial Intelligence (AI) in Nijmegen was released today. In this podcast on spotify and anchor Bram van Ginneken, Ritse Mann and Eva van Rikxoort talk about AI and medical image processing. The next podcast is about smart chatbots. read more

Is Google's lung cancer AI ready for implementation?

19 December 2019 In Nature Reviews Clinical Oncology, Colin Jacobs and Bram van Ginneken discussed the recent paper by Google AI in Nature Medicine on deep learning models for automatic detection of lung cancer from computed tomography. read more

Awarded KWF grants for Radboudumc researchers

18 December 2019 KWF is investing 2.7 million euros in five different studies at Radboudumc. The awards are part of the new round of funding by DCS, in which over 34 million euros will be granted to Dutch cancer research. We congratulate our researchers with this funding and wish them success with their great work. read more

Making pregnancy ultrasound available in developing countries

24 January 2019 Many pregnant women in developing countries do not receive an ultrasound examination. As a result, pregnancy risks often go unnoticed, which contributes to the high maternal mortality rates in these countries. read more