Deep learning algorithm outperforms physicians detecting active pulmonary TB

Bradley van Paridon
Jan. 9, 2019, 9:27 p.m.

A deep-learning algorithm performed better than radiologists at detecting active pulmonary tuberculosis (TB) on chest radiographs, according to data published in Clinical Infectious Diseases.

The deep-learning-based automatic detection (DLAD) algorithm was developed using 54,221 normal chest radiographs and 6768 chest radiographs with active TB that had been previously labeled and annotated by 13 board-certified radiologists. The deep-learning-based automatic detection algorithm was evaluated using 6 external multicenter, multinational data sets. The performances of the deep-learning-based automatic detection algorithm and physicians were compared using an observer performance test conducted by 15 physicians including physicians with specialties outside of radiology, board-certified radiologists, and thoracic radiologists.

The algorithm demonstrated imagewise classification performance, based on area under the receiver operating characteristic (AUROC) curves of 0.977 to 1.000. The lesion-wise localization performance, determined using area under the alternative free response receiver operating characteristic curves, was 0.973 to 1.000. The sensitivity for classification was 94.3% to 100% and 91.1% to 100%, using a high-sensitivity cutoff of 98%. Specificities for classification were 84.1% to 99.0% and 99.1% to 100%, using a high-specificity cutoff of 98%. The deep-learning-based automatic detection algorithm performed significantly better compared with all groups of physicians in both classification (AUROC 0.993 vs 0.746-0.971, respectively) and localization (AUROC 0.993 vs 0.664-0.925, respectively).

The study investigators highlighted several limitations to consider. The validation of deep-learning-based automatic detection algorithm was performed using retrospectively collected data sets, which is not identical to the real-world setting, as various abnormalities may be present and the prevalence of TB might be lower than the validation settings. In addition, reference standards for the development of data sets were defined by radiologists and not a more definitive measure such as chest computed tomography images. In addition, the algorithm was designed to identify active TB, and other TB manifestations or significant thoracic diseases were not considered. Therefore, it is unknown how well the deep-learning-based automatic detection algorithm can differentiate between active TB and these other manifestations, nor its ability to detect other potential abnormalities. Study investigators advised that future work dealing with these scenarios is warranted.

The study authors hypothesized that the deep-learning-based automatic detection algorithm has 2 potential clinical applications based on its "excellent and consistent performance" in the detection of active pulmonary TB on chest radiograph. The first was that the algorithm, acting as a second reader of chest radiographs, may improve physician performance, especially in primary healthcare or community-based settings. The second was that the results suggested that deep-learning-based automatic detection algorithm has the potential to be a stand-alone option for active TB screening or in triaging chest radiographs for further screening.

Disclosure

This work received funding from Lunit Inc and the Seoul Research & Business Development Program (grant number FI170002).

Reference

Hwang EJ, Park S, Jin KN, et al. Development and validation of a deep learning-based automatic detection algorithm for active pulmonary tuberculosis on chest radiographs [published online November 12, 2018]. Clin Infect Dis. doi: 10.1093/cid/ciy967


Source: Infectious Disease Advisor