Skip to main page content
Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2017 Apr 19;7:46479.
doi: 10.1038/srep46479.

Towards Automatic Pulmonary Nodule Management in Lung Cancer Screening With Deep Learning

Free PMC article

Towards Automatic Pulmonary Nodule Management in Lung Cancer Screening With Deep Learning

Francesco Ciompi et al. Sci Rep. .
Free PMC article

Erratum in


The introduction of lung cancer screening programs will produce an unprecedented amount of chest CT scans in the near future, which radiologists will have to read in order to decide on a patient follow-up strategy. According to the current guidelines, the workup of screen-detected nodules strongly relies on nodule size and nodule type. In this paper, we present a deep learning system based on multi-stream multi-scale convolutional networks, which automatically classifies all nodule types relevant for nodule workup. The system processes raw CT data containing a nodule without the need for any additional information such as nodule segmentation or nodule size and learns a representation of 3D data by analyzing an arbitrary number of 2D views of a given nodule. The deep learning system was trained with data from the Italian MILD screening trial and validated on an independent set of data from the Danish DLCST screening trial. We analyze the advantage of processing nodules at multiple scales with a multi-stream convolutional network architecture, and we show that the proposed deep learning system achieves performance at classifying nodule type that surpasses the one of classical machine learning approaches and is within the inter-observer variability among four experienced human observers.

Conflict of interest statement

Colin Jacobs received a research grant from MeVis Medical Solutions AG, Bremen, Germany. Bram van Ginneken receives research support from MeVis Medical Solutions and is co-founder and stockholder of Thirona.


Figure 1
Figure 1. Examples of triplets of patches for different nodule types in axial, coronal and sagittal views.
Each triplet is depicted using three different patch sizes, namely 10 mm, 20 mm and 40 mm.
Figure 2
Figure 2. Examples of classified nodules from the test set (DLCST).
Each row depicts nodules from one class as labeled in the DLCST trial, and nodules are sorted from left to right based on the probability given by the (3-scale) deep learning system. Examples with low probability (on the left) are a-typical cases of each nodule type, while a high probability (on the right) is given to typical examples of each nodule type.
Figure 3
Figure 3. Multidimensional scaling of nodules in the test set using the t-SNE algorithm.
Close nodules have similar characteristics. In (a), clusters of similar nodules are highlighted and grouped with different boxes. A zoomed-in version of each cluster is also shown and a representative name is given based on their appearance. The nodule label assigned in the DLCTS trial is also reported as a coloured dot for each nodule patch (see legend for nodule types).
Figure 4
Figure 4
(a) Examples of triplets of nodules extracted by varying the parameter N. (b) Examples of pyramidal triplets of patches used to feed the proposed deep learning systems. The system consists of three groups of three streams, one for each considered scale (namely 10 mm, 20 mm and 40 mm for patch size). Convolutional layers, max-pooling layers, fully-connected layers and one soft-max layer are the building blocks of the proposed network. The last fully-connected layer with 256 neurons serves as a combiner of the three sets of three streams, and a 6-value probability vector is generated as output.

Similar articles

See all similar articles

Cited by 34 articles

See all "Cited by" articles


    1. Aberle D. R. et al. . Reduced lung-cancer mortality with low-dose computed tomographic screening. New England Journal of Medicine 365, 395–409 (2011). - PMC - PubMed
    1. de Koning H. J. et al. . Benefits and harms of computed tomography lung cancer screening strategies: A comparative modeling study for the U.S. preventive services task force. Annals of Internal Medicine (2013). - PMC - PubMed
    1. Messay T., Hardie R. C. & Rogers S. K. A new computationally efficient CAD system for pulmonary nodule detection in CT imagery. Medical Image Analysis 14, 390–406 (2010). - PubMed
    1. Jacobs C. et al. . Automatic detection of subsolid pulmonary nodules in thoracic computed tomography images. Medical Image Analysis 18, 374–384 (2014). - PubMed
    1. Setio A. A. A., Jacobs C., Gelderblom J. & van Ginneken B. Automatic detection of large pulmonary solid nodules in thoracic CT images. Medical Physics 42, 5642–5653 (2015). - PubMed

Publication types

MeSH terms