Detection and Localization of Pneumonia using Mask Region Based

Abstract

Keywords: Pneumonia Detection, Densely Connected Neural Network, Mask Region based CNN

Introduction

Winter fever or Pneumonia; an acute respiratory tract infection, that inflames air sacs in one or both lungs, is mainly identified by fever, coughing and dyspnea (Duthey et al., 2013). A certain group of people is at the highest risk of getting this infection and they mainly include children under the age of 2 and adults older than 65 years. In the United States alone, about 1 million adults are hospitalized with Pneumonia andalmost 50,000 dies from the same. Treatments for Pneumonia is based on what kind of germ is causing the infection along with the severity of the symptoms and antibiotics, cough medicine, fever reducers are the most common means of treatments.

In order to diagnose this infection, the doctor will ask questions about symptoms and health history. Certain tests, including chest x-ray, blood tests, and sputum (phlegm) culture, helps in diagnosing Pneumonia. Among them, Chest X-rays provide detailed images of the lungs and is one of the most feasible tests used in diagnosis. On interpreting the X-rays, the radiologist will look for white spots on the lungs, which are known as infiltrates that indicates the presence of an infection. Even though this is the best means to diagnose Pneumonia, a radiologist may fail to precisely spot the same due to its vague appearance, and similarities with any other benign abnormalities such as lung cancer. So, it will be really beneficial if there are some means to automate the detection of Pneumonia.

Most of the existing Pneumonia detection methods use deep learning techniques (LeCun et al., 2015) which make use of pre-trained networks that first extract features and then classifies them as infected or not. In this paper, we make use of two separate networks in which the first one uses a Densely Connected Convolutional Neural Network which will take an input sample and detect whether it is infected with pneumonia or not. So, it can be viewed as a binary classification problem which results in either 0 or 1. The second network works only if the first result is positive and it uses a novel technique made of Mask Region based Convolutional Neural Network, to locate the areas of infection.

We focus our experimental analysis on the kaggle dataset using the well defined training and testing splits. Our contributions in this work are three fold:

• We propose a fully automatic method to detect Pneumonia from chest X-ray.
• To identify whether the sample is infected or not a Densely Connected Convolutional Neural Network (DenseNet) is used.
• To segment a lung region and then to find the areas infected, a novel method that make use of Mask R-CNN is used with an accuracy greater than existing methods.

Section 2 describes an overview about pneumonia and its diagnosis methods, along with a general idea about various localization networks and Section 3 gives a detailed description about the proposed method, followed by the results and other discussions.

Preprint submitted to Journal of LATEX Templates May 9, 2019

Related Work

Pneumonia; a bacterial, viral, or fungal infection of the lungs causes the air sacs of the lungs to fill up with fluid or pus and thus making breathing painful and also limits oxygen intake. Figure 1 illustrates a typical case of Pneumonia infection showing (a) normal and (b) infected alveoli. The infected regions are swollen with the presence of fluid within them.

• Based on how a person gets infection there are different categories of Pneumonia:
• Community-Acquired Pneumonia (CAP). It is the most common type of Pneumonia caused by pneumococcus bacteria. (Franquet et al., 2018).
• Hospital - Acquired Pneumonia (HAP). HAP is when people catch Pneumonia during a hospital stay for another illness. It is more serious than CAP because you’re already sick.
• Atypical Pneumonia. Atypical Pneumonia is a type of CAP, Caused by lung infections with less common bacteria than the pneumococcus bacteria. Atypical bacteria include Legionella pneumophila, Mycoplasma Pneumoniae, or Chlamydia Pneumoniae and other protozoa.
• Aspiration Pneumonia. This type of Pneumonia can occur during inhaling food, drink, vomit, or saliva from mouth into your lungs. It is dominant form of community-acquired Pneumonia (CAP) and healthcare-associated Pneumonia.

Pioneer work on Pneumonia can be traced back to BC 460 by Hippocrates and he described the symptoms of the same in his work Pneumonia strikes. Succeeding in 1875 German scientist Edwin Klebs observed bacteria responsible for Pneumonia under the microscope and it was a break-through in the fight against pulmonary diseases. Sir William Osler described Pneumonia as the captain of men of death in 1918 as it had overtaken tuberculosis as one of the leading cause of death. With scientific advancements, there are several methods that could automatically detect pneumonia from the chest X-ray images which are the most commonly used diagnostic tool.

Approaches to Pneumonia Detection

Pneumonia detection from chest radiographs can be roughly classified into two categories: those based on image processing and those based on deep neural networks. Image processing method focused on Otsu thresholding (Sharma et al., 2017) which could detect the non-cloudy region of the lung and its area. The ratio of areas of this noncloudy lung region to the total area of the extracted lung region would give an indication of the amount of cloud formation in the lungs. Even though it was a simple method, the accuracy was very less and thus making radiologists to think about some other alternatives.

Another approach to Pneumonia detection employs Deep neural network models. Thoracic diseases including Pneumonia were identified and localized using weakly supervised multi-label image classification and pathology localization framework, that makes use of Deep Convolutional Neural Network (DCNN) architecture as its base (Wang et al., 2017). A combination of deep activations from the transition layer and the weights of the prediction inner-product layer can enable to find the plausible spatial locations of diseases. Later, a 121-layer convolutional neural network was used specifically to detect Pneumonia that takes a chest X-ray image as input and outputs the probability of Pneumonia along with a heatmap localizing the areas of the image most indicative of Pneumonia (Rajpurkar et al., 2017). There are certain network models which are purely dedicated for segmentation task. So, our focus here is using such a model for segmenting lung regions and thus locating areas of infection.

Localization Network

R-CNN

The Region-based CNN (R-CNN) approach on object detection tasks having bounding boxes is to generate some manageable number of object regions and evaluate them using traditional convolutional networks(CNN) independently on each Region of Interests. R-CNN was extended so that they can be work on RoIs on feature maps using RoIPool, which leads to fast speed and better accuracy.

Fast R-CNN

Fast R-CNN(Girshick et al., 2015) is an advanced form of R-CNN, and it improves its speed by feeding input image to CNN instead of feeding the region proposals to the CNN and generate a convolutional feature map. From this convolutional feature map, it is possible to identify the region of proposals.

Faster R-CNN

Both of the above algorithms(R-CNN Fast R-CNN) uses selective search to find out the region proposals. But, Selective search is a slow and time-consuming process affecting the performance of the network. In order to eliminate this drawback, Faster R-CNN was introduced that uses object detection algorithm which eliminates the selective search algorithm and lets the network learn the region proposals.

Faster R-CNN operates in two stages. The first stage, Region Proposal Network (RPN), proposes candidate object bounding boxes. The second stage extracts feature using RoIPool from each candidate box and perform classification as well as bounding-box regression. The features used by both stages are shared for faster inference. Figure 2 shows a faster RCNN network in which image is provided as an input to a convolutional network that generates convolutional feature map. A distinct network is used to predict the region proposals. The predicted regions arу reshaped using a RoI pooling layer which is used to classify the image within the proposed region and predict the offset values for the bounding boxes.

MaskRCNN

Mask R-CNN (He et al., 2017) is again an extension of Faster R-CNN that can carry out pixel level segmentation. Mask R-CNN does this by adding a branch to Faster R-CNN that outputs a binary mask that says whether or not a given pixel is part of an object.So, a better way to think about Mask RCNN is as a combination of Faster R-CNN that does object detection and FCN (Fully Convolutional Network) that does pixel-wise boundary. Figure 3 shows a Mask RCNN framework with Faster R-CNN and FCN. As illustrated by the figure, Faster RCNN has two outputs for each sample, a class label and a bounding-box offset; to which there is an additional output namely the object mask, which is a binary mask that indicates the pixels where the object is in the bounding box. For doing so Mask R-CNN uses the Fully Convolution Network (FCN) which is a popular algorithm for doing semantic segmentation.

References

1. Duthey, B., 2013. Priority medicines for europe and the world:a public health approach to innovation. WHO Background paper, 6.
2. LeCun, Yann, Yoshua Bengio, and Geoffrey Hinton. ”Deep learning.” nature 521.7553 (2015): 436.
3. Boyd-Graber, Jordan, et al. ”Adding dense, weighted connections to WordNet.” Proceedings of the third international WordNet conference. 2006.
4. He, K., Gkioxari, G., Dollr, P. and Girshick, R., 2017. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision (pp. 2961-2969).
5. Franquet, T., 2018. Imaging of community-acquired Pneumonia. Journal of thoracic imaging, 33(5), pp.282-294.
6. Rajpurkar, Pranav, Hannun, Awni Y, Haghpanahi, Masoumeh, Bourn, Codie, and Ng, Andrew Y. Cardiologist-level arrhythmia detection with convolutional neural networks. arXiv preprint arXiv:1707.01836, 2017.
7. Grewal, Monika, Srivastava, Muktabh Mayank, Kumar, Pulkit, and Varadarajan, Srikrishna. Radnet: Radiologist level accuracy using deep learning for hemorrhage detection in ct scans. arXiv preprint arXiv:1710.04934, 2017.
8. Lakhani, Paras and Sundaram, Baskaran. Deep learning at chest radiography: Automated classification of pulmonary tuberculosis by using convolutional neural networks. Radiology, pp. 162326, 2017.
9. Huang, Peng, Park, Seyoun, Yan, Rongkai, Lee, Junghoon, Chu, Linda C, Lin, Cheng T, Hussien, Amira, Rathmell, Joshua, Thomas, Brett, Chen, Chen,
   et al. Added value of computer-aided ct image features for early lung cancer diagnosis with small pulmonary nodules: A matched case-control study. Radiology, pp. 162725, 2017.
10. Sharma, Abhishek, Daniel Raju, and Sutapa Ranjan. ”Detection of Pneumonia clouds in chest X-ray using image processing approach.” 2017 Nirma University
    International Conference on Engineering (NUiCONE). IEEE, 2017.
11. Wang, Xiaosong, et al. ”Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly supervised classification and localization of common
    thorax diseases.” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017.
12. Rajpurkar, Pranav, et al. ”Chexnet: Radiologist level Pneumonia detection on chest x-rays with deep learning.” arXiv preprint arXiv:1711.05225 (2017).
13. Girshick, Ross. ”Fast r-cnn.” Proceedings of the IEEE international conference on computer vision. 2015.
14. Zagoruyko, Sergey, and Nikos Komodakis. ”Wide residual networks.” arXiv preprint arXiv:1605.07146 (2016).
15. Dollr, Piotr, et al. ”Fast feature pyramids for object detection.” IEEE Transactions on Pattern Analysis and Machine Intelligence 36.8 (2014): 1532-1545.
16. Huang, Gao, et al. ”Densely connected convolutional networks.” Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.
17. Mildenberger, Peter, Marco Eichelberg, and Eric Martin. ”Introduction to the DICOM standard.” European radiology 12.4 (2002): 920-927.
18. He, Kaiming, et al. ”Deep residual learning for image recognition.” Proceedings of the IEEE conference on computer vision and pattern recognition. 2016.
19. Romera-Paredes, Bernardino, and Philip Hilaire Sean Torr. ”Recurrent instance segmentation.” European conference on computer vision. Springer, Cham, 2016.