Covid-19 classification by FGCNet with deep feature fusion from graph convolutional network and convolutional neural network

Shui-Hua Wang,Vishnu Varthanan Govindaraj,Juan Manuel Górriz,Xin Zhang,Yu-Dong Zhang
DOI: https://doi.org/10.1016/j.inffus.2020.10.004
IF: 18.6
2021-03-01
Information Fusion
Abstract:<p>(<strong>Aim</strong>) COVID-19 is an infectious disease spreading to the world this year. In this study, we plan to develop an artificial intelligence based tool to diagnose on chest CT images. (<strong>Method</strong>) On one hand, we extract features from a self-created convolutional neural network (CNN) to learn individual image-level representations. The proposed CNN employed several new techniques such as rank-based average pooling and multiple-way data augmentation. On the other hand, relation-aware representations were learnt from graph convolutional network (GCN). Deep feature fusion (DFF) was developed in this work to fuse individual image-level features and relation-aware features from both GCN and CNN, respectively. The best model was named as FGCNet. (<strong>Results</strong>) The experiment first chose the best model from eight proposed network models, and then compared it with 15 state-of-the-art approaches. (<strong>Conclusion</strong>) The proposed FGCNet model is effective and gives better performance than all 15 state-of-the-art methods. Thus, our proposed FGCNet model can assist radiologists to rapidly detect COVID-19 from chest CT images.Keywords:Deep feature fusion; Convolutional neural network; Graph convolutional network; Multiple-way data augmentation; Batch normalization; Dropout; Rank-based average pooling</p>
computer science, artificial intelligence, theory & methods
What problem does this paper attempt to address?
The problem that this paper attempts to solve is to diagnose COVID - 19 by developing an artificial - intelligence - based tool using chest CT images. Specifically, the researchers proposed a new deep feature fusion technique (Deep Feature Fusion, DFF), which combines the features extracted by the convolutional neural network (Convolutional Neural Network, CNN) and the graph convolutional network (Graph Convolutional Network, GCN) to improve the accuracy of COVID - 19 classification. ### Main Problems and Solutions 1. **Problems**: - **Limitations of Single Feature Representation (SFR)**: Most existing COVID - 19 algorithms use a single feature representation and ignore the fusion of multiple feature representations (Multiple Feature Representations, MFRs). And MFRs can usually provide more abundant and accurate information, thereby improving classification performance. - **Feature Selection and Fusion**: How to effectively select and fuse features from different sources to avoid problems caused by high - dimensionality and possible distortion introduced during the feature fusion process. - **Classifier Selection**: How to select an appropriate classifier to ensure the efficiency and accuracy of the model in practical applications. 2. **Solutions**: - **Deep Feature Fusion (DFF)**: The researchers proposed the DFF technique for fusing the features extracted by CNN and GCN. CNN is used to extract individual image - level features (Individual Image - Level Representation, IIR), while GCN is used to extract relation - aware features (Relation - Aware Representation, RAR). These two types of features are fused at the feature level, improving the performance of the classifier. - **Improved CNN**: The researchers made several improvements to the traditional CNN, including batch normalization, Dropout, rank - based average pooling, and multi - way data augmentation, to improve the robustness and generalization ability of the model. - **Graph Convolutional Network (GCN)**: GCN is used to capture the relationships between images and extract relation - aware features. These features, together with the features extracted by CNN, are used for the final classification task. ### Experimental Results The experimental results show that the proposed FGCNet model outperforms 15 state - of - the - art methods in multiple metrics, demonstrating the effectiveness of the DFF technique. Specifically, FGCNet performs excellently in the COVID - 19 classification task and can help radiologists quickly and accurately detect COVID - 19 cases. ### Formula Summary - **Batch Normalization**: \[ \gamma_i'=\frac{\gamma_i-\mu_e}{\sqrt{\phi_e+\alpha_s}} \] \[ t_i = A_1\times\gamma_i'+A_2 \] - **Dropout**: \[ w_T(i, j)=\begin{cases} w(i, j)&\text{if }N(i, j)\in D\\ 0&\text{if }N(i, j)\notin D \end{cases} \] \[ w_I(i, j)=\alpha_{rp}\times w(i, j) \] - **Rank - based Average Pooling (RAP)**: \[ P_{RAP}(\Psi)=