Convolutional neural network models for high spatial resolution satellite imagery classification
Mingfei Zhou,Xili Wang,Lei Wang,Fen Chen
DOI: https://doi.org/10.11834/jig.160627
2017-01-01
Journal of Image and Graphics
Abstract:Objective Satellite imagery classification is a task that uses classification models to divide a set of satellite images into several classes.The satellite images discussed in this paper are collected from the Quickbird satellite imagery dataset.Satellite images are divided into six classes,namely,airplanes,dense residential areas,harbors,intersections,overpasses,and parking lots.Generally,the task of satellite imagery classification is difficult because of the complex targets and backgrounds in satellite images.Traditional methods,such as the artificial neural networks and support vector machines,usually use low-level and manually selected features.These features are insufficient and cannot represent the multilevel and intrinsic features of satellite images.Simultaneously,obtaining high accuracy is difficult through the classification methods,which use low-level features.Some deep learning methods use pre-trained convolutional neural networks to extract the high-level features of satellite images and some classifier to classify satellite images.These methods can improve their performance than the traditional methods.However,these methods ignore the inherent classification capability of convolutional neural networks because considerable labeled training data of satellite images are required to train a convolutional neural network,which could extract features and classify images simultaneously;however,training data are limited in practice.Other methods use a stack of shallow convolutional neural networks to classify satellite images.However,the stack of low-level features remains insufficient representative to substantially improve the classification accuracy of satellite image.In this paper,a new approach using deep convolutional neural networks is presented to improve the classification accuracy for satellite imagery.The classification accuracy of satellite images could be improved using the deep features extracted by convolutional neural networks.Method An end-to-end training and classification method is proposed.This method does not require additional classifiers and stack of shallow convolutional neural networks to improve the capability of feature extraction from satellite images.First,a new satellite imagery dataset,which contains six classes,is proposed to deal with the problem of lacking labeled training data.Second,three kinds of pre-trained deep convolutional neural network models and a directly trained shallow convolutional neural network model are used to perform the classification task for satellite images.The shallow model has low training weights and can be trained directly on the satellite image dataset to classify satellite images.The proposed three kinds of deep models should be pre-trained on an auxiliary dataset,because the amount of the training weights of the three deep models is too large to be directly trained on the proposed satellite images dataset.The three kinds of deep models are pre-trained on a large auxiliary dataset,which contains roughly 1 200 000 labeled training images of 1 000 classes.All of the images contain the common objects,which could be viewed everywhere in the daily life.The weights of the three deep architectures of convolutional neural networks can be trained adequately after pre-training on the large auxiliary dataset.The capability of the deep models to extract representative features and to classify images can be improved after pre-training,and the application objects of the models can be transferred from the daily common objects to the satellite image objects.The key point of such transformation is fine-tuning the pre-trained deep models on the proposed satellite image dataset.The architectures of the three deep models should be changed slightly and then they could be finetuned on the proposed dataset.After fine-tuning,the three deep convolutional neural network models could be used to classify satellite images directly without using other classifiers or stacked shallow models.Result The proposed convolutional neural network models are validated on two datasets.One of the testing satellite images datasets is the proposed dataset,and the other dataset is the famous UC Merced land use dataset.The four proposed models demonstrate high performance on the proposed dataset.The classification accuracies of the three deep models are higher than the accuracy of the shallow odel.In particular,the deepest convolutional neural network model achieves the highest accuracy of 99.50％ on the proposed dataset.The results of the UC Merced land use dataset of the three similar methods in the literature are compared with the results of the proposed models.Two of the three comparative methods use the features extracted from the pretrained convolutional neural networks without fine-tuning and use an additional classifier to classify satellite images.The other method uses a multi-view convolutional neural network to perform the classification task.Experimental results indicate that the proposed deep models in this paper achieve the highest accuracy (96.44％) among all the models.Conclusion This paper proposes a new satellite image dataset,which is representative of satellite images.Convolutional neural networks could be trained adequately through the proposed satellite image dataset.Shallow convolutional neural networks can possibly be directly trained on this dataset.Pre-trained convolutional neural networks obtain better classification accuracy on other satellite imagery dataset after fine-tuning on the proposed dataset.Furthermore,the proposed deep convolutional neural network models are effective in terms of deep feature extraction and satellite image classification.The proposed models obtain more competitive results than other reported methods in literature.The proposed deep models exhibit good generalization capability and could achieve high accuracy on the UC Merced land use dataset,in which images differ from those in the finetuning dataset in terms of scale and quality.The effective pre-training and fine-tuning with the depth of the proposed deep models contribute to the good performance of the models.In addition,the proposed models are end-to-end models.Additional classifiers and stack of the shallow models are not required to classify satellite images.