GE Hinton Imagenet classification with deep convolutional neural networks., 2012, 25

A Krizhevsky, I Sutskever

2024-01-01

Abstract:

What problem does this paper attempt to address?

ImageNet classification with deep convolutional neural networks

Alex Krizhevsky,Ilya Sutskever,Geoffrey E. Hinton

DOI: https://doi.org/10.1145/3065386

IF: 22.7

2017-05-24

Communications of the ACM

Abstract:We trained a large, deep convolutional neural network to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes. On the test data, we achieved top-1 and top-5 error rates of 37.5% and 17.0%, respectively, which is considerably better than the previous state-of-the-art. The neural network, which has 60 million parameters and 650,000 neurons, consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully connected layers with a final 1000-way softmax. To make training faster, we used non-saturating neurons and a very efficient GPU implementation of the convolution operation. To reduce overfitting in the fully connected layers we employed a recently developed regularization method called "dropout" that proved to be very effective. We also entered a variant of this model in the ILSVRC-2012 competition and achieved a winning top-5 test error rate of 15.3%, compared to 26.2% achieved by the second-best entry.

computer science, theory & methods, software engineering, hardware & architecture
Research on Image Classification Method of Features of Combinatorial Convolution

Chao Wu,Yaqian Li,Zhibiao Zhao,Bin Liu

DOI: https://doi.org/10.1007/s12652-019-01433-9

IF: 3.662

2019-01-01

Journal of Ambient Intelligence and Humanized Computing

Abstract:In image classification, shallow convolutional features and deep convolutional features are not fully utilized by many network frameworks. To solve this problem, we propose a combinatorial convolutional network (CCNet) that integrates convolutional features of all levels. According to its own structure, the convolutional features of shallow, medium, and deep levels are extracted. These features are combined by weighted concatenation and convolutional fusion, and the coefficients of each channel of final combination feature are again weighted to improve the identification degree of features. CCNet can improve the single case where most network only add or concatenate shallow and deep features, so that the network can achieve lower classification error rate while generating low-dimensional features. Extensive experiments are performed on CIFAR-10 and CIFAR-100 respectively. The experimental results show that the low-dimensional image feature vectors generated by CCNet effectively reduce the classification error rate when the number of convolutional layers does not exceed 100 layers.
Deep Tessellated Retinal Image Detection Using Convolutional Neural Networks

Xingzheng Lyu,Hai Li,Yi Zhen,Xin Ji,Sanyuan Zhang

DOI: https://doi.org/10.1109/EMBC.2017.8036915

2017-01-01

Abstract:Tessellation in fundus is not only a visible feature for aged-related and myopic maculopathy but also confuse retinal vessel segmentation. The detection of tessellated images is an inevitable processing in retinal image analysis. In this work, we propose a model using convolutional neural network for detecting tessellated images. The input to the model is pre-processed fundus image, and the output indicate whether this photograph has tessellation or not. A database with 12,000 colour retinal images is collected to evaluate the classification performance. The best tessellation classifier achieves accuracy of 97.73% and AUC value of 0.9659 using pretrained GoogLeNet and transfer learning technique.
Research on image classification leveraging deep convolutional neural networks and visual cognition

Chen Liu

DOI: https://doi.org/10.54254/2755-2721/32/20230212

2024-01-31

Abstract:The field of image classification has experienced remarkable improvements with the advent of deep learning techniques, especially Deep Convolutional Neural Networks. The present study provides an extensive exploration of the junction where image classification based on Deep Convolutional Neural Networks meets human visual cognition. Utilizing the inherent ability of these networks to automatically learn hierarchical features from raw pixel data, this research examines their potential in classifying images from diverse complex datasets, emphasizing predominantly on the extensively utilized ImageNet dataset. The initial aspect of this study involves training and evaluating models based on Deep Convolutional Neural Networks on the ImageNet dataset, which comprises millions of labeled images spanning across thousands of categories. Well-established network architectures such as AlexNet, VGGNet, GoogLeNet, and ResNet are employed, and their performance in the challenging task of image classification is assessed. Rigorous experiments highlight the strengths and weaknesses of each model while emphasizing the prospects of transfer learning and fine-tuning. Following this, the interpretability of Deep Convolutional Neural Networks is explored by using visualization techniques to comprehend the learned feature representations. By visualizing activation maps and class-specific saliency maps, valuable insights are gained into the regions of interest that guide the decision-making of these models. Moreover, the correlation between the features extracted by these models and human visual attention mechanisms is examined to shed light on the focus of attention of the models. The study also addresses the difficulties that adversarial attacks, data bias, and generalization capabilities present to Deep Convolutional Neural Networks. Strategies to enhance the robustness and adaptability of the models across various domains are examined, linking these observations to human cognitive behavior.
Going deeper with convolutions

Wei Liu,Christian Szegedy,Dragomir Anguelov,P. Sermanet,Vincent Vanhoucke,D. Erhan,Yangqing Jia,Scott E. Reed,Andrew Rabinovich

DOI: https://doi.org/10.1109/CVPR.2015.7298594

2014-09-16

Computer Vision and Pattern Recognition

Abstract:We propose a deep convolutional neural network architecture codenamed Inception that achieves the new state of the art for classification and detection in the ImageNet Large-Scale Visual Recognition Challenge 2014 (ILSVRC14). The main hallmark of this architecture is the improved utilization of the computing resources inside the network. By a carefully crafted design, we increased the depth and width of the network while keeping the computational budget constant. To optimize quality, the architectural decisions were based on the Hebbian principle and the intuition of multi-scale processing. One particular incarnation used in our submission for ILSVRC14 is called GoogLeNet, a 22 layers deep network, the quality of which is assessed in the context of classification and detection.

Computer Science
SwGridNet: A Deep Convolutional Neural Network based on Grid Topology for Image Classification

Atsushi Takeda

DOI: https://doi.org/10.48550/arXiv.1709.07646

2017-10-03

Abstract:Deep convolutional neural networks (CNNs) achieve remarkable performance on image classification tasks. Recent studies, however, have demonstrated that generalization abilities are more important than the depth of neural networks for improving performance on image classification tasks. Herein, a new neural network called SwGridNet is proposed. A SwGridNet includes many convolutional processing units which connect mutually as a grid network where many processing paths exist between input and output. A SwGridNet has high generalization capability because the multipath architecture has the same effect of ensemble learning. As described in this paper, details of the SwGridNet network architecture are presented. Experimentally obtained results presented in this paper show that SwGridNets respectively achieve test error rates of 2.95% and 15.67% in a CIFAR-10 and CIFAR-100 classification tasks. The results indicate that the SwGridNet performance approximates that of state-of-the-art deep CNNs.

Computer Vision and Pattern Recognition
Evaluation of Deep Convolutional Nets for Document Image Classification and Retrieval

Adam W. Harley,Alex Ufkes,Konstantinos G. Derpanis

DOI: https://doi.org/10.48550/arXiv.1502.07058

2015-02-25

Abstract:This paper presents a new state-of-the-art for document image classification and retrieval, using features learned by deep convolutional neural networks (CNNs). In object and scene analysis, deep neural nets are capable of learning a hierarchical chain of abstraction from pixel inputs to concise and descriptive representations. The current work explores this capacity in the realm of document analysis, and confirms that this representation strategy is superior to a variety of popular hand-crafted alternatives. Experiments also show that (i) features extracted from CNNs are robust to compression, (ii) CNNs trained on non-document images transfer well to document analysis tasks, and (iii) enforcing region-specific feature-learning is unnecessary given sufficient training data. This work also makes available a new labelled subset of the IIT-CDIP collection, containing 400,000 document images across 16 categories, useful for training new CNNs for document analysis.

Computer Vision and Pattern Recognition,Information Retrieval,Machine Learning,Neural and Evolutionary Computing
Very Deep Convolutional Networks for Large-Scale Image Recognition

K. Simonyan,Andrew Zisserman

2014-09-04

Abstract:In this work we investigate the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting. Our main contribution is a thorough evaluation of networks of increasing depth using an architecture with very small (3x3) convolution filters, which shows that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 weight layers. These findings were the basis of our ImageNet Challenge 2014 submission, where our team secured the first and the second places in the localisation and classification tracks respectively. We also show that our representations generalise well to other datasets, where they achieve state-of-the-art results. We have made our two best-performing ConvNet models publicly available to facilitate further research on the use of deep visual representations in computer vision.

Computer Science,Engineering
Deep Convolution Neural Networks in Computer Vision: a Review

Hyeon-Joong Yoo

DOI: https://doi.org/10.5573/ieiespc.2015.4.1.035

2015-02-28

IEIE Transactions on Smart Processing and Computing

Abstract:Over the past couple of years, tremendous progress has been made in applying deep learning (DL) techniques to computer vision. Especially, deep convolutional neural networks (DCNNs) have achieved state-of-the-art performance on standard recognition datasets and tasks such as ImageNet Large-Scale Visual Recognition Challenge (ILSVRC). Among them, GoogLeNet network which is a radically redesigned DCNN based on the Hebbian principle and scale invariance set the new state of the art for classification and detection in the ILSVRC 2014. Since there exist various deep learning techniques, this review paper is focusing on techniques directly related to DCNNs, especially those needed to understand the architecture and techniques employed in GoogLeNet network.

English Else
Multi-column Deep Neural Networks for Image Classification

Dan Cireşan,Ueli Meier,Juergen Schmidhuber

DOI: https://doi.org/10.48550/arXiv.1202.2745

2012-02-13

Abstract:Traditional methods of computer vision and machine learning cannot match human performance on tasks such as the recognition of handwritten digits or traffic signs. Our biologically plausible deep artificial neural network architectures can. Small (often minimal) receptive fields of convolutional winner-take-all neurons yield large network depth, resulting in roughly as many sparsely connected neural layers as found in mammals between retina and visual cortex. Only winner neurons are trained. Several deep neural columns become experts on inputs preprocessed in different ways; their predictions are averaged. Graphics cards allow for fast training. On the very competitive MNIST handwriting benchmark, our method is the first to achieve near-human performance. On a traffic sign recognition benchmark it outperforms humans by a factor of two. We also improve the state-of-the-art on a plethora of common image classification benchmarks.

Computer Vision and Pattern Recognition,Artificial Intelligence
Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs

Liang-Chieh Chen,George Papandreou,Iasonas Kokkinos,Kevin Murphy,Alan L. Yuille

DOI: https://doi.org/10.48550/arXiv.1412.7062

2014-12-22

Computer Vision and Pattern Recognition

Abstract:Deep Convolutional Neural Networks (DCNNs) have recently shown state of the art performance in high level vision tasks, such as image classification and object detection. This work brings together methods from DCNNs and probabilistic graphical models for addressing the task of pixel-level classification (also called "semantic image segmentation"). We show that responses at the final layer of DCNNs are not sufficiently localized for accurate object segmentation. This is due to the very invariance properties that make DCNNs good for high level tasks. We overcome this poor localization property of deep networks by combining the responses at the final DCNN layer with a fully connected Conditional Random Field (CRF). Qualitatively, our "DeepLab" system is able to localize segment boundaries at a level of accuracy which is beyond previous methods. Quantitatively, our method sets the new state-of-art at the PASCAL VOC-2012 semantic image segmentation task, reaching 71.6% IOU accuracy in the test set. We show how these results can be obtained efficiently: Careful network re-purposing and a novel application of the 'hole' algorithm from the wavelet community allow dense computation of neural net responses at 8 frames per second on a modern GPU.
Deep Convolutional Neural Networks for Image Classification: A Comprehensive Review

Waseem Rawat,Zenghui Wang

DOI: https://doi.org/10.1162/neco_a_00990

IF: 3.278

2017-09-01

Neural Computation

Abstract:Convolutional neural networks (CNNs) have been applied to visual tasks since the late 1980s. However, despite a few scattered applications, they were dormant until the mid-2000s when developments in computing power and the advent of large amounts of labeled data, supplemented by improved algorithms, contributed to their advancement and brought them to the forefront of a neural network renaissance that has seen rapid progression since 2012. In this review, which focuses on the application of CNNs to image classification tasks, we cover their development, from their predecessors up to recent state-of-the-art deep learning systems. Along the way, we analyze (1) their early successes, (2) their role in the deep learning renaissance, (3) selected symbolic works that have contributed to their recent popularity, and (4) several improvement attempts by reviewing contributions and challenges of over 300 publications. We also introduce some of their current trends and remaining challenges.

computer science, artificial intelligence,neurosciences
CIFAR-10 Image Classification Using Feature Ensembles

Felipe O. Giuste,Juan C. Vizcarra

DOI: https://doi.org/10.48550/arXiv.2002.03846

2020-02-07

Computer Vision and Pattern Recognition

Abstract:Image classification requires the generation of features capable of detecting image patterns informative of group identity. The objective of this study was to classify images from the public CIFAR-10 image dataset by leveraging combinations of disparate image feature sources from both manual and deep learning approaches. Histogram of oriented gradients (HOG) and pixel intensities successfully inform classification (53% and 59% classification accuracy, respectively), yet there is much room for improvement. VGG16 with ImageNet trained weights and a CIFAR-10 optimized model (CIFAR-VGG) further improve upon image classification (60% and 93.43% accuracy, respectively). We further improved classification by utilizing transfer learning to re-establish optimal network weights for VGG16 (TL-VGG) and Inception ResNet v2 (TL-Inception) resulting in significant performance increases (85% and 90.74%, respectively), yet fail to surpass CIFAR-VGG. We hypothesized that if each generated feature set obtained some unique insight into the classification problem, then combining these features would result in greater classification accuracy, surpassing that of CIFAR-VGG. Upon selection of the top 1000 principal components from TL-VGG, TL-Inception, HOG, pixel intensities, and CIFAR-VGG, we achieved testing accuracy of 94.6%, lending support to our hypothesis.
Hyperspectral Image Classification via Discriminant Gabor Ensemble Filter

Ke-Kun Huang,Chuan-Xian Ren,Hui Liu,Zhao-Rong Lai,Yu-Feng Yu,Dao-Qing Dai

DOI: https://doi.org/10.1109/TCYB.2021.3051141

IF: 11.8

2022-01-01

IEEE Transactions on Cybernetics

Abstract:For a broad range of applications, hyperspectral image (HSI) classification is a hot topic in remote sensing, and convolutional neural network (CNN)-based methods are drawing increasing attention. However, to train millions of parameters in CNN requires a large number of labeled training samples, which are difficult to collect. A conventional Gabor filter can effectively extract spatial information with different scales and orientations without training, but it may be missing some important discriminative information. In this article, we propose the Gabor ensemble filter (GEF), a new convolutional filter to extract deep features for HSI with fewer trainable parameters. GEF filters each input channel by some fixed Gabor filters and learnable filters simultaneously, then reduces the dimensions by some learnable <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$1\times 1$ </tex-math></inline-formula> filters to generate the output channels. The fixed Gabor filters can extract common features with different scales and orientations, while the learnable filters can learn some complementary features that Gabor filters cannot extract. Based on GEF, we design a network architecture for HSI classification, which extracts deep features and can learn from limited training samples. In order to simultaneously learn more discriminative features and an end-to-end system, we propose to introduce the local discriminant structure for cross-entropy loss by combining the triplet hard loss. Results of experiments on three HSI datasets show that the proposed method has significantly higher classification accuracy than other state-of-the-art methods. Moreover, the proposed method is speedy for both training and testing.
GaborNet: Gabor filters with learnable parameters in deep convolutional neural networks

Andrey Alekseev,Anatoly Bobe

DOI: https://doi.org/10.48550/arXiv.1904.13204

2019-04-30

Abstract:The article describes a system for image recognition using deep convolutional neural networks. Modified network architecture is proposed that focuses on improving convergence and reducing training complexity. The filters in the first layer of the network are constrained to fit the Gabor function. The parameters of Gabor functions are learnable and are updated by standard backpropagation techniques. The system was implemented on Python, tested on several datasets and outperformed the common convolutional networks.

Computer Vision and Pattern Recognition,Machine Learning,Image and Video Processing
Deep Convolutional Neural Networks for Hyperspectral Image Classification

Wei Hu,Yangyu Huang,Li Wei,Fan Zhang,Hengchao Li

DOI: https://doi.org/10.1155/2015/258619

IF: 2.336

2015-01-01

Journal of Sensors

Abstract:Recently, convolutional neural networks have demonstrated excellent performance on various visual tasks, including the classification of common two-dimensional images. In this paper, deep convolutional neural networks are employed to classify hyperspectral images directly in spectral domain. More specifically, the architecture of the proposed classifier contains five layers with weights which are the input layer, the convolutional layer, the max pooling layer, the full connection layer, and the output layer. These five layers are implemented on each spectral signature to discriminate against others. Experimental results based on several hyperspectral image data sets demonstrate that the proposed method can achieve better classification performance than some traditional methods, such as support vector machines and the conventional deep learning-based methods.

engineering, electrical & electronic,instruments & instrumentation
Image retrieval and classification on deep convolutional SparkNet

Hongyang Li,Peng Su,Zhizhen Chi,Jingjing Wang

DOI: https://doi.org/10.1109/ICSPCC.2016.7753615

2016-01-01

Abstract:Image retrieval and classification are hot topics in computer vision and have attracted great attention nowadays with the emergence of large-scale data. We propose a new scheme to use both deep learning models and large-scale computing platform and jointly learn powerful feature representations in image classification and retrieval. We achieve a superior performance on the ImageNet dataset, where the framework is easy to be embedded for daily user experience. First we conduct the classification task using deep convolutional neural networks with several novel techniques, including batch normalization and multi-crop testing to obtain a better performance. Then we transfer the network's knowledge to image retrieval task by comparing the feature codebook of the query image with those feature database extracted from the deep model. Such a search pipeline is implemented in a MapReduce framework on the Spark platform, which is suitable for large-scale and real-time data processing. At last, the system outputs to users some textual information of the predicted object searching from Internet as well as similar images from the retrieval stage, making our work a real application.
Deep Isometric Learning for Visual Recognition

Haozhi Qi,Chong You,Xiaolong Wang,Yi Ma,Jitendra Malik

DOI: https://doi.org/10.48550/arXiv.2006.16992

2020-08-15

Abstract:Initialization, normalization, and skip connections are believed to be three indispensable techniques for training very deep convolutional neural networks and obtaining state-of-the-art performance. This paper shows that deep vanilla ConvNets without normalization nor skip connections can also be trained to achieve surprisingly good performance on standard image recognition benchmarks. This is achieved by enforcing the convolution kernels to be near isometric during initialization and training, as well as by using a variant of ReLU that is shifted towards being isometric. Further experiments show that if combined with skip connections, such near isometric networks can achieve performances on par with (for ImageNet) and better than (for COCO) the standard ResNet, even without normalization at all. Our code is available at <a class="link-external link-https" href="https://github.com/HaozhiQi/ISONet" rel="external noopener nofollow">this https URL</a>.

Computer Vision and Pattern Recognition
Computer Vision : History , the Rise of Deep Networks , and Future Vistas Panel on Perception and Cognition , MORS Meeting on Artificial Intelligence and Autonomy

Herschel Seder

Abstract:Recent results indicate that the generic descriptors extracted from the convolutional neural networks are very powerful [10, 29, 48]. This pap r adds to th mounting evidence that this is indeed the case. We report on a series of experiments conducted for different recognition tasks using the publicly available code and model of the OverFeat network which was trained to perform object classification on ILSVRC13. We use features extracted from the OverFeat network as a generic image representation to tackle the diverse range of recognition tasks of object image classification, scene recognition, fine grained recognition, attribute detection a d image retr eval applied to a diverse set of datasets. W selected th se tasks and datasets as they gradually mov further away from the riginal task and data the OverFea network w s trained to solve. Astonishingly, we report consistent superior results compared to the highly tuned state-of-the-art systems in all the visual classificatio tasks on various datasets. For instance retrieval it consistently outperforms low memory footprint methods except for sculptures dataset. The results are achieved using a linear VM classifier (or L2 distance in case of re ri v l) applied o a f ature representation of size 4096 extracted from a layer in the net. Th repres ntations ar further modified using simple augme ation techniques e.g. jittering. The results strongly suggest that f atures obtained from deep learning with c nvolutional nets should be the primary candidate in most visual recognition

Computer Science
Image Classification Using Transfer Learning and Deep Learning

C. Desai

DOI: https://doi.org/10.18535/IJECS/V10I9.4622

2021-09-23

International Journal Of Engineering And Computer Science

Abstract:Deep learning models have demonstrated improved efficacy in image classification since the ImageNet Large Scale Visual Recognition Challenge started since 2010. Classification of images has further augmented in the field of computer vision with the dawn of transfer learning. To train a model on huge dataset demands huge computational resources and add a lot of cost to learning. Transfer learning allows to reduce on cost of learning and also help avoid reinventing the wheel. There are several pretrained models like VGG16, VGG19, ResNet50, Inceptionv3, EfficientNet etc which are widely used. This paper demonstrates image classification using pretrained deep neural network model VGG16 which is trained on images from ImageNet dataset. After obtaining the convolutional base model, a new deep neural network model is built on top of it for image classification based on fully connected network. This classifier will use features extracted from the convolutional base model.

Computer Science

GE Hinton Imagenet classification with deep convolutional neural networks., 2012, 25

ImageNet classification with deep convolutional neural networks

Research on Image Classification Method of Features of Combinatorial Convolution

Deep Tessellated Retinal Image Detection Using Convolutional Neural Networks

Research on image classification leveraging deep convolutional neural networks and visual cognition

Going deeper with convolutions

SwGridNet: A Deep Convolutional Neural Network based on Grid Topology for Image Classification

Evaluation of Deep Convolutional Nets for Document Image Classification and Retrieval

Very Deep Convolutional Networks for Large-Scale Image Recognition

Deep Convolution Neural Networks in Computer Vision: a Review

Multi-column Deep Neural Networks for Image Classification

Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs

Deep Convolutional Neural Networks for Image Classification: A Comprehensive Review

CIFAR-10 Image Classification Using Feature Ensembles

Hyperspectral Image Classification via Discriminant Gabor Ensemble Filter

GaborNet: Gabor filters with learnable parameters in deep convolutional neural networks

Deep Convolutional Neural Networks for Hyperspectral Image Classification

Image retrieval and classification on deep convolutional SparkNet

Deep Isometric Learning for Visual Recognition

Computer Vision : History , the Rise of Deep Networks , and Future Vistas Panel on Perception and Cognition , MORS Meeting on Artificial Intelligence and Autonomy

Image Classification Using Transfer Learning and Deep Learning