ImageNet classification with deep convolutional neural networks (AlexNet) ImageNet classification with deep convolutional neural networks (AlexNet)

A Krizhevsky, I Sutskever, G Hinton

2023-01-01

Abstract:

What problem does this paper attempt to address?

ImageNet classification with deep convolutional neural networks

Alex Krizhevsky,Ilya Sutskever,Geoffrey E. Hinton

DOI: https://doi.org/10.1145/3065386

IF: 22.7

2017-05-24

Communications of the ACM

Abstract:We trained a large, deep convolutional neural network to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes. On the test data, we achieved top-1 and top-5 error rates of 37.5% and 17.0%, respectively, which is considerably better than the previous state-of-the-art. The neural network, which has 60 million parameters and 650,000 neurons, consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully connected layers with a final 1000-way softmax. To make training faster, we used non-saturating neurons and a very efficient GPU implementation of the convolution operation. To reduce overfitting in the fully connected layers we employed a recently developed regularization method called "dropout" that proved to be very effective. We also entered a variant of this model in the ILSVRC-2012 competition and achieved a winning top-5 test error rate of 15.3%, compared to 26.2% achieved by the second-best entry.

computer science, theory & methods, software engineering, hardware & architecture
Research on image classification leveraging deep convolutional neural networks and visual cognition

Chen Liu

DOI: https://doi.org/10.54254/2755-2721/32/20230212

2024-01-31

Abstract:The field of image classification has experienced remarkable improvements with the advent of deep learning techniques, especially Deep Convolutional Neural Networks. The present study provides an extensive exploration of the junction where image classification based on Deep Convolutional Neural Networks meets human visual cognition. Utilizing the inherent ability of these networks to automatically learn hierarchical features from raw pixel data, this research examines their potential in classifying images from diverse complex datasets, emphasizing predominantly on the extensively utilized ImageNet dataset. The initial aspect of this study involves training and evaluating models based on Deep Convolutional Neural Networks on the ImageNet dataset, which comprises millions of labeled images spanning across thousands of categories. Well-established network architectures such as AlexNet, VGGNet, GoogLeNet, and ResNet are employed, and their performance in the challenging task of image classification is assessed. Rigorous experiments highlight the strengths and weaknesses of each model while emphasizing the prospects of transfer learning and fine-tuning. Following this, the interpretability of Deep Convolutional Neural Networks is explored by using visualization techniques to comprehend the learned feature representations. By visualizing activation maps and class-specific saliency maps, valuable insights are gained into the regions of interest that guide the decision-making of these models. Moreover, the correlation between the features extracted by these models and human visual attention mechanisms is examined to shed light on the focus of attention of the models. The study also addresses the difficulties that adversarial attacks, data bias, and generalization capabilities present to Deep Convolutional Neural Networks. Strategies to enhance the robustness and adaptability of the models across various domains are examined, linking these observations to human cognitive behavior.
Research on Image Classification Method of Features of Combinatorial Convolution

Chao Wu,Yaqian Li,Zhibiao Zhao,Bin Liu

DOI: https://doi.org/10.1007/s12652-019-01433-9

IF: 3.662

2019-01-01

Journal of Ambient Intelligence and Humanized Computing

Abstract:In image classification, shallow convolutional features and deep convolutional features are not fully utilized by many network frameworks. To solve this problem, we propose a combinatorial convolutional network (CCNet) that integrates convolutional features of all levels. According to its own structure, the convolutional features of shallow, medium, and deep levels are extracted. These features are combined by weighted concatenation and convolutional fusion, and the coefficients of each channel of final combination feature are again weighted to improve the identification degree of features. CCNet can improve the single case where most network only add or concatenate shallow and deep features, so that the network can achieve lower classification error rate while generating low-dimensional features. Extensive experiments are performed on CIFAR-10 and CIFAR-100 respectively. The experimental results show that the low-dimensional image feature vectors generated by CCNet effectively reduce the classification error rate when the number of convolutional layers does not exceed 100 layers.
Enhancing Image Classification Accuracy Based on AlexNet

Yu Zeng

DOI: https://doi.org/10.54097/9c8kkn59

2024-03-13

Abstract:This study explores the potential of deep learning for small-scale image classification tasks through the utilization of the classical AlexNet model on the CIFAR-10 dataset. The methodology involves meticulous architectural adjustments, comprehensive data preprocessing, and strategic optimization of the learning rate decay, resulting in substantial improvements in model performance. In addition, innovative data augmentation techniques are introduced to enhance the model's robustness and generalization capabilities. The experimental outcomes unequivocally underscore the efficacy of deep learning in addressing small-scale image classification challenges, offering versatile applications across diverse domains such as image recognition, autonomous driving, and medical diagnostics. Acknowledging the dynamic nature of deep learning, ongoing research endeavors are warranted. Future directions may encompass the exploration of more intricate neural network architectures, advanced data augmentation methodologies, and the implementation of interpretability tools to augment both performance and comprehensibility. This study provides compelling evidence of deep learning's aptitude for small-scale image classification tasks, offering valuable insights and guidance for future research and practical implementations. As the field of deep learning continues to evolve, this work aims to serve as a valuable reference and catalyst for further advancements in the domain.
Deep Convolutional Neural Networks for Image Classification: A Comprehensive Review

Waseem Rawat,Zenghui Wang

DOI: https://doi.org/10.1162/neco_a_00990

IF: 3.278

2017-09-01

Neural Computation

Abstract:Convolutional neural networks (CNNs) have been applied to visual tasks since the late 1980s. However, despite a few scattered applications, they were dormant until the mid-2000s when developments in computing power and the advent of large amounts of labeled data, supplemented by improved algorithms, contributed to their advancement and brought them to the forefront of a neural network renaissance that has seen rapid progression since 2012. In this review, which focuses on the application of CNNs to image classification tasks, we cover their development, from their predecessors up to recent state-of-the-art deep learning systems. Along the way, we analyze (1) their early successes, (2) their role in the deep learning renaissance, (3) selected symbolic works that have contributed to their recent popularity, and (4) several improvement attempts by reviewing contributions and challenges of over 300 publications. We also introduce some of their current trends and remaining challenges.

computer science, artificial intelligence,neurosciences
Review of AlexNet for Medical Image Classification

Wenhao Tang,Junding Sun,Shuihua Wang,Yudong Zhang

DOI: https://doi.org/10.4108/eetel.4389

2023-12-21

EAI Endorsed Transactions on e-Learning

Abstract:In recent years, the rapid development of deep learning has led to a wide range of applications in the field of medical image classification. The variants of neural network models with ever-increasing performance share some commonalities: to try to mitigate overfitting, improve generalization, avoid gradient vanishing and exploding, etc. AlexNet first utilizes the dropout technique to mitigate overfitting and the ReLU activation function to avoid gradient vanishing. Therefore, we focus our discussion on AlexNet, which has contributed greatly to the development of CNNs in 2012. After reviewing over 40 papers, including journal papers and conference papers, we give a narrative on the technical details, advantages, and application areas of AlexNet.
Research on Improved Algorithm of Image Classification Based on Convolutional Neural Network

Hongyan Liu

DOI: https://doi.org/10.25236/ajcis.2023.060402

2023-01-01

Academic Journal of Computing & Information Science

Abstract:With the continuous exploration of artificial intelligence, the convolutional neural network, as one of the representative algorithms, has also developed rapidly.The convolutional neural network extracts more high-dimensional and abstract features from the data, summarizes the distributed feature representation of the data, and discovers complex nonlinear relationships.Due to the rapid increase in the amount of calculations in the era of big data, the structure of convolutional neural networks is also more complex, so the difficulty of computing tasks continues to increase.Aiming at these difficulties, this paper optimizes the convolutional neural network model AlexNet.This paper first introduces the basic principles of artificial neural networks and related technologies of convolutional neural networks, and analyzes the development prospects and research directions of convolutional neural network algorithms.Then introduce the convolutional neural network model AlexNet, and analyze and summarize its shortcomings.
Deep Sparse Coding Network for Image Classification

Sijun Zhou,Shizhou Zhang,Jinjun Wang

DOI: https://doi.org/10.1145/2808492.2808516

2015-01-01

Abstract:In this paper, we introduce a novel deep model called Deep Sparse Coding Network (DeepSCNet) for Image classification. This new model includes four types of layers: the Sparse-coding layer, the Pooling layer, the Normalization layer and the Map reduction layer. The Sparse-coding laying does the general linear coding work for local patch within the receptive field. The Pooling layer and the Normalization layer do the same work as that in CNN. The Map reduction layer reduces the CPU and memory consumptions by reducing the number of feature maps. The paper further discusses the multi-scale, multi-locality extension to the basic DeepSCNet. Compared to CNN, training DeepSCNet is relatively easier even with a training set of moderate size. Experiments show that DeepSCNet can automatically discover highly discriminative feature directly from raw image pixels.
SwGridNet: A Deep Convolutional Neural Network based on Grid Topology for Image Classification

Atsushi Takeda

DOI: https://doi.org/10.48550/arXiv.1709.07646

2017-10-03

Abstract:Deep convolutional neural networks (CNNs) achieve remarkable performance on image classification tasks. Recent studies, however, have demonstrated that generalization abilities are more important than the depth of neural networks for improving performance on image classification tasks. Herein, a new neural network called SwGridNet is proposed. A SwGridNet includes many convolutional processing units which connect mutually as a grid network where many processing paths exist between input and output. A SwGridNet has high generalization capability because the multipath architecture has the same effect of ensemble learning. As described in this paper, details of the SwGridNet network architecture are presented. Experimentally obtained results presented in this paper show that SwGridNets respectively achieve test error rates of 2.95% and 15.67% in a CIFAR-10 and CIFAR-100 classification tasks. The results indicate that the SwGridNet performance approximates that of state-of-the-art deep CNNs.

Computer Vision and Pattern Recognition
Image Classification Using Transfer Learning and Deep Learning

C. Desai

DOI: https://doi.org/10.18535/IJECS/V10I9.4622

2021-09-23

International Journal Of Engineering And Computer Science

Abstract:Deep learning models have demonstrated improved efficacy in image classification since the ImageNet Large Scale Visual Recognition Challenge started since 2010. Classification of images has further augmented in the field of computer vision with the dawn of transfer learning. To train a model on huge dataset demands huge computational resources and add a lot of cost to learning. Transfer learning allows to reduce on cost of learning and also help avoid reinventing the wheel. There are several pretrained models like VGG16, VGG19, ResNet50, Inceptionv3, EfficientNet etc which are widely used. This paper demonstrates image classification using pretrained deep neural network model VGG16 which is trained on images from ImageNet dataset. After obtaining the convolutional base model, a new deep neural network model is built on top of it for image classification based on fully connected network. This classifier will use features extracted from the convolutional base model.

Computer Science
Deep Neural Network Concepts for Classification using Convolutional Neural Network: A Systematic Review and Evaluation

Mohammad Galety,Firas Hussam Al Mukthar,Rebaz Jamal Maaroof,Fanar Rofoo

DOI: https://doi.org/10.47577/technium.v3i8.4554

2021-09-28

Technium Romanian Journal of Applied Sciences and Technology

Abstract:In recent years, artificial intelligence (AI) has piqued the curiosity of researchers. Convolutional Neural Networks (CNN) is a deep learning (DL) approach commonly utilized to solve problems. In standard machine learning tasks, biologically inspired computational models surpass prior types of artificial intelligence by a considerable margin. The Convolutional Neural Network (CNN) is one of the most stunning types of ANN architecture. The goal of this research is to provide information and expertise on many areas of CNN. Understanding the concepts, benefits, and limitations of CNN is critical for maximizing its potential to improve image categorization performance. This article has integrated the usage of a mathematical object called covering arrays to construct the set of ideal parameters for neural network design due to the complexity of the tuning process for the correct selection of the parameters used for this form of neural network.
Study On Image Classification With Convolution Neural Networks

Lei Wang,Yanning Zhang,Runping Xi

DOI: https://doi.org/10.1007/978-3-319-23989-7_32

2015-01-01

Abstract:Classification is one of the most popular topics in image process. In this paper, it provides the specific process of convolutional neural network in deep learning. Here builds typical convolution neural network which parameters and the connection mode can be adjusted. In addition, we present our preliminary classification results. Through the experiments of a convolutional neural network on the Mixed National Institute of Standards and Technology database (MNIST), we compared with the classification results and analyzed in the experimental results with the parameters. The experimental results show that image classification effect is very good used by convolutional neural network.
Review of Deep Convolution Neural Network in Image Classification

Ahmed Ali Mohammed Al-Saffar,Hai Tao,Mohammed Ahmed Talab

DOI: https://doi.org/10.1109/icramet.2017.8253139

2017-01-01

Abstract:With the development of large data age, Convolutional neural networks (CNNs) with more hidden layers have more complex network structure and more powerful feature learning and feature expression abilities than traditional machine learning methods. The convolution neural network model trained by the deep learning algorithm has made remarkable achievements in many large-scale identification tasks in the field of computer vision since its introduction. This paper first introduces the rise and development of deep learning and convolution neural network, and summarizes the basic model structure, convolution feature extraction and pooling operation of convolution neural network. Then, the research status and development trend of convolution neural network model based on deep learning in image classification are reviewed, which is mainly introduced from the aspects of typical network structure construction, training method and performance. Finally, some problems in the current research are briefly summarized and discussed, and the new direction of future development is forecasted.
Convolutional neural network of deep learning in computer vision and image classification problems

K E Tokarev,V M Zotov,V N Khavronina,O V Rodionova

DOI: https://doi.org/10.1088/1755-1315/786/1/012040

2021-06-01

IOP Conference Series: Earth and Environmental Science

Abstract:Abstract The article considers the possibilities of using the deep learning convolutional neural network ResNet in computer vision and image classification problems. The interpretation of the ResNet network and the datasets used for its training are presented, as well as a method for training a deep convolutional neural network with stochastic depth, which allows significantly reducing errors in the test sample.
Evaluation of Deep Convolutional Nets for Document Image Classification and Retrieval

Adam W. Harley,Alex Ufkes,Konstantinos G. Derpanis

DOI: https://doi.org/10.48550/arXiv.1502.07058

2015-02-25

Abstract:This paper presents a new state-of-the-art for document image classification and retrieval, using features learned by deep convolutional neural networks (CNNs). In object and scene analysis, deep neural nets are capable of learning a hierarchical chain of abstraction from pixel inputs to concise and descriptive representations. The current work explores this capacity in the realm of document analysis, and confirms that this representation strategy is superior to a variety of popular hand-crafted alternatives. Experiments also show that (i) features extracted from CNNs are robust to compression, (ii) CNNs trained on non-document images transfer well to document analysis tasks, and (iii) enforcing region-specific feature-learning is unnecessary given sufficient training data. This work also makes available a new labelled subset of the IIT-CDIP collection, containing 400,000 document images across 16 categories, useful for training new CNNs for document analysis.

Computer Vision and Pattern Recognition,Information Retrieval,Machine Learning,Neural and Evolutionary Computing
Simple convolutional neural network on image classification

Tianmei Guo,Jiwen Dong,Henjian Li,Yunxing Gao

DOI: https://doi.org/10.1109/icbda.2017.8078730

2017-03-01

Abstract:In recent years, deep learning has been used in image classification, object tracking, pose estimation, text detection and recognition, visual saliency detection, action recognition and scene labeling. Auto Encoder, sparse coding, Restricted Boltzmann Machine, Deep Belief Networks and Convolutional neural networks is commonly used models in deep learning. Among different type of models, Convolutional neural networks has been demonstrated high performance on image classification. In this paper we bulided a simple Convolutional neural network on image classification. This simple Convolutional neural network completed the image classification. Our experiments are based on benchmarking datasets minist [1] and cifar-10. On the basis of the Convolutional neural network, we also analyzed different methods of learning rate set and different optimization algorithm of solving the optimal parameters of the influence on image classification.
Deep Convolution Neural Networks in Computer Vision: a Review

Hyeon-Joong Yoo

DOI: https://doi.org/10.5573/ieiespc.2015.4.1.035

2015-02-28

IEIE Transactions on Smart Processing and Computing

Abstract:Over the past couple of years, tremendous progress has been made in applying deep learning (DL) techniques to computer vision. Especially, deep convolutional neural networks (DCNNs) have achieved state-of-the-art performance on standard recognition datasets and tasks such as ImageNet Large-Scale Visual Recognition Challenge (ILSVRC). Among them, GoogLeNet network which is a radically redesigned DCNN based on the Hebbian principle and scale invariance set the new state of the art for classification and detection in the ILSVRC 2014. Since there exist various deep learning techniques, this review paper is focusing on techniques directly related to DCNNs, especially those needed to understand the architecture and techniques employed in GoogLeNet network.

English Else
Performance Evaluation of Deep Learning Classification Network for Image Features

Qiang Li,Yingjian Yang,Yingwei Guo,Wei Li,Yang Liu,Han Liu,Yan Kang

DOI: https://doi.org/10.1109/access.2020.3048956

IF: 3.9

2021-01-01

IEEE Access

Abstract:Deep learning (DL) has emerged as a powerful image processing technique that learns the features of the data and produces state-of-the-art prediction results. The decade from 2010 to 2020 is a real revival of DL, which has come to a turning point in history. In image classification, many deep learning networks have been proposed by scholars, and each of them has its own strengthness. It is very important and efficient for the researchers and the developers to know the performance of these networks, especially for the beginners, so as to give them a transplant instruction by an objective evaluation index. In this paper, we constructed different data sets from three aspects, texture, shape, and measurement scale to test the performance of nine mainstream image classification networks. Cross-contrast experiments were performed to analyze the sensitivity of factors which influence the stability of image classification networks. Experimental results shown that in the 27 image datasets generated by the three image factors, the classification performance of AlexNet, GoogleNet, VggNet, and DenseNet is better. The perfomance comparison of these networks are showed and discussed in details. Code and pretrained models are available at https://github.com/liqiang12689/image-classification-finall.

computer science, information systems,telecommunications,engineering, electrical & electronic
Major Convolutional Neural Networks in Image Classification: A Survey

Navdeep Kumar,Nirmal Kaur,Deepti Gupta

DOI: https://doi.org/10.1007/978-981-15-3020-3_23

2020-01-01

Abstract:Deep learning is a vital technique to implement artificial intelligence and a significant part of machine learning. In the last decades, deep learning gained enormous popularity due to the remarkable enhancement in computational ability and machine learning experimentation. The high computational time taken by the processes in deep learning because of large data sets can be compensated by increased computational ability. In deep learning, Convolutional Neural Network (CNN) or ConvNet is among the eminent approaches used for image classification. In recent image recognition competitions, CNN is outperforming other techniques of image classification. In this review paper, we have been discussed the basics of CNN, and significant developments in the history of CNN concerning image classification.
Convolutional Deep Networks for Visual Data Classification

Shusen Zhou,Qingcai Chen,Xiaolong Wang

DOI: https://doi.org/10.1007/s11063-012-9260-y

IF: 2.565

2012-01-01

Neural Processing Letters

Abstract:This paper develops a semi-supervised learning algorithm called convolutional deep networks (CDN), to address the image classification problem with deep learning. First, we construct the previous several hidden layers using convolutional restricted Boltzmann machines, which can reduce the dimension and abstract the information of the images effectively. Second, we construct the following hidden layers using restricted Boltzmann machines, which can abstract the information of images quickly. Third, the constructed deep architecture is fine-tuned by gradient-descent based supervised learning with an exponential loss function. CDN can reduce the dimension and abstract the information of the images at the same time efficiently. More importantly, the abstraction and classification procedure of CDN use the same deep architecture to optimize the same parameter in different steps continuously, which can improve the learning ability effectively. We did several experiments on two standard image datasets, and show that CDN are competitive with both representative semi-supervised classifiers and existing deep learning techniques.

ImageNet classification with deep convolutional neural networks (AlexNet) ImageNet classification with deep convolutional neural networks (AlexNet)

ImageNet classification with deep convolutional neural networks

Research on image classification leveraging deep convolutional neural networks and visual cognition

Research on Image Classification Method of Features of Combinatorial Convolution

Enhancing Image Classification Accuracy Based on AlexNet

Deep Convolutional Neural Networks for Image Classification: A Comprehensive Review

Review of AlexNet for Medical Image Classification

Research on Improved Algorithm of Image Classification Based on Convolutional Neural Network

Deep Sparse Coding Network for Image Classification

SwGridNet: A Deep Convolutional Neural Network based on Grid Topology for Image Classification

Image Classification Using Transfer Learning and Deep Learning

Deep Neural Network Concepts for Classification using Convolutional Neural Network: A Systematic Review and Evaluation

Study On Image Classification With Convolution Neural Networks

Review of Deep Convolution Neural Network in Image Classification

Convolutional neural network of deep learning in computer vision and image classification problems

Evaluation of Deep Convolutional Nets for Document Image Classification and Retrieval

Simple convolutional neural network on image classification

Deep Convolution Neural Networks in Computer Vision: a Review

Performance Evaluation of Deep Learning Classification Network for Image Features

Major Convolutional Neural Networks in Image Classification: A Survey

Convolutional Deep Networks for Visual Data Classification