Abstract:This paper proposes an optimization of an existing Deep Neural Network (DNN) that improves its hardware utilization and facilitates on-device training for resource-constrained edge environments. We implement efficient parameter reduction strategies on Xception that shrink the model size without sacrificing accuracy, thus decreasing memory utilization during training. We evaluate our model in two experiments: Caltech-101 image classification and PCB defect detection and compare its performance against the original Xception and lightweight models, EfficientNetV2B1 and MobileNetV2. The results of the Caltech-101 image classification show that our model has a better test accuracy (76.21%) than Xception (75.89%), uses less memory on average (847.9MB) than Xception (874.6MB), and has faster training and inference times. The lightweight models overfit with EfficientNetV2B1 having a 30.52% test accuracy and MobileNetV2 having a 58.11% test accuracy. Both lightweight models have better memory usage than our model and Xception. On the PCB defect detection, our model has the best test accuracy (90.30%), compared to Xception (88.10%), EfficientNetV2B1 (55.25%), and MobileNetV2 (50.50%). MobileNetV2 has the least average memory usage (849.4MB), followed by our model (865.8MB), then EfficientNetV2B1 (874.8MB), and Xception has the highest (893.6MB). We further experiment with pre-trained weights and observe that memory usage decreases thereby showing the benefits of transfer learning. A Pareto analysis of the models' performance shows that our optimized model architecture satisfies accuracy and low memory utilization objectives.

Lightweight Deep Neural Network Model With Padding-Free Downsampling

Deep Neural Network Acceleration with Sparse Prediction Layers

Padding Module: Learning the Padding in Deep Neural Networks

Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding

LPRNet: Lightweight Deep Network by Low-rank Pointwise Residual Convolution

Smart-DNN: Efficiently Reducing the Memory Requirements of Running Deep Neural Networks on Resource-constrained Platforms

Memory-efficient Deep Learning Inference with Incremental Weight Loading and Data Layout Reorganization on Edge Systems.

Achieving Pareto Optimality using Efficient Parameter Reduction for DNNs in Resource-Constrained Edge Environment

Lightweight Compression of Intermediate Neural Network Features for Collaborative Intelligence

Deep Neural Network Interlayer Feature Map Compression Based on Least-Squares Fitting.

Learning Efficient Convolutional Networks Through Network Slimming.

CCDepth: A Lightweight Self-supervised Depth Estimation Network with Enhanced Interpretability

Prune the Convolutional Neural Networks with Sparse Shrink

LighTN: Light-weight Transformer Network for Performance-overhead Tradeoff in Point Cloud Downsampling

Deep Adaptive Network: An Efficient Deep Neural Network with Sparse Binary Connections

Lightweight compression of neural network feature tensors for collaborative intelligence

Lightweight image de-snowing: A better trade-off between network capacity and performance

LRPRNet: Lightweight Deep Network by Low-Rank Pointwise Residual Convolution

DeepRebirth: Accelerating Deep Neural Network Execution on Mobile Devices

Deep Anchored Convolutional Neural Networks

TB-DNN: A Thin Binarized Deep Neural Network with High Accuracy