Abstract:Deep learning models in computer vision have achieved significant success but pose increasing concerns about energy consumption and sustainability. Despite these concerns, there is a lack of comprehensive understanding of their energy efficiency during inference. In this study, we conduct a comprehensive analysis of the inference energy consumption of 1,200 ImageNet classification models - the largest evaluation of its kind to date. Our findings reveal a steep diminishing return in accuracy gains relative to the increase in energy usage, highlighting sustainability concerns in the pursuit of marginal improvements. We identify key factors contributing to energy consumption and demonstrate methods to improve energy efficiency. To promote more sustainable AI practices, we introduce an energy efficiency scoring system and develop an interactive web application that allows users to compare models based on accuracy and energy consumption. By providing extensive empirical data and practical tools, we aim to facilitate informed decision-making and encourage collaborative efforts in developing energy-efficient AI technologies.

What problem does this paper attempt to address?

The main problems that this paper attempts to solve are the increasing energy consumption of deep - learning models during the inference process and its negative impacts on the environment and economy. Specifically, the author focuses on the following aspects: 1. **The relationship between marginal accuracy improvement and energy consumption**: Although deep - learning models have achieved remarkable success in the field of computer vision, the growth in energy consumption brought about by the pursuit of higher accuracy has led to the problem of diminishing returns. The author hopes to reveal whether the marginal benefits of accuracy improvement are worth the additional energy costs. 2. **Key factors affecting energy consumption**: In order to better understand which factors contribute to the increase in energy consumption, the author analyzes the impact of different model architectures, hardware configurations, and optimization techniques on energy efficiency. 3. **The effectiveness of current acceleration techniques**: Evaluate whether existing inference optimization techniques (such as TensorRT) can indeed improve energy efficiency and explore the possibility of further improvement. 4. **Energy - efficiency evaluation system**: Propose a standardized energy - efficiency scoring system to help researchers and practitioners select and evaluate models more rationally. 5. **Promote sustainable AI practices**: By providing a large amount of empirical data and practical tools (such as interactive Web applications), encourage cooperation within the community and promote the development of more energy - efficient AI technologies. ### Specific problems and solutions - **How to measure the cost of marginal accuracy improvement?** - By conducting a large - scale evaluation of 1,200 ImageNet classification models, the author shows that as energy consumption increases, the improvement in accuracy gradually becomes less obvious, emphasizing the importance and necessity of re - evaluating the pursuit of minor accuracy improvements. - **Which factors contribute to the energy consumption of AI models?** - The study finds that the number of parameters, FLOPs (floating - point operations per second), and the number of activation functions are strongly correlated with energy consumption. In addition, an increase in the input image size will also significantly increase energy consumption, but with limited improvement in accuracy. - **Are current acceleration techniques effective?** - Using optimization tools such as TensorRT can significantly reduce energy consumption, with an average reduction of about four times, especially for models that originally have high energy consumption, the effect is more obvious. - **How to balance energy consumption and accuracy?** - A scoring method based on the weighted Manhattan distance is proposed, which combines accuracy and energy consumption to comprehensively evaluate the performance of the model. Users can adjust the weights according to specific requirements, thereby finding the optimal balance between accuracy and energy consumption. Through these studies, the author hopes to provide valuable insights and tools for the sustainable development of the AI field and promote more environmentally - friendly and efficient deep - learning model design and application.

Double-Exponential Increases in Inference Energy: The Cost of the Race for Accuracy

Survey on Energy-Efficient Deep Neural Networks for Computer Vision

Full-Cycle Energy Consumption Benchmark for Low-Carbon Computer Vision

Watt For What: Rethinking Deep Learning's Energy-Performance Relationship

Accuracy is not the only Metric that matters: Estimating the Energy Consumption of Deep Learning Models

Unveiling Energy Efficiency in Deep Learning: Measurement, Prediction, and Scoring across Edge Devices

Uncovering Energy-Efficient Practices in Deep Learning Training: Preliminary Steps Towards Green AI

How green is continual learning, really? Analyzing the energy consumption in continual training of vision foundation models

EfficientBioAI: making bioimaging AI models efficient in energy and latency

Power Hungry Processing: Watts Driving the Cost of AI Deployment?

Measuring the Energy Consumption and Efficiency of Deep Neural Networks: An Empirical Analysis and Design Recommendations

Computing Within Limits: An Empirical Study of Energy Consumption in ML Training and Inference

Towards energy-efficient Deep Learning: An overview of energy-efficient approaches along the Deep Learning Lifecycle

Carbon Emissions and Large Neural Network Training

How to use model architecture and training environment to estimate the energy consumption of DL training

Carbon Intensity-Aware Adaptive Inference of DNNs

NeuralPower: Predict and Deploy Energy-Efficient Convolutional Neural Networks

Energy Efficiency of Training Neural Network Architectures: An Empirical Study

Effect of Architecture and Inference Parameters of Artificial Neural Network Models in the Detection Task on Energy Demand

EPAM: A Predictive Energy Model for Mobile AI

The Power of Training: How Different Neural Network Setups Influence the Energy Demand