Abstract:The computation demand for machine learning (ML) has grown rapidly recently, which comes with a number of costs. Estimating the energy cost helps measure its environmental impact and finding greener strategies, yet it is challenging without detailed information. We calculate the energy use and carbon footprint of several recent large models-T5, Meena, GShard, Switch Transformer, and GPT-3-and refine earlier estimates for the neural architecture search that found Evolved Transformer. We highlight the following opportunities to improve energy efficiency and CO2 equivalent emissions (CO2e): Large but sparsely activated DNNs can consume <1/10th the energy of large, dense DNNs without sacrificing accuracy despite using as many or even more parameters. Geographic location matters for ML workload scheduling since the fraction of carbon-free energy and resulting CO2e vary ~5X-10X, even within the same country and the same organization. We are now optimizing where and when large models are trained. Specific datacenter infrastructure matters, as Cloud datacenters can be ~1.4-2X more energy efficient than typical datacenters, and the ML-oriented accelerators inside them can be ~2-5X more effective than off-the-shelf systems. Remarkably, the choice of DNN, datacenter, and processor can reduce the carbon footprint up to ~100-1000X. These large factors also make retroactive estimates of energy cost difficult. To avoid miscalculations, we believe ML papers requiring large computational resources should make energy consumption and CO2e explicit when practical. We are working to be more transparent about energy use and CO2e in our future research. To help reduce the carbon footprint of ML, we believe energy usage and CO2e should be a key metric in evaluating models, and we are collaborating with MLPerf developers to include energy usage during training and inference in this industry standard benchmark.

Carbon Intensity-Aware Adaptive Inference of DNNs

Chasing Low-Carbon Electricity for Practical and Sustainable DNN Training

Carbon Emissions and Large Neural Network Training

CarbonCP: Carbon-Aware DNN Partitioning with Conformal Prediction for Sustainable Edge Intelligence

Efficient Online Processing with Deep Neural Networks

Estimating Environmental Cost Throughout Model's Adaptive Life Cycle

Energy Efficiency of Training Neural Network Architectures: An Empirical Study

LLMCO2: Advancing Accurate Carbon Footprint Prediction for LLM Inferences

Energy-Aware Dynamic Neural Inference

Energy saving of buildings for reducing carbon dioxide emissions using novel dendrite net integrated adaptive mean square gradient

AutoScale: Optimizing Energy Efficiency of End-to-End Edge Inference under Stochastic Variance

Resource-aware Deployment of Dynamic DNNs over Multi-tiered Interconnected Systems

Incremental Training and Group Convolution Pruning for Runtime DNN Performance Scaling on Heterogeneous Embedded Platforms

Towards A Flexible Accuracy-Oriented Deep Learning Module Inference Latency Prediction Framework for Adaptive Optimization Algorithms

Energy-Aware Adaptive Multi-Exit Neural Network Inference Implementation for a Millimeter-Scale Sensing System

A Progressive Subnetwork Searching Framework for Dynamic Inference

Full-Cycle Energy Consumption Benchmark for Low-Carbon Computer Vision

The carbon footprint of artificial intelligence in materials science

Achieving Pareto Optimality using Efficient Parameter Reduction for DNNs in Resource-Constrained Edge Environment

A methodological framework for optimizing the energy consumption of deep neural networks: a case study of a cyber threat detector

Energy-Constrained Compression for Deep Neural Networks via Weighted Sparse Projection and Layer Input Masking