Abstract:The dynamic conditions and internal states of portable energy storage system (PESS), such as temperature, electricity price, state of charge (SOC), and state of health (SOH), significantly impact battery degradation. Current decision-making models for PESS operation often oversimplify the modeling of battery degradation. To address this, we introduce an environment-adaptive online learning framework that effectively integrates deep neural networks and reinforcement learning to exploit and explore external environments (i.e., electricity prices and temperature) and internal dynamics (i.e., battery degradation), providing decision support for PESS operation. This framework dynamically updates battery degradation and decision-making models in real-time, enhancing adaptive responses to external changes. Specifically, we developed a neural network based on porous electrode theory that considers multi-physical factors, such as charging power, initial and terminal SOC, SOH, and temperature to accurately assess battery degradation. This network is embedded within a deep reinforcement learning algorithm, enabling real-time, adaptive decision-making for PESS amidst varying environmental conditions. Furthermore, to navigate complex operational environments, a fine-tuning mechanism is incorporated into the degradation neural network. Application of this framework to the energy arbitrage of PESS in the California power grid demonstrates an average benefit increase of 37% compared to traditional degradation assessment models. Note to Practitioners —In this work, we develop a novel approach to addressing the critical issue of battery degradation in PESS. Existing models often oversimplify degradation, hindering accurate assessments of performance and lifespan projections. More recently, learning-based algorithms have demonstrated outstanding performance in both battery degradation modeling and real-time decision-making. In this sense, we introduce a sophisticated neural network model grounded in a porous electrode model. This model considers multi-physics factors involving charging/discharging power, initial and terminal SOC, SOH, and temperature. Complementing this, we integrate the aforementioned model into an online learning framework, enabling real-time decision-making for PESS. Furthermore, to tackle the challenges of complex operating environments, a fine-tuning mechanism is incorporated into the battery degradation neural network. We validate the effectiveness of the proposed methods through an energy arbitrage application of PESS and also reveal its potential for on-demand applications in energy and transportation systems. This note anticipates a positive impact on PESS management and contributes significantly to the evolution of energy storage systems, offering practitioners invaluable decision support for commercial applications involving battery sharing, trading, and renting.

Load Forecasting-Based Learning System for Energy Management With Battery Degradation Estimation: A Deep Reinforcement Learning Approach

Reinforcement Learning-Enhanced Adaptive Scheduling of Battery Energy Storage Systems in Energy Markets

Deep Reinforcement Learning-Based Energy Storage Arbitrage With Accurate Lithium-Ion Battery Degradation Model

Mobile battery energy storage system control with knowledge‐assisted deep reinforcement learning

Reinforcement Learning Driven Self-Optimizing Operation for Distributed Battery Electrical Storage System

An Integrated Demand Response-Based Energy Management Strategy for Integrated Energy System Based on Deep Reinforcement Learning

Reinforcement Learning-Based Scheduling of Multi-Battery Energy Storage System

Environment-Adaptive Online Learning for Portable Energy Storage Based on Porous Electrode Model

Risk-Sensitive Mobile Battery Energy Storage System Control With Deep Reinforcement Learning and Hybrid Risk Estimation Method

Hierarchical Deep Learning Model for Degradation Prediction per Look-Ahead Scheduled Battery Usage Profile

Demand Forecasting and Resource Scheduling of Independent Energy Storage Market in Power Grid with Deep Learning

Data-based Energy Storage System Configuration and Operation Considering Degradation Cost

Online Operation of Battery Energy Storage System for Demand Charge Reduction Considering Degradation

Reinforcement Learning and Stochastic Optimization with Deep Learning-Based Forecasting on Power Grid Scheduling

Computational Enhancement for Day-Ahead Energy Scheduling with Sparse Neural Network-based Battery Degradation Model

Real-time Economic Dispatch of Thermal-Wind-Battery Hybrid Systems Based on Deep Reinforcement Learning

Reinforcement Learning-Based Optimal Battery Control Under Cycle-Based Degradation Cost

Stochastic coordinated operation of wind and battery energy storage system considering battery degradation

A Control Strategy Based on Deep Reinforcement Learning Under the Combined Wind-Solar Storage System

Deep Reinforcement Learning Based Microgrid Expansion Planning with Battery Degradation and Resilience Enhancement