Abstract:This paper studies the synergy of solar-battery energy storage system (BESS) and develops a viable strategy for the BESS to unlock its economic potential by serving as a backup to reduce solar curtailments while also participating in the electricity market. We model the real-time bidding of the solar-battery system as two Markov decision processes for the solar farm and the BESS, respectively. We develop a novel deep reinforcement learning (DRL) algorithm to solve the problem by leveraging attention mechanism (AC) and multi-grained feature convolution to process DRL input for better bidding decisions. Simulation results demonstrate that our AC-DRL outperforms two optimization-based and one DRL-based benchmarks by generating 23%, 20%, and 11% higher revenue, as well as improving curtailment responses. The excess solar generation can effectively charge the BESS to bid in the market, significantly reducing solar curtailments by 76% and creating synergy for the solar-battery system to be more viable.

What problem does this paper attempt to address?

The main focus of this paper is to optimize the operational strategy of solar energy storage systems in real-time electricity markets. The research aims to solve the problem of solar battery energy storage systems (BESS) participating in electricity market transactions while reducing solar energy consumption losses by using deep reinforcement learning (DRL) algorithms combined with attention mechanism (AC) and multi-scale feature convolution. The paper points out that although BESS can serve as a backup power source for solar farms to smooth the fluctuation of solar power generation, relying solely on ancillary services cannot fully exploit its economic potential. Therefore, BESS can increase revenue streams by engaging in energy arbitrage in the wholesale market. However, this requires an effective coordination strategy to simultaneously manage solar energy consumption and market participation in response to the uncertainty of solar power generation, market price fluctuations, and the coupling between BESS decisions. The paper proposes a new DRL algorithm called AC-DRL, which can learn the uncertainty of solar energy and electricity prices without a model and derive coordination strategies from historical experience through interactive training without relying on prior knowledge or prediction models. The AC-DRL network structure includes an attention mechanism to reveal the correlation between features and emphasize the relative importance of each feature, and then makes decisions through multi-scale feature convolution. This approach enables BESS and solar farms to focus on different input features during the dynamic bidding process. By using actual solar farm data for simulation, the AC-DRL method significantly outperforms optimization-based methods and existing DRL methods in terms of revenue and solar energy consumption management. The results show that effective management of solar energy consumption is crucial for successfully coordinating the two assets, improving energy arbitrage, and enhancing the economic feasibility of the integrated system. In summary, the primary contributions of this paper are as follows: 1. The proposal of a coordinated strategy that simultaneously reduces solar energy consumption losses and optimizes BESS bidding in real-time electricity markets. 2. The design of a DRL bidding network structure (AC-DRL) with attention mechanism and multi-scale feature convolution. 3. Simulation using real-world data demonstrates the effectiveness of the AC-DRL method, showing its superiority in terms of revenue and solar energy consumption management.

Attentive Convolutional Deep Reinforcement Learning for Optimizing Solar-Storage Systems in Real-Time Electricity Markets

Temporal-Aware Deep Reinforcement Learning for Energy Storage Bidding in Energy and Contingency Reserve Markets

Deep Reinforcement Learning for Strategic Bidding in Electricity Markets

Deep Reinforcement Learning for Wind and Energy Storage Coordination in Wholesale Energy and Ancillary Service Markets

Optimal Energy Storage Scheduling for Wind Curtailment Reduction and Energy Arbitrage: A Deep Reinforcement Learning Approach

Adaptive BESS and Grid Setpoints Optimization: A Model-Free Framework for Efficient Battery Management under Dynamic Tariff Pricing

A Control Strategy Based on Deep Reinforcement Learning Under the Combined Wind-Solar Storage System

Learning a local trading strategy: deep reinforcement learning for grid-scale renewable energy integration

Operational Optimization for Off-Grid Renewable Building Energy System Using Deep Reinforcement Learning

Wind power bidding coordinated with energy storage system operation in real-time electricity market: A maximum entropy deep reinforcement learning approach

Reinforcement Learning-Enhanced Adaptive Scheduling of Battery Energy Storage Systems in Energy Markets

Deep Reinforcement Learning for Community Battery Scheduling under Uncertainties of Load, PV Generation, and Energy Prices

Multi-Market Bidding Behavior Analysis of Energy Storage System Based on Inverse Reinforcement Learning

Proximal Policy Optimization Based Reinforcement Learning for Joint Bidding in Energy and Frequency Regulation Markets

Deep-Reinforcement-Learning-Based Capacity Scheduling for PV-Battery Storage System

Deep Reinforcement Learning-Based Scheduling Optimization of a Wind-solar Coupled CHP Unit System Considering Source-load Uncertainty

An optimal solutions-guided deep reinforcement learning approach for online energy storage control

Deep reinforcement learning-based optimal bidding strategy for real-time multi-participant electricity market with short-term load

Deep Reinforcement Learning-Based Method for Joint Optimization of Mobile Energy Storage Systems and Power Grid with High Renewable Energy Sources

Application-oriented assessment of grid-connected PV-battery system with deep reinforcement learning in buildings considering electricity price dynamics

Reinforcement Learning Driven Self-Optimizing Operation for Distributed Battery Electrical Storage System