Bidding Strategy for Trading Wind Energy and Purchasing Reserve of Wind Power Producer – A DRL Based Approach

Di Cao,Weihao Hu,Xiao Xu,Tomislav Dragicevic,Qi Huang,Zhou Liu,Zhe Chen,Frede Blaabjerg
DOI: https://doi.org/10.1016/j.ijepes.2019.105648
IF: 5.659
2020-01-01
International Journal of Electrical Power & Energy Systems
Abstract:Wind power producers (WPP) are punished when take part in the short-term electricity market due to the inaccuracy of wind power prediction. The profit loss can be partially offset by strategic reserve purchasing. Due to the uncertainty of real-time wind power production, the reserve capacity price and the highly dynamic price regulation, it is difficult to determine the quantity of reserve to purchase, which might have a great impact on the profit of WPP. This paper investigates the possible influence on the revenue of WPP if they take part in both the energy and reserve market. This problem is first formulated as a Markov decision process (MDP). After that, the asynchronous advantage actor-critic (A3C) method is used to take care of this problem. Several agents are applied to explore the action space simultaneously. Neural networks are applied to fit the policy function and value function, which are trained against each other to learn the optimal dynamic bidding strategy from historical data by constantly trial and error. Simulation results of a wind farm in Denmark validate that the profit loss of the WPP can be significantly reduced when the A3C algorithm is used for bidding strategy formulation when WPP participate in both the energy and reserve market.
What problem does this paper attempt to address?