Energy management method of Integrated Energy System based on energy and carbon pricing strategy and Reinforcement Learning approach

Hao Zhang ¹ Xuntao Shi ^2* Hai Lu ² Enbo Luo ¹ Yuanlong Zhang ³ Kairan Li ² Tong Liu ² Min Xu ²

• ¹ Electric Power Research Institute of Yunnan Electric Power Grid Co., Ltd, Kunming, Yunnan Province, China
• ² CSG Electric Power Research Institute Co., Guangdong, China
• ³ Yunnan Power Grid Co., Ltd. Honghe Power Supply Bureau, Honghe, China

Focusing on the low-carbon economic operation of an integrated energy system (IES), this paper proposes a novel energy-carbon pricing and energy management method to promote carbon emission reductions in the IES based on the carbon emission flow theory and reinforcement learning (RL) approach. Firstly, an energy-carbon integrated pricing model is proposed. The proposed pricing method charges prosumers by tracing the embedded carbon emissions of energy usages, and establishes an energy-carbon-prices relationship between the power grid, IES and prosumers. Secondly, an energy management model considering the energy-carbon integrated pricing strategy is established based on the Markov decision processes (MDP), including prosumers energy consumption cost model and energy service provider (ESP) profit model. Then, a solving method based on the RL approach is proposed. Finally, numerical results show that the proposed method can improve operation economy and reduce carbon emissions of IES. When carbon price accompanying electricity and thermal is considered in the process of pricing and energy management, the profit of ESP can be improved and the cost of prosumers can be reduced, and the total carbon emission of IES can be reduced by 5.75% compared with not considering carbon price..