Zhiyuan Zhang ¹ Yongjun Wu ¹ Zhenkun Hao ¹ Minghui Song ¹ Peipei Yu ^2*

• ¹ State Grid Beijing Electric Power Company, Beijing, China
• ² Engineering Research Center of Offshore Wind Technology Ministry of Education (Shanghai University of Electric Power), Shanghai, China

Nowadays, real-time price (RTP) is an effective approach to cope with increasingly uncertain renewables, which can guide demand-side behavior to achieve peak shaving and valley filling.In modern cities, district cooling systems (DCSs) are considered ideal resources providing demand response, because they account for a large share of electricity and connect with multiple buildings with large thermal inertia. Especially, thermal energy storage in DCSs can be controlled strategically through storing/melting ice to maximize cost savings, following dynamic RTP. However, it is challenging to obtain an accurate prediction of future cooling demands and RTP, which further complicates the strategic real-time control. This study proposes a novel curiosity-driven reinforcement learning (RL) approach to realize real-time optimal control of thermal energy storage (i.e., ice storage) in DCSs. Firstly, the proposed approach introduces an intrinsic curiosity as compensation to conventional RL, which can facilitate RL explorations and improve training efficiency. Moreover, coupled with a long short-term memory (LSTM) network module, the proposed approach can further capture the future trends of the time-series data, e.g., demands and RTP. Finally, simulated experiments are carried out based on real-world data in Hengqin, China, to verify the effectiveness of the proposed approach.