AUTHOR=Liu Zhao , Li Jiateng , Zhang Pei , Ding Zhenhuan , Zhao Yanshun TITLE=An AGC Dynamic Optimization Method Based on Proximal Policy Optimization JOURNAL=Frontiers in Energy Research VOLUME=10 YEAR=2022 URL=https://www.frontiersin.org/journals/energy-research/articles/10.3389/fenrg.2022.947532 DOI=10.3389/fenrg.2022.947532 ISSN=2296-598X ABSTRACT=

The increasing penetration of renewable energy introduces more uncertainties and creates more fluctuations in power systems than ever before, which brings great challenges for automatic generation control (AGC). It is necessary for grid operators to develop an advanced AGC strategy to handle fluctuations and uncertainties. AGC dynamic optimization is a sequential decision problem that can be formulated as a discrete-time Markov decision process. Therefore, this article proposes a novel framework based on proximal policy optimization (PPO) reinforcement learning algorithm to optimize power regulation among each AGC generator in advance. Then, the detailed modeling process of reward functions and state and action space designing is presented. The application of the proposed PPO-based AGC dynamic optimization framework is simulated on a modified IEEE 39-bus system and compared with the classical proportional−integral (PI) control strategy and other reinforcement learning algorithms. The results of the case study show that the framework proposed in this article can make the frequency characteristic better satisfy the control performance standard (CPS) under the scenario of large fluctuations in power systems.