Cooperated Online Learning and Optimization Operation of the Low Voltage Distribution System Considering User Electrical Characteristics

With the acceleration of energy reform, photovoltaic, energy storage, electric vehicles, and other new loads in low-voltage distribution networks have been rapidly developed. However, the distribution network with distributed power supply has some problems, such as imprecise power flow modeling and difficult coordination between various energy sources and loads, which bring challenges to the online optimization of the distribution network load curve. In this study, a multi-time scale online load optimal operation scheme of the distribution network is proposed by using the Bayesian online learning method. This scheme transforms the online power optimization of the distribution network into a Markov decision process. The output time of different energy sources is different, and the load with different user load characteristics is optimized. The scheme can track the state of the distribution network in real time and make the optimization scheme of multi-energy output online. Finally, an example is given to verify the effectiveness of the proposed method, which has theoretical significance for promoting the diversified development of low-voltage user-side load.

Introduction

With the rapid development of energy, a large number of photovoltaic (PV), electric vehicles (EV), and energy storage systems appear in the low-voltage distribution system. The Photovoltaic cell is an integral part of the main power supply, energy storage system as backup power supply, the photovoltaic power output when there is strong electricity storage, for lack of photovoltaic power generation, the energy storage discharge, and the load of power supply such as electric cars. The energy storage system can alleviate the influence of the randomness and fluctuation of photovoltaic power generation on the whole system operation. PV systems and energy storage systems are preferentially used to charge EV loads, avoiding the strong impact on the power system caused by EV loads directly connected to the large power grid. It can not only increase the consumption of new energy but also use the energy storage system to cut peak load and fill the valley, saving the cost of power distribution and capacity increase. The green operation mode of multi-load coordinated support in a low-voltage distribution system has great development potential. Li et al. (2022)and Xiong et al. (2020) proposed modeling and stability issues of voltage-source converter-dominated power systems. Cao et al. (2017) proposed a charging station construction scheme that combines photovoltaic and energy storage, namely, integrated optical storage and charging power station. The main components of the system and how to manage the power supply side of the system are introduced. Zhou et al. (2016) and Wu et al. (2021) constructed the topological structure of a single-bus DC microgrid containing distributed photovoltaic, mixed energy storage, and EV loads from the microlevel and proposed the hierarchical control strategy of line voltage for coordinating the operation of microgrid. On the basis of meeting the charging needs of EV users, Lu et al. (2014) took residential charging stations as the research object and established an optimization model aiming at minimizing the peak–valley difference of load. Through the demand response model, the optimized peak–flat–valley electricity price and the corresponding user responsiveness are solved. Zhao et al. (2015) proposed the optimal configuration of optical storage and grid-connected microgrid considering demand side response. Yu et al. (2017) proposed an optimization control model of microgrid with electric vehicles based on the multi-agent method in automatic demand response mode, aiming at the microgrid with distributed power supply and energy storage system as the power side and conventional load and electric vehicle load as the user side. Zhang et al. (2021) proposed to use the hydrogen production system to realize the optimal coordinated control of energy. Ren et al. (2018) proposed an optimization method for multi-time scale active and reactive power coordinated dispatching in an active distribution network based on model predictive control; however, there are many problems in the aforementioned methods , such as heavy computation, heavy reliance on prediction data, and difficulty in online control. Therefore, it is necessary to actively explore the collaborative optimization of optical storage and charge in the low-voltage distribution system and design a reasonable optimized operation and scheduling scheme, which can not only meet the needs of the power grid but also adapt to dynamic load change and achieve the maximum benefit and efficiency.

Studies have shown that the learning algorithm can be used to realize the learning optimization of part of the observable system, independent of the predicted data, and the reward feedback mechanism can realize the online optimization of the system. F et al. (2020) proposed a conditional probability prediction model by the dynamic Bayes method. R et al. (2020) designed a distributed optimization method based on Q-learning algorithm. The results showed that the reinforcement learning algorithm can quickly adapt to the power grid tidal wave motion in learning. However, traditional model-based algorithms need constant adjustment during operation. L et al. (2016) proposed a new algorithm with the combination of opposition learning and adaptive cross-generation differential evolution algorithm.

A low-voltage distribution system is an important part of electricity consumption. Because of the large quantity and wide distribution of low-voltage power consumption, it is necessary to standardize the data of low-voltage station areas. There has been a lot written about user characteristics. In the low voltage distribution system, customers have similar power characteristics to adjacent. Zhao et al. (2020), put forward in view of the low-pressure area of topology identification, showed that under the same area the user has electric similarity. Luo et al. (2016) provided effective data support for demand response such as peak-time electricity price formulation, staggered peak management, and load regulation. So this study, from the perspective of power characteristics, studies new coordinated control in the low voltage-power distribution system load, using online Bayesian learning methods, under the time-sharing electricity price, which is established aiming at the peak–valley load cutting optimization model; the real-time tracking online load distribution network, at the same time, meets under the dynamic change of load distribution network optimization, to implement the dynamic coordinated control of the low-voltage power distribution system; the effectiveness of the proposed strategy is demonstrated by applying it to a typical low-voltage platform area.

Mathematic Model

Photovoltaic Module

The generation power of the photovoltaic module at time t is closely related to the illumination intensity, and its output power is:

$P_{PV}=\frac{P_{SCC}{\overline{R}}_{AC}}{{\overline{R}}_{STC}}\left[1+{\gamma }_T\left({\mathrm{T}}_{ct}-T_{stc}\right)\right],(1)$

where $P_{\text{scc}}$ is the maximum output power of a photovoltaic module, kW; ${\overline{R}}_{\text{AC}}$ is actual irradiance, kW/m2; ${\overline{R}}_{\text{sTc}}$ is irradiance under standard test conditions; ${\gamma }_{\text{T}}$ is the power temperature coefficient; $T_{\text{ct}}$ is the actual operating temperature of the PV module; and $T_{\text{stc}}$ is the temperature of the PV module under standard test conditions (temperature 25°C).

Energy Storage Battery

The state of charge (SOC) of the battery reflects the ratio of the remaining power of the battery to the total capacity, and the mathematical model of the charge and discharge state is:

$S_{SOC}\left(\mathrm{t}+1\right)=\{\begin{array}{c}\left(1-\alpha \right)S_{SOC}\left(\mathrm{t}\right)+\frac{P_C{\eta }_c\mathrm{\Delta }t}{C_{bat}},\\ \left(1-\alpha \right)S_{SOC}\left(\mathrm{t}\right)+\frac{P_C\mathrm{\Delta }t}{{\eta }_d{C}_{bat}},\end{array}(2)$

where $S_{soc}\left(t+1\right)$ and $S_{soc}\left(t\right)$ are the charged state of the battery at the end of $\left(t+1\right)$ period and $\left(t\right)$ period, respectively; $\alpha$ is the self-discharge coefficient of the battery; $P_{\text{c}}$ and $P_d$ represent charging power and discharging power, respectively; $\text{Δ}t$ represents time intervals; and $C_{bat}$ represents battery capacity.

Electric Vehicles

EVs play a very important role in environmental protection and energy consumption reduction. They can be used not only as charging load in the microgrid but also as a reserve power source to release electric energy during peak load to maintain the energy balance and power supply reliability of the grid. Referring to the social survey data of family electric vehicle travel, it can be seen that the relationship between the driving distance and time of family electric vehicle travel basically corresponds to the lognormal distribution. Thus, the expression of its probability density function can be obtained through the calculation as follows:

$f\left(\mathrm{x}\right)=\frac{1}{2\delta \sqrt{2\mathrm{\Pi }}}\exp \left[-\frac{{\left(\ln \mathrm{x}-\mu \right)}^2}{2{\delta }^2}\right],(3)$

where $\mu$ means value, $\mu$ = 0.32; and $\delta$ means standard deviation, $\delta$ = 0.88.

EVs in the peak load can release the power according to their range and set the initial state of the power battery charged state to the lowest power battery charged state. Assuming that the electric car is charged at night, at the end of the trip, the power battery charged state calculation as shown in the following type, orderly in the electric vehicle charging and discharging mode, generally chooses the load slack period:

$g_i\left(\mathrm{x}\right)=1-\frac{E_i}{{\eta }_i{C}_{EV,i}}{f}_i\left(\mathrm{x}\right),(4)$

where $E_i$ represents the energy consumed per kilometer by the electric car, ${\eta }_i$ represents the discharge efficiency of each vehicle, $C_{\text{EV},i}$ represents the power battery capacity of each car, and $f_i\left(x\right)$ represents the daily mileage of each car.

Load Optimization Model Based on the Online Bayesian Learning Method

Description of the Project

In the distribution network with multiple new loads, the output scheme of new loads is optimized with the goal of minimizing the electricity cost of users under the time-sharing price. In addition, real-time tracking of the distribution network status and updating of the optimization scheme can be achieved in the case that new load access of the system changes rapidly over time. Because different load output and electricity price have different time characteristics, this study designs a multi-time scale online optimization scheme. According to different time intervals, the day is divided into several periods, and the load output is dynamically optimized according to the different time-shared price in each period, so as to realize the dynamic cooperative control of the low-voltage distribution system.

Online Bayesian Learning

Online learning is a model training method of machine learning. It can adjust the model quickly in real time according to the changes of online data, so that the model can reflect the changes online in time and improve the accuracy of online prediction. The goal of online learning is to minimize the loss of the entire function. Online learning does not need to determine the training data set in advance. The training data arrive one by one in the training process. Every time a training sample comes, the model will be iterated according to the loss function value, objective function value, and gradient generated by the sample. The main flow of online learning includes: the prediction results of the model are presented to the user and then the feedback data of the user are collected, which are used to train the model and form a closed-loop system, as shown in Figure 1.

FIGURE 1

FIGURE 1. Flow chart of online learning training.

In a Bayesian neural network by using the Bayesian formula on the prior probability and posterior probability, the relationship between the prior probability and posterior probability, with the initial set of standard normal distribution, can be found as a posteriori probability as the independent variable changes, in order to find out a posteriori probability, points to the independent variable on the value space. In order to simplify the process, MCMC (Markov Chains Monte Carlo) sampling is considered here to approximate the integral of the denominator, so that the posterior probability of the training set can be calculated and the neural network model can be obtained.

The Bayesian method can naturally derive the training method of online learning given the parameter in prior, calculate a posteriori probability according to feedback, take it as the prior probability of the next prediction, and then calculate a posteriori probability according to feedback.

Objective Function

When the system is running islands, if the daily load curve of the peak load is too big, it can lead to a line load increase. Considering the randomness of the efforts of photovoltaic, PV power, and energy storage discharge maximum constraints, the system stability and reliability in operation possibly cannot meet the needs of all load condition, thus appearing in the system suspend operation state. When the peak–valley load difference is large, the operating cost of the photovoltaic system and energy storage system in the low-voltage distribution system will increase. Therefore, in order to avoid too high peak value and too large peak–valley difference, this chapter takes peak clipping and valley filling as the objective function. The optimization time is 24 h in a day, and the decision variable is the change of electricity price in each period. To be sure, the objective function is calculated by using the load curve after collaborative optimization, so it is simplified here. The modules involved in collaborative control can be restricted by relevant constraints. Then, the objective function can be expressed as:

$C_1=\min \left(\max {\mathrm{E}}_T\right),(5)$

$C_2=\min \left(\max {\mathrm{E}}_T-\min {\mathrm{E}}_T\right),(6)$

where $E_T$ is the load after time-shared electricity price response. The first objective function represents the minimum peak value of system load, and the second objective function represents the minimum peak–valley difference of intra-day load. The two goals belong to the same latitude, so this chapter adopts the method of the weight coefficient to transform the two goals into a single goal problem. The objective function after transformation is:

$C_3={\lambda }_1{C}_1+{\lambda }_2{C}_2,(7)$

${\lambda }_1+{\lambda }_2=1,(8)$

where ${\lambda }_1$ and ${\lambda }_2$ are weight coefficients. Both goals reflect the user’s impact on the system’s peak cutting and valley filling, so they can both be set to 0.5.

Constraint Conditions

1) Customer satisfaction with electricity:

Customer satisfaction constraints:

$\alpha =1-\frac{{\displaystyle {\sum }_{t=1}^{24}\left|\mathrm{\Delta }E\left(\mathrm{t}\right)\right|}}{{\displaystyle {\sum }_{t=1}^{24}{E}_0\left(\mathrm{t}\right)}},(9)$

where $\Delta E\left(t\right)$ is the electric quantity change at moment t, $E_0\left(t\right)$ is the electric quantity consumption at moment t before implementing time-of-use electricity price, and A is the minimum satisfaction value of the electricity consumption mode.

2) Continuity constraint of energy storage state of charge:

$S_{\mathrm{O}\mathrm{C}}\left(i\right)=S_{\mathrm{O}\mathrm{C}}\left(i-1\right)+\frac{\left[{\eta }_{\mathrm{d}}{P}_{\mathrm{d}\mathrm{i}\mathrm{s}}\left(i\right)-{\eta }_{\mathrm{c}}{P}_{\mathrm{c}\mathrm{h}}\left(i\right)\right]\mathrm{\Delta }t}{E_{\max }},(10)$

where ${\eta }_{\text{c}}$ and ${\eta }_{\text{d}}$ correspond to the charge–discharge efficiency of BESS, respectively.

3) Charge/discharge state constraints:

$B_{dis}\left(i\right)$ and $B_{ch}\left(i\right)$

are 0–1 variables, where 1 represents the state of charge and 0 represents the state of discharge, satisfying the constraint.

$B_{\mathrm{d}\mathrm{i}\mathrm{s}}\left(i\right)+B_{\mathrm{c}\mathrm{h}}\left(i\right)\le 1.(11)$

4) Energy storage charge/discharge constraints:

During the operation of BESS, the power of each charge/discharge should not exceed its rated value, and the total discharge power should not exceed the rated power capacity of the energy storage:

$\{\begin{array}{l}0\le P_{\text{dis}}\left(i\right)\le B_{\text{dis}}\left(i\right)P_{\max },\hfill \\ 0\le P_{\text{ch}}\left(i\right)\le B_{\text{ch}}\left(i\right)P_{\max }.\hfill \end{array}(12)$

5) Limit the total amount of battery charges and discharges

The life of the energy storage battery is mainly affected by the charge and discharge state transition, that is, the charge and discharge times of energy storage. According to relevant studies, energy storage life is closely related to the total amount of charge and discharge in a day. Therefore, in order to reduce the number of charge and discharge times of household energy storage, the total amount of charge and discharge in a day is constrained.

${\displaystyle \sum _{t=1}^m{Q}_c\left(t\right)}\le Q_{\max },(13)$

where $Q_{max}$ is the biggest power.

Case Analysis

Parameter Settings

In this study, according to the reference that users in the same low-voltage station area have similar electrical characteristics, users with similar electrical characteristics in a low-voltage station area are directly selected as the typical scenario in this case. Moreover, different time-shared price users have different power consumption characteristics, and the user load curve will also be different. Here, industrial users are taken as an example, and the load curve is as follows: the PV curve of the area is shown in Figure 2, and the time-of-use electricity price table is shown in Table 1.

FIGURE 2

FIGURE 2. The PV curve of the area.

TABLE 1

TABLE 1. Time-shared electricity price.

It is assumed that the actual price of electricity before the implementation of time-of-use electricity is 0.56 yuan/KWH. Since this study is only for research and analysis, the value of the elasticity coefficient matrix of electricity price in reference (Yu et al., 2017) is directly taken, as shown in the following formula:

$M=\left[\begin{array}{ccc}-0.087& 0.0502& 0.0201\\ 0.0912& -0.0526& 0.0211\\ 0.0915& 0.0528& -0.0211\end{array}\right].(14)$

The demand response charging electricity price is set, the user’s peak electricity price is 0.35 yuan/KWH, the normal electricity price is 0.85 yuan/KWH, and the parameters required by the energy storage system are shown in Table 1.

The genetic algorithm was used to solve the model. The number of population was set at 100, the maximum genetic algebra was set at 500, the crossover rate was 0.8, and the mutation rate was 0.05. The proportional coefficient was set at 0.25, and the result of selecting the minimum objective function value after multiple calculations was the final result.

The model established in this study is a mixed-integer programming model, which uses the commercial solution CPLEX12.8 to solve and uses MATLAB by Yong (2016) for graph drawing and data analysis.

Operation Result

The optimization results are shown in Figure 3. According to the analysis of the calculation results, the user’s load in multiple time scales will change with the change of the new load, and the load curve and peak–valley difference will also change accordingly without being affected by other constraints. By comparing the load curves obtained by online learning and offline learning, it can be found that when online learning is not adopted, the load curve obtained by users under the objective function has little change; but when online learning is adopted, the load curve obtained by users under the objective function is relatively smooth, and the effect of peak clipping and valley filling is more obvious. It shows that the model in this study can realize the coordination of various new loads, get a better optimization effect, and effectively improve the economy of the power grid operation.

FIGURE 3

FIGURE 3. Load contrast diagram.

Conclusion

In the new multi-load low-voltage distribution system, the goal of peak cutting and valley filling is realized through the cooperative management and control of user load on a multi-time scale. At the same time, online Bayesian learning can track the distribution network in real time, adapt to the change of new load, dynamically adjust the load distribution, and realize the optimal collaborative optimization strategy. The aforementioned example verifies the effectiveness of the strategy.

Data Availability Statement

The original contributions presented in the study are included in the article/Supplementary Material; further inquiries can be directed to the corresponding author.

Author Contributions

Manuscript writing: JZ and JL; data collection: XJ and XX; content and format correction: ZD and KS. All authors have read and agreed to the published version of the manuscript.

Funding

This study received funding from the Collective Enterprise Science and Technology Project of State Grid Zhejiang Electric Power Co., Ltd.(HZJTKJ2021-14).

Conflict of Interest

JZ, JL, XJ, XX, ZD, and KS were employed by the company State Grid Zhejiang Anji County Power Supply Co., Ltd.

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors, and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Cao, L. (2017). Research on Key Technologies of Integrated Optical Storage and Charging Power Station Construction [J]. Electr. power energy 38 (06), 746–749+755. doi:10.19399/j.cnki.tpt.2018.12.012

CrossRef Full Text | Google Scholar

Fan, Y. N., and Lang, B. (2020). Probabilistic Prediction Method for Online Learning Performance Based on Dynamic Bayesian Networks [J]. Comput. Digit. Eng. 48 (10), 2313–2315+2326. doi:10.3969/j.issn.1672-9722.2020.10.002

CrossRef Full Text | Google Scholar

Li, J. L., Tan, Y. L., Wang, H., and Huang, J. (2022). Optimization Strategy of Energy Storage System Configuration in Distribution Network and Optical Storage Micro Network. High. Volt. Technol. 47 12,1–12. doi:10.13336/J.1003-6520.hVE-20201333

CrossRef Full Text | Google Scholar

Lu, X. Y., Liu, N., Chen, Z., Zhang, J. H., and Xiao, X. N. (2014). Multi-objective Optimization Scheduling Method of Photovoltaic Electric Car Charging Station [J]. Diangong Jishu Xuebao/Transactions China Electrotech. Soc. 29 (8), 46–56. doi:10.19595/j.cnki.1000-6753.tces.2014.08.006

CrossRef Full Text | Google Scholar

Luo, D. S., Du, G., Bie, S. Y., Pang, Z. G., and Li, S. G. (2016). Analysis of Residential Differentiated Power Consumption Behavior Based on Load Decomposition. Electr. power Syst. Prot. control 44 (21), 29–33. doi:10.7667/PSPC151908

CrossRef Full Text | Google Scholar

Ren, J. Y., Gu, W., Wang, Y., and Xiao, P. (2018). Multi-temporal Active Distribution Network Based on Model Predictive Control. Proc. CSEE 38 (5), 1397–1407. doi:10.13334/j.0228-8013.pcsee.2018.21.010

CrossRef Full Text | Google Scholar

Rozada, S., Apostolopoulou, D., and Alonso, E. (2020). “Load Frequency Control: A Deep Multi-Agent Reinforcement Learning Approach,” in Proceedings of the 2020 IEEE Power & Energy Society General Meeting (PESGM), Montreal, QC, Canada, Aug. 2020 (IEEE), 1–5. doi:10.1109/PESGM41954.2020.9281614

CrossRef Full Text | Google Scholar

Wu, F., Zhou, Y., Feng, D. H., Shi, Y. W., Li, H. J., and Lei, T. (2021). Light ChuChong Integration Quick Charge Station Has Run Strategy [J]. Electr. Meas. Instrum. 58 (12), 104–109. doi:10.19753/j.issn1001-1390.2021.12.015

CrossRef Full Text | Google Scholar

Xiong, L. S., Liu, X. K., Liu, Y. H., and Zhuo, F. (2020). Modeling and Stability Issues of Voltage-Source Converter Dominated Power Systems: a Review. CSEE J. Power Energy Syst. 8 12, 78–92. doi:10.17775/CSEEJPES.2020.03590

CrossRef Full Text | Google Scholar

Yong, L., Lei, F., and Yu, W. (2016). “Opposition Learning Adaptive Cross-Generation Differential Evolution Algorithm Based Multi-Objective Optimization of Rolling Schedule for Tandem Cold Rolling,” in Proceedings of the 2016 IEEE International Conference on Information and Automation (ICIA), Ningbo, China, Aug. 2016 (IEEE), 319–323. doi:10.1109/ICInfA.2016.7831843

CrossRef Full Text | Google Scholar

Yu, W. W., Liu, S. L., Chen, Q. G., and Zhang, C. (2017). Multi-objective Optimization Scheduling Method for Photovoltaic Microgrid Considering Demand Side Management [J]. J. Sol. energy 38 (11), 2972–2981.

Google Scholar

Zhang, K., Zhou, B., Or, S. W., Li, C., Chung, C. Y., and Voropai, N. (2021). Optimal Coordinated Control of Multi-Renewable-To-Hydrogen Production System for Hydrogen Fueling Stations. IEEE Trans. Ind. Appl. 58, 2728–2739. doi:10.1109/TIA.2021.3093841

CrossRef Full Text | Google Scholar

Zhao, B., Bao, K. K., Xu, Z. C., and Zhang, Y. B. (2015). Optimal Configuration of Optical Storage and Grid-Connected Microgrid Considering Demand Side Response [J]. Proc. csee 35 (21), 5465–5474. doi:10.13334/j.0258-8013.pcsee.2015.21.010

CrossRef Full Text | Google Scholar

Zhao, J., Li, L., Xu, Z., Wang, X., Wang, H., and Shao, X. (2020). Full-Scale Distribution System Topology Identification Using Markov Random Field. IEEE Trans. Smart Grid 11 (6), 4714–4726. doi:10.1109/TSG.2020.2995164

CrossRef Full Text | Google Scholar

Zhou, N., Fan, W., Liu, N., Lin, X., and Zhang, J. H. (2016). Multi-Objective Capacity Optimization Configuration of Photovoltaic Microgrid Energy Storage System Based on Demand Response. Power grid Technol. 40 (06), 1709–1716. doi:10.13335/j.1000-3673.pst.2016.06.015

CrossRef Full Text | Google Scholar