Analysis on Influence of Residents’ Response Probability Distribution on Load Aggregation Effect

Residential loads are essential resources, and they can participate in grid scheduling through aggregation, with great potential. However, inherent uncertainties challenge the realization of accurate load aggregation (LA), leading to the failure to fully play a role in grid operation. In this context, this paper analyzes the influence of the probability of residents participating in LA on the actual aggregation effect. Firstly, the LA problem is modeled, and the optimization objective is equivalently transformed from minimizing the mismatch between the actual load regulation and the target to minimizing the sum of variances of the overall response probabilities. Response probabilities of residents are then simulated based on the β distribution model, and various typical distributions can be generated by modifying only two hyperparameters. In order to evaluate the actual aggregation effect, a multi-armed bandit model that can be used for LA is adopted, and an evaluation framework is designed. The simulation results show that for different probability distributions, the smaller the sum of variances of all residents’ response probabilities are, the more accurate aggregation can be achieved.

1 Introduction

Modern power grids are transitioning to a low-carbon energy system, and traditional coal-fired units are gradually retiring (Zhou et al., 2021). Renewable energy sources (RES) represented by photovoltaics and wind power have gained unprecedented development opportunities. According to IRENA, the total installed capacity of global RES in 2021 is 3064 GW, accounting for 38%, with a year-on-year increase of 257 GW (IRENA, 2022). The widespread access of a high proportion of RES will become the fundamental feature of new power systems in the future. However, the output of RES is random and uncertain, and large-scale grid connections will bring significant challenges to grid scheduling. In recent years, many severe power outages have occurred, such as “2019·8·9” in the United Kingdom (Sun et al., 2019) and “2020·8·14” in California (Hu et al., 2020).

To solve these problems, researchers pay attention to the flexible, controllable loads with increasing proportion, which have great potential to regulate their operating power quickly without affecting users’ comfort (Chen et al., 2018; Qi et al., 2020). In this context, load aggregation technology emerged as the times required. Appropriate control strategies aggregate many scattered demand-side resources into a whole managed by load aggregators (Burger et al., 2017). Through specific electricity prices or incentives means, users are urged to regulate their power consumption patterns to assist grid scheduling (Sun et al., 2017). For residential thermostatically controlled loads (RTCLs) with energy storage properties (Hao et al., 2015), such as air conditioners and water heaters, the power regulation speed is faster than traditional thermal power plants and hydropower plants. Moreover, RTCLs have flexible control methods and low response costs, which are ideal for participating in LA (Gong et al., 2020).

In order to fully utilize the effect of RTCLs on the grid, a vital requirement is to achieve accurate and reliable load aggregation (Saleh et al., 2017). However, this is a challenging problem: the residents’ response behavior to the load regulation signal is uncertain (Mathieu et al., 2013), which is affected by many factors, such as incentives (Qi et al., 2020), weather (Kou et al., 2021), electricity usage habits (Baek et al., 2021), and the load aggregator’s knowledge of these factors is limited. Accurately estimating these uncertainties is essential. Overestimating will reduce the utilization of RTCLs while underestimating will blindly send load regulation instructions, which will cause the actual aggregation to deviate from the target and bring risks to the power grid (Syrri and Mancarella, 2016).

Much research has been done on the uncertainty of residents’ response behavior. Li et al. (2017) derived the relationship between aggregated power and temperature based on the first-order thermal parameter model of RTCLs and established an approximate aggregation model of RTCLs. Luo et al. (2020) established the uncertainty model of the load regulation by analyzing the characteristics of different types of loads and using the probability method and fuzzy theory. Sun et al. (2020) considered the uncertainty of electricity consumption habits to establish a user preference model and obtained a probability estimate of load regulation amount through Bayesian inference. Gong et al. (2020) designed a hierarchical robust control mechanism to achieve precise power tracking by aggregating RTCLs in uncertain time-varying environments.

Load aggregation is a behavior that residents voluntarily participate in. Most of the existing studies only model the regulation power of residents in the LA scenario, ignoring the relevant research on their response probabilities. In addition, in existing pilots, in order to improve the probabilities of users actively participating in LA, blind incentives such as currency are often used, resulting in high costs. Therefore, analyzing the impact of the residents’ response probability distribution on the aggregation effect can provide a reference for load aggregators to formulate plans and make more effective use of RTCLs.

In order to fill the above research gap, aiming at the scenario of residential loads aggregation, this paper derives and verifies the mapping relationship between the response probability and the actual aggregation effect. The main contributions of this paper include the following:

1) A response probability generation model based on the β distribution is proposed, which can simulate a variety of typical distributions by modifying only two hyperparameters, and derives the relationship between the aggregation effect and the response probability distribution through theoretical analysis.

2) An evaluation framework of the load aggregation effect based on the multi-armed bandit (MAB) model is designed, which can be utilized to analyze the effect of response probability on the aggregation effect.

The rest of the paper is organized as follows: Section 2 models the load aggregation problem from a probabilistic perspective and derives a preliminary theory. In Section 3, a simulation method of residents’ response probability distribution is proposed, and the corresponding analysis is carried out. Section 4 designs a framework for evaluating the effect of LA, and based on this, a case study is carried out to verify our theory in Section 5. Section 6 concludes this work.

2 Load Aggregation Model

Residents’ response to aggregation instructions is uncertain, influenced by various factors, such as age, education, and temperature. The concrete manifestation of this uncertainty is whether the user agrees to regulate the load. This paper abstracts it from the perspective of probability and sets the probability that user i agrees to regulate as p_i, that is, the response probability, and can regulate the load of 1 unit. The binary variable X_i is introduced to characterize the response result. Obviously, X_i obeys the Bernoulli distribution, that is, X_i ∼ Bern(p_i):

$X_i=\{\begin{array}{cc}1& p_i\\ 0& 1-p_i\end{array}(1)$

According to the properties of Bernoulli distribution, when the instruction is issued to resident i, the expected value of its load regulation E (X_i) = p_i, and the variance ${\sigma }_i^2=p_i\left(1-p_i\right)$.

We consider such a scenario: in the t-th event, the power system needs to curtail the load D, and the load aggregator should select a subset S_t ⊆ [n] among all n users in the area A_n to send instructions. The goal is to minimize the deviation G_t(t) between the actual curtailment and D:

$\min G_t\left(t\right)={\left({\displaystyle {\sum }_{i\in S_t}{X}_{t,i}-D}\right)}^2(2)$

Since X_t,i is a random variable, Eq. 2 should be modified to minimize the mathematical expectation of G_t(t). Moreover, the expectation operator can be removed by rigorous derivation, and the simplified result is as follows:

$\underset{S_t\subseteq \left[n\right]}{\min }\mathbb{E}{G}_t\left(t\right)\iff \underset{S_t\subseteq \left[n\right]}{\min }{\left({\displaystyle \sum _{i\in S_t}{p}_i-D}\right)}^2+{\displaystyle \sum _{i\in S_t}{p}_i\left(1-p_i\right)}(3)$

Eq. 3 is a combinatorial optimization problem, and the decision variable is the set S_t of selected residents. However, it is NP-hard, and only approximate algorithms can be adopted. Especially when the number of residents n and the target D are large, many residents need to be selected each time, which will produce a “Combinatorial Explosion.”

It can be seen from Eq. 3 that the actual load aggregation deviation G_t(t) at the t-th event is affected by three factors, the target D, the selected set S_t, and the response probability set ℙ_t = {p_i: i∈S_t}. These three are not independent of each other because D affects the determination of S_t, and S_t determines ℙ_t. It should be noted that the power grid may be in various working states, such as different levels of frequency fluctuations, which results in a time-varying value of D issued to the load aggregator in A_n rather than a fixed value. Under this background, in order to minimize G_t(t) for any D, two critical aspects should be considered: 1) the determination of S_t, that is, the users’ selection algorithm, mainly affects the first term on the right side of Eq. 3; 2) the set of response probabilities of all residents in A_n: ℙ = {p_i: ∀ i ∈[n]}, affecting the second term.

The core of this paper is not how to solve S_t accurately but mainly focuses on analyzing the effect of ℙ on the aggregation effect, which is essentially more fundamental than studying the precise selection strategy. For the convenience of analysis, intuition is to approximately transform Eq. 3 into the following optimization problem and propose our theory:

$\min {\displaystyle \sum _{i\in \left[n\right]}{p}_i\left(1-p_i\right)}\iff \min {\displaystyle \sum _{i\in \left[n\right]}{\sigma }_i^2}=f\left(\mathrm{\mathbb{P}}\right)(4)$

Theorem 1. For M different ℙ residents groups Ψ = {ℙ₁, ℙ₂, … , ℙ_M}, the one with the smallest sum of variances can achieve the most accurate load aggregation, defined as the optimal response probability distribution ℙ*:

${\mathrm{\mathbb{P}}}^{\ast }=\left\{{\mathrm{\mathbb{P}}}_u|\forall {\mathrm{\mathbb{P}}}_v\in \mathcal{P}:f\left({\mathrm{\mathbb{P}}}_u\right)\le f\left({\mathrm{\mathbb{P}}}_v\right)\right\}(5)$

For rigorous analysis, the probability sets ∀ ℙ∈Ψ studied in this paper should all have the same mean p_mean, ${\overline{\mathrm{\mathbb{P}}}}_1={\overline{\mathrm{\mathbb{P}}}}_2=\dots ={\overline{\mathrm{\mathbb{P}}}}_M=p_{mean}$, which is an important constraint. For instance, for two probability sets ${\overline{\mathrm{\mathbb{P}}}}_1$ and ${\overline{\mathrm{\mathbb{P}}}}_2$ with mean values of 1 and 0.1, respectively, given the same regulation target D, it is clear that the response of the latter will be poor or even no response, resulting in a high aggregation mismatch. In this case of inequality, it is pointless to analyze still the effect of the probability distribution on the aggregation, so it should be restricted to have the same mean.From the non-negativeness of the variance ${\sigma }_i^2$, it can be seen that the sum f(ℙ) ≥ 0, if and only if p_i is 0 or 1, the equal sign is true, as shown in Eq. 6. Therefore, for sets with the mean p_mean, the optimal distribution ℙ* that can achieve the minimum aggregation mismatch is given by Eq. 7.

$f\left(\mathrm{\mathbb{P}}\right)=0\iff p_i\in \left\{\mathrm{0,1}\right\},\forall p_i\in \mathrm{\mathbb{P}}(6)$

${\mathrm{\mathbb{P}}}^{\ast }=\left\{{\mathrm{\mathbb{P}}}_u|{\mathrm{\mathbb{P}}}_u^1={\mathrm{\complement }}_{{\mathrm{\mathbb{P}}}_u}{\mathrm{\mathbb{P}}}_u^0:\left|{\mathrm{\mathbb{P}}}_u^1\right|=\left[n{p}_{mean}\right],\left|{\mathrm{\mathbb{P}}}_u^0\right|=n-\left|{\mathrm{\mathbb{P}}}_u^1\right|\right\}(7)$

where the symbol " $\mathrm{\complement }$" represents the complement, and ${\Psi }_u^1$ and ${\Psi }_u^0$ represent the subsets in ℙ_u whose values are 0 and 1, respectively. “|·|" is defined as the cardinality of a finite set, and " [·]" means rounding.

3 Response Probability Sets Generation Method

The core of this paper is to analyze the impact of residents’ response probabilities set ℙ on the actual aggregation effect, and a response probability generation model should be designed to simulate ℙ. The shape of the probability distribution is infinitely different, not all cases can be obtained, and it is not necessary. Therefore, this paper adopts the β distribution, which is vital in the field of machine learning, which is a parametric probability distribution model that only contains two parameters, α and β > 0, usually denoted as Y ∼ Be (α, β). By setting the parameters, a variety of typical probability distributions can be simulated, all within the interval (0, 1), and then M × n random sampling is performed to generate a set Ψ containing M kinds of ℙ. Some properties of the β distribution are as follows:

3.1 Probability Density Function

$f\left(Y;\alpha ,\beta \right)=\frac{\mathrm{\Gamma }\left(\alpha +\beta \right)}{\mathrm{\Gamma }\left(\alpha \right)\Gamma \left(\beta \right)}{Y}^{\alpha -1}{\left(1-Y\right)}^{\beta -1}(8)$

where Γ(z) is the Γ function.

3.2 Expectation and Variance

$\mu =E\left(Y\right)=\frac{\alpha }{\alpha +\beta }(9)$

$Var\left(Y\right)=E{\left(Y-\mu \right)}^2=\frac{\alpha \beta }{{\left(\alpha +\beta \right)}^2\left(\alpha +\beta +1\right)}(10)$

Assuming that the elements ℙ within Ψ have the same mean μ, combined with Eq. 9, we can derive the relationship between α and β:

$\beta =\alpha \left(\frac{1}{\mu }-1\right)(11)$

Therefore, a probability set ℙ is generated based on the β distribution, denoted by ℙ ∼ Be [α, α(1/μ-1)]. Only by modifying the parameter α, the set Ψ of M probability distributions can be obtained. In addition, the β distribution has another property: as α increases monotonically, the variance Var(ℙ) of ∀ p_i ∈ ℙ will decrease monotonically according to the trend of the inverse proportional function, which means that p_i gradually approaches the fixed mean μ, and the degree of dispersion decreases. The relationship between the two is as follows:

$Var\left(\mathrm{\mathbb{P}}\right)=\frac{\alpha \beta }{{\left(\alpha +\beta \right)}^2\left(\alpha +\beta +1\right)}=\frac{{\mu }^2\left(1-\mu \right)}{\alpha +\mu }(12)$

Further, when the number of residents n is large enough, we can simplify Eq. 4 to eliminate the summed symbol Σ and analyze the analytical relationship between Eqs 4, 12. Before that, two assumptions are listed first due to the limited sample size.

Assumption 1. The mean of the probability sample p_i is equal to the expectation of the β distribution, which ℙ follows:

${\displaystyle \sum _{i\in \left[n\right]}{p}_i}/n=\mu$

Assumption 2. The sample mean of the probability square $p_i^2$ is equal to the expectation of the distribution, which follows:

${\displaystyle \sum _{i\in \left[n\right]}{p}_i^2}/n=E\left(p^2\right)$

Theorem 2. The relationship between the sum of variances f(ℙ) of the probabilities and the variance Var(ℙ) of the β distribution is:

$f\left(\mathrm{\mathbb{P}}\right)=n\left[\mu \left(1-\mu \right)-Var\left(\mathrm{\mathbb{P}}\right)\right]=\frac{n\mu \left(1-\mu \right)}{1+\frac{\mu }{\alpha }}(13)$

As can be seen from Eq. 13, the trend of change between the two is precisely the opposite. f(ℙ) is a monotonically increasing function of α, which means that the aggregation mismatch becomes larger and less precise as the α increases.

4 Evaluation Framework of Load Aggregation Effect

In this section, we first introduce a CUCB-Avg algorithm that can be adopted to select user sets in load regulation events, and based on this, we design a framework to evaluate the impact of the residents’ response probability set ℙ on the LA effect.

4.1 Users Selection Algorithm

The response probability set ℙ of residents is unknown to the load aggregator, and it is a complex problem how to select appropriate users to send the regulation instructions in load regulation events. This is similar to the setting of the classic MAB problem: for a slot machine with multiple arms, after each arm is pulled, it will obtain a payoff that obeys an unknown distribution. How can the gambler obtain the maximum cumulative payoff within a limited number of pulls? If each resident corresponds to an arm, its actual curtailment X_i can be regarded as the reward of this arm. The common denominator is to understand the uncertainty of an arm/user to make a choice. Unlike the MAB problem, the goal of LA is to minimize the mismatch Eq. 2.

MAB is an essential framework for dealing with uncertainty problems, and typical sub-methods include CUCB (Chen et al., 2013), UCB1 (Auer et al., 2002), and LinUCB (Li et al., 2010), etc., For the LA scenario, this paper introduces an improved CUCB, the CUCB-Avg algorithm (Li et al., 2020), to select the residents set S_t. This method considers the sample mean ${\overline{p}}_i$ of p_i and assigns it a confidence interval. From an optimistic point of view, the actual value of p_i is estimated with upper confidence bound and bounded to [0,1],

$G_i\left(t\right)=\min \left({\overline{p}}_i\left(t-1\right)+\sqrt{\frac{\alpha \ln t}{2T_i\left(t-1\right)}},1\right)(14)$

where α is a positive constant used to weigh the sample mean and confidence interval. T_i (t-1) is the number of times user i has been selected in the past (t-1) events. Based on Eq. 14, the CUCB-Avg algorithm mainly includes three steps to select users: 1) calculating G_i(t) for each user i, 2) selecting the highest m_t users in descending order and sending instructions, 3) updating incrementally according to the actual response feedback. The specific process of the algorithm is as follows:

Algorithm 1. CUCB-Avg.

4.2 Evaluation Framework

In load regulation events, due to the uncertainty and randomness of residents, CUCB-Avg’s estimation of the actual probability p_i is not entirely accurate, and there is still a mismatch between the actual aggregation amount and the target D. In this paper, the normalized root mean square error (NRMSE_D) is adopted to evaluate the effect of aggregation, where D_act,k is the actual aggregation amount of K times for the target D.

$NRMS{E}_D=\sqrt{\frac{{\displaystyle \sum _{k=1}^K{\left(D_{act,k}-D\right)}^2}}{K}}/D(15)$

Based on this, we propose a framework for evaluating the impact of response probability on the aggregation effect, as shown in Figure 1.

FIGURE 1

FIGURE 1. Evaluation framework.

Step 1. Set M values {α₁, α₂, ..., α_M } of the parameter α, and calculate β according to Eq. 11. Generate the corresponding sets of response probabilities for all residents based on the β distribution Ψ = {ℙ₁, ℙ₂, … ,ℙ_M};

Step 2. Train the CUCB-Avg model with different ℙ as inputs;

Step 3. Based on the algorithm’s understanding of uncertainties, the Monte Carlo method is adopted to perform K events, and NRMSE_D is calculated according to Eq. 15;

Step 4. Compare NRMSE_D corresponding to different α, and evaluate the effect of ℙ on the aggregation effect.

5 Case Study

To verify the theories derived in the previous section, we conduct simulations based on the proposed evaluation framework to analyze the impact of different sets ℙ of response probabilities on the actual aggregation effect. We focus on the area A_n with n = 1000 residents, equipped with intelligent terminals that can participate in LA, and the adjustable capacity contracted with the load aggregator is 2.5 kW, which is equivalent to 1 unit load.

5.1 Setting of Response Probability Sets

Before analysis, Ψ should be generated containing various probability distributions based on the β distribution model. Additionally, we set all distributions with the same mean p_mean = μ = 0.7. Specifically, it includes the following steps:

1) Set the parameter α in the interval [0.01, 5], and take M = 20 values in equal steps;

2) Calculate β based on Eq. 11;

3) Generate n random numbers following the β distribution, which correspond to the response probabilities of n residents, and the set is ℙ_j, j = 1, 2 … , M;

As α increases, Var(ℙ) is shown in Figure 2. It can be seen from the figure that Var(ℙ) is monotonically decreasing and similar to the hyperbolic function, which is consistent with Eq. 12.

FIGURE 2

FIGURE 2. The variance of different β distributions.

Through the above steps, Ψ of M probability distributions with the same mean μ is generated. Due to a large amount of data, it is not clear to visualize the dataset with a histogram. Therefore, we adopt a curve graph and heat map as follows:

1) Curve graph. For ℙ_j, sort the response probability p_i in descending order, and draw M curves. The horizontal axis represents the residents, and the vertical axis represents the response probability, as shown in Figure 3:

FIGURE 3

FIGURE 3. Curve graph.

It can be seen that as α increases, the curve gradually slopes and approaches the horizontal dashed line with a mean μ = 0.7. This is because the variance Var(ℙ) gradually decreases, and the value of p_i shrinks to 0.7.

2) Heat map. It is similar to the previous graph but is equivalent to looking down on the X-Y plane on the Z-axis of the three-dimensional coordinate system. M curves are arranged in parallel at equal intervals. Different colors represent the value of p_i. Y-axis represents α, and the corresponding M color bands are drawn, as shown in Figure 4:

FIGURE 4

FIGURE 4. Heat map.

Figure 4 shows the distribution of each ℙ_j more intuitively. When α = 0.01, ℙ₁ has almost only probability values at both ends of the color band, namely 0 and 1. As α grows, the probabilities corresponding to the color of the middle segment gradually increase and approach 0.7.

In addition, the variation of f(ℙ) with the parameter α should also be supplemented, as shown in Figure 5. It should be noted that although Figures 2, 5 are both about variance, their meanings are entirely different: the former means that α will produce different β distributions, and Var(ℙ) of the distribution is calculated based on Eq. 10; the latter is related to the setting that the load regulation follows the Bernoulli distribution, and the sum f(ℙ) of variances is calculated based on Eq. 4. The mathematical relationship between the two is shown in Eq. 13.

FIGURE 5

FIGURE 5. The sum of variances corresponding to different α

5.2 Load Aggregation Deviation

CUCB-Avg model is trained for T = 1000 times and performs K = 300 Monte Carlo simulations to calculate NRMSE_D. Additionally, to highlight the superiority of CUCB-Avg, here we introduce a conventional random selection (RS) method as the benchmark (Chen et al., 2020). RS only focuses on the average p_mean of ℙ. For a given target D, calculate the number of users N_RS to be selected based on Eq. 16, and then send instructions randomly.

$N_{RS}=D/p_{mean}(16)$

The relative relationship between the target D and the number of residents n (or the maximum adjustable capacity Q) also affects the aggregation mismatch. To eliminate this effect and purely analyze the function of ℙ, we set D to be 0.8, 1.2, and 1.6 MW, respectively. NRMSE_D is calculated based on CUCB-Avg and RS, as shown in Figures 6–8.

FIGURE 6

FIGURE 6. NRMSE_D of D = 0.8 MW.

FIGURE 7

FIGURE 7. NRMSE_D of D = 1.2 MW.

FIGURE 8

FIGURE 8. NRMSE_D of D = 1.6 MW.

As can be seen from the three figures,

1) The curve obtained by CUCB-Avg is significantly lower than RS, achieving smaller NRMSE_D and better performance.

2) The curve corresponding to CUCB-Avg rises significantly, consistent with Theorem 1 and Theorem 2: a larger α will lead to a larger f(ℙ) and a worse aggregation accuracy. Especially when α is 0.01, the error achieved by CUCB-Avg is only 0.15%. Moreover, as α increases, the two curves gradually approach because the variance of the β distribution decreases, making all p_i close to μ, and S_t selected by the two methods are similar, resulting in an approximate aggregation effect.

3) The curve corresponding to RS fluctuates wildly, and there is no apparent upward trend when D is small. This does not mean that ℙ does not affect the aggregation effect. It is because RS has been randomly selected without a strategy, and a different set of users is determined in each Monte Carlo simulation. When D = 1.6 MW is close to the maximum adjustable capacity $Q=2.5n{p}_{mean}$ kW = 1.75 MW, almost all users are selected to send instructions. Therefore, the effect of ℙ on the aggregation effect is more prominent, showing a slight upward trend.

4) As D increases, the gap between the two curves gradually narrows. When D is small, only a part of users need to be selected to send instructions, and CUCB-Avg strategically determines S_t, so it has more obvious advantages than RS.

6 Conclusion

With the transition from modern power grids to low-carbon energy systems, residential load resources on the demand side play an increasingly important role. Aiming at the uncertainty of residents’ response behavior in the LA scenario, this paper first models the problem. In order to facilitate the analysis of the response probabilities, the optimization objective is equivalently transformed from minimizing the mismatch between the actual load regulation amount and the target to minimizing the sum of variances of the overall response probabilities. Then a response probability generation model based on the β distribution is proposed, which can simulate a variety of typical distributions only by modifying two hyperparameters. Moreover, explore the connection between the β distribution and the sum of probabilities variances.

Furtherly, this paper adopts a CUCB-Avg algorithm whose performance is better than the conventional random method, which can be utilized to select appropriate users to send instructions in each event. Based on this, a framework for evaluating the effect of the sum of variances on the aggregation effect is designed. The simulation results show that the smaller the sum of variances for different probability distributions, the more accurate load aggregation can be achieved.

Residents’ response probabilities can be improved through incentives. Future work will study incentives allocation strategies for limited costs to improve the probability distribution and further enhance the aggregation effect.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Author Contributions

WZ: model construction and writing the manuscript. QH: CUCB-Avg algorithm and literature review. XY: literature review.

Funding

This research is supported by the National Natural Science Foundation of China (51907026), the Natural Science Foundation of Jiangsu Province (BK20190361), the Key Research and Development Program of Jiangsu Province (BE2020081-2).

Conflict of Interest

XY was employed by the Company Taizhou Power Supply Company State Grid Jiangsu Electric Power Co., Ltd.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Auer, P., Cesa-Bianchi, N., and Fischer, P. (2002). Finite-time Analysis of the Multiarmed Bandit Problem. Mach. Learn. 47, 235–256. doi:10.1023/A:1013689704352

CrossRef Full Text | Google Scholar

Baek, K., Lee, E., and Kim, J. (2021). Resident Behavior Detection Model for Environment Responsive Demand Response. IEEE Trans. Smart Grid 12, 3980–3989. doi:10.1109/TSG.2021.3074955

CrossRef Full Text | Google Scholar

Burger, S., Chaves-Ávila, J. P., Batlle, C., and Pérez-Arriaga, I. J. (2017). A Review of the Value of Aggregators in Electricity Systems. Renew. Sustain. Energy Rev. 77, 395–405. doi:10.1016/j.rser.2017.04.014

CrossRef Full Text | Google Scholar

Chen, W., Wang, Y., and Yuan, Y. (2013). “Combinatorial Multi-Armed Bandit: General Framework, Results and Applications,” in Proceedings of the 30th International Conference on Machine Learning, Atlanta, GA, June 6–21, 2013, 151–159.

Google Scholar

Chen, X., Hu, Q., Shi, Q., Quan, X., Wu, Z., and Li, F. (2020). Residential HVAC Aggregation Based on Risk-Averse Multi-Armed Bandit Learning for Secondary Frequency Regulation. J. Mod. Power Syst. Clean. Energy 8, 1160–1167. doi:10.35833/MPCE.2020.000573

CrossRef Full Text | Google Scholar

Chen, X., Wang, J., Xie, J., Xu, S., Yu, K., and Gan, L. (2018). Demand Response Potential Evaluation for Residential Air Conditioning Loads. IET Gener. Transm. & Distrib. 12, 4260–4268. doi:10.1049/iet-gtd.2018.5299

CrossRef Full Text | Google Scholar

Gong, X., Castillo-Guerra, E., Cardenas-Barrera, J. L., Cao, B., Saleh, S. A., and Chang, L. (2021). Robust Hierarchical Control Mechanism for Aggregated Thermostatically Controlled Loads. IEEE Trans. Smart Grid 12, 453–467. doi:10.1109/tsg.2020.3009989

CrossRef Full Text | Google Scholar

Hao, H., Sanandaji, B. M., Poolla, K., and Vincent, T. L. (2015). Aggregate Flexibility of Thermostatically Controlled Loads. IEEE Trans. Power Syst. 30, 189–198. doi:10.1109/TPWRS.2014.2328865

CrossRef Full Text | Google Scholar

Hu, Q., Ding, H., Chen, X., Chen, T., Ding, Y., and Li, Y. (2020). Analysis on Rotating Power Outage in California, USA in 2020 and its Enlightenment to Power Grid of China. Autom. Electr. Power Syst. 44, 11–18. doi:10.7500/AEPS20201016002

CrossRef Full Text | Google Scholar

IRENA (2022). Renewable Capacity Statistics 2022. Available at: https://www.irena.org/publications/2022/Apr/Renewable-Capacity-Statistics-2022 (Accessed May 15, 2022).

Google Scholar

Kou, X., Li, F., Dong, J., Olama, M., Starke, M., Chen, Y., et al. (2021). A Comprehensive Scheduling Framework Using SP-ADMM for Residential Demand Response with Weather and Consumer Uncertainties. IEEE Trans. Power Syst. 36, 3004–3016. doi:10.1109/TPWRS.2020.3029272

CrossRef Full Text | Google Scholar

Li, L., Chu, W., Langford, J., and Schapire, R. E. (2010). “A Contextual-Bandit Approach to Personalized News Article Recommendation,” in Proceedings of the 19th International Conference on World Wide Web (WWW 2010), Raleigh, NC, April 26–30, 2010, 661–670. doi:10.1145/1772690.1772758

CrossRef Full Text | Google Scholar

Li, Y., Hu, Q., and Li, N. (2020). A Reliability-Aware Multi-Armed Bandit Approach to Learn and Select Users in Demand Response. Automatica 119, 109015. doi:10.1016/j.automatica.2020.109015

CrossRef Full Text | Google Scholar

Li, Y., Yao, J., Yong, T., Ju, P., Yang, S., and Shi, X. (2017). Estimation Approach to Aggregated Power and Response Potential of Residential Thermostatically Controlled Loads. Proc. CSEE 37, 5519–5528. doi:10.13334/j.0258-8013.pcsee.161493

CrossRef Full Text | Google Scholar

Luo, J., Zhao, S., Feng, Y., Liu, Z., Dong, W., and Yanhong, Y. (2020). Optimal Operation of Integrated Electricity-Gas System Considering Uncertainty of Integrated Demand Response. Electr. POWER 53, 119–126.

Google Scholar

Mathieu, J. L., Vaya, M. G., and Andersson, G. (2013). “Uncertainty in the Flexibility of Aggregations of Demand Response Resources,” in IECON 2013 - 39th Annual Conference of the IEEE Industrial Electronics Society, Vienna, Austria, 10-13 November 2013, 8052–8057. doi:10.1109/IECON.2013.6700479

CrossRef Full Text | Google Scholar

Qi, N., Cheng, L., Xu, H., Wang, Z., and Zhou, X. (2020). Practical Demand Response Potential Evaluation of Air-Conditioning Loads for Aggregated Customers. Energy Rep. 6, 71–81. doi:10.1016/j.egyr.2020.12.019

CrossRef Full Text | Google Scholar

Saleh, S. A., Pijnenburg, P., and Castillo-Guerra, E. (2017). Load Aggregation from Generation-Follows-Load to Load-Follows-Generation: Residential Loads. IEEE Trans. Ind. Appl. 53, 833–842. doi:10.1109/TIA.2016.2626261

CrossRef Full Text | Google Scholar

Sun, H., Xu, T., Guo, Q., Li, Y., Lin, W., Yi, J., et al. (2019). Analysis on Blackout in Great Britain Power Grid on August 9th, 2019 and its Enlightenment to Power Grid in China. Proc. CSEE 39, 6183–6192. doi:10.13334/j.0258-8013.pcsee.191632

CrossRef Full Text | Google Scholar

Sun, L., Gao, C., Tan, J., and Cui, G. (2017). Load Aggregation Technology and its Applications. Autom. Electr. Power Syst. 41, 159–167. doi:10.7500/AEPS20161023001

CrossRef Full Text | Google Scholar

Sun, W., Liu, X., Xiang, W., and Li, H. (2020). User Preference Learning and Response Optimization Based on Bayesian Inference. Autom. Electr. Power Syst. 44, 92–100. doi:10.7500/AEPS20200310002

CrossRef Full Text | Google Scholar

Syrri, A. L. A., and Mancarella, P. (2016). Reliability and Risk Assessment of Post-contingency Demand Response in Smart Distribution Networks. Sustain. Energy, Grids Netw. 7, 1–12. doi:10.1016/j.segan.2016.04.002

CrossRef Full Text | Google Scholar

Zhou, X., Zhao, Q., Zhang, Y., and Sun, L. (2021). Integrated Energy Production Unit: An Innovative Concept and Design for Energy Transition toward Low-Carbon Development. CSEE J. Power Energy Syst. 7, 1133–1139. doi:10.17775/CSEEJPES.2021.05950

CrossRef Full Text | Google Scholar