An effective surrogate model assisted algorithm for multi-objective optimization: application to wind farm layout design

Due to the intricate and diverse nature of industrial systems, traditional optimization algorithms require a significant amount of time to search for the optimal solution throughout the entire design space, making them unsuitable for meeting practical industrial demands. To address this issue, we propose a novel approach that combines surrogate models with optimization algorithms. Firstly, we introduce the Sparse Gaussian Process regression (SGP) into the surrogate model, proposing the SGP surrogate-assisted optimization method. This approach effectively overcomes the computational expense caused by the large amount of data required in Gaussian Process model. Secondly, we use grid partitioning to divide the optimization problem into multiple regions, and utilize the multi-objective particle swarm optimization algorithm to optimize particles in each region. By combining the advantages of grid partitioning and particle swarm optimization, which overcome the limitations of traditional optimization algorithms in handling multi-objective problems. Lastly, the effectiveness and robustness of the proposed method are verified through three types of 12 test functions and a wind farm layout optimization case study. The results show that the combination of meshing and SGP surrogate enables more accurate identification of optimal solutions, thereby improving the accuracy and speed of the optimization results. Additionally, the method demonstrates its applicability to a variety of complex multi-objective optimization problems.

1 Introduction

With the continuous advancement of technology, optimization problems have been widely applied in various fields, including industrial manufacturing, energy management, finance, and transportation. The primary objective of research on optimization problems is to achieve the optimal performance of a specific system or process. Generally, optimization problems can be divided into static optimization problems and dynamic optimization problems (Mavrovouniotis et al., 2017). Static optimization problems usually involve determining certain input conditions to maximize or minimize specific goals. Dynamic optimization problems consider how to adjust control parameters or certain input conditions to achieve the best effect as time progresses. However, as the scale of industrial systems continues to expand, the complexity of controlled processes also continues to rise (Vafadar et al., 2021). Consequently, optimization problems have become more challenging and intricate. In some cases, complex systems are often represented using high-precision models. To solve these problems, complex systems are often described using high-precision models. However, this approach requires a substantial investment of time and manpower, leading to a lack of efficient methods for solving optimization problems. As a result, there is a growing emphasis on discovering faster and more effective optimization techniques to enhance both efficiency and accuracy.

The multi-objective particle swarm optimization (MOPSO) algorithm plays a significant role in addressing complex multi-objective optimization problems (Parsopoulos and Vrahatis, 2008). By optimizing the position of each particle, the algorithm searches for the optimal solution set and offers several advantages: Firstly, it efficiently identifies and outputs the Pareto optimal solution set, providing more decision-making options compared to other common multi-objective optimization algorithms. Secondly, the algorithm can simultaneously handle multiple-objective functions by adjusting weights to balance the priorities between objectives, a swiftly computing the optimal solution for multiple objectives. Lastly, the algorithm demonstrates strong performance in solving intricate optimization problems, including non-linearity and high dimensionality. MOPSO have proven successful in tackling real-world multi-objective problems, such as feature selection for medical diagnosis and other applications, optimization of stamping process parameters, optimization of indoor CO₂ and PM2.5 levels, ridesharing, concentration analysis, and optimization of manufacturing processes. However, when dealing with multi-objective problems, certain challenges arise, such as selecting the individual best solutions, identifying the global best solutions, and managing computational resources. For MOPSO, the first issue is addressed by randomly selecting one of the relatively superior particles in the space as the historical best. To solve the problem of global best solutions, MOPSO select the global best solution from the optimal set based on particle density, and preferably choose positions where particles are relatively rare in space as “elite particles” to lead other particles in searching for the optimal solution (Palmer, 2019). Furthermore, solving multi-variable optimization problems with MOPSO algorithms has increased the computational difficulty of the algorithm, leading to significant time and cost overhead. In recent years, optimization algorithms based on surrogate models have attracted increasing attention and are widely used due to their efficiency and applicability (Gu et al., 2021).

With the rapid development of computer testing and analysis methods, surrogate models have emerged as the primary approach for solving the black box problems, replacing time-consuming experiments. Surrogate model aided optimization algorithms can be divided into two categories: global surrogate model and local surrogate model (Zhou et al., 2006; Han and Zhang, 2012; Liu et al., 2016). The global surrogate model is employed to approximate the overall behavior of the problem and capture the general search direction, such as response surface methodology (RSM) (Wang Z. et al., 2022), Gaussian process (GP) (Shadab et al., 2022), radial basis function (RBF) model (Chen et al., 2022) and spline method (SM) (Grimstad et al., 2016). On the other hand, the local surrogate model is utilized to enhance the approximation accuracy within a confined search space, such as Kriging model (Jeong et al., 2005), locally weighted regression (Talgorn et al., 2018), and support vector machine regression (SVM) (Ciccazzo et al., 2015), etc. Compared to methods like artificial neural network (ANN) models, surrogate models based on the Bayesian framework have been widely adopted due to their high model fitting accuracy and the ability to provide approximate values of uncertainty, which proves highly effective in model management (Lystad et al., 2023). Among these, surrogate optimization based on the Kriging model and GP model are typical examples of Bayesian-based surrogate optimization methods (Liu J. et al., 2023). The Kriging surrogate model, initially used in geology, has evolved into an optimization design method applicable to various disciplines (Currin et al., 1991; Kudela and Matousek, 2022). Jones constructed the most widely used improvement expectation criterion in the optimization field by using the Kriging model to predict the function estimation and its mean square error (Jones et al., 1998). Additionally, Jones proposed a highly efficient global optimization algorithm called Efficient Global Optimization, which maximizes the expected improvement criterion. Subsequently, researchers further improved the model and introduced methods such as Blind Kriging (Joseph et al., 2008), Co-Kriging (Liu et al., 2022), and Gradient Kriging (Liu F. et al., 2023), thereby expanding the selection range of Kriging models in surrogate modeling. Currently, the Kriging surrogate model has been widely applied in fields such as uncertainty analysis, reliability assessment, and optimization (Wang C. et al., 2022; Ling et al., 2022; Zhao et al., 2022). However, these methods have significant requirements for the initial data, making them unsuitable for situations with a limited number of samples or incomplete information during the early stages of product or equipment development. In addition, the GP model is considered a surrogate model with low modeling complexity that balances accuracy and correlation. It is extensively applied in complex structures, particularly those with multiple failure modes, and situations where reliability calculations are time-consuming (Liu et al., 2013; Satria Palar et al., 2020). Many researchers have conducted in-depth research and exploration on the GP surrogate model, expanding its application fields and making it one of the most commonly used optimization methods. Su et al. (2017) proposed a dynamic GPR surrogate model based on Monte Carlo simulation for the reliability analysis of bridge engineering structures. Avendaño-Valencia et al. (2021) proposed a GP surrogate model method based on Bayesian hyperparameter calibration to predict short-term fatigue damage equivalent loads of wind turbines, achieving a prediction error of less than 4%. Golparvar et al. (2021) examined the impact of wind wave-related factors on the variability of offshore wind power using a GP surrogate model, and revealed the potential relationship between controlling offshore weather and power conversion. Furthermore, Preen et al. (2019) simulated tumor growth using a GP surrogate model, opening up broad prospects for dealing with complex, multiscale tumor models.

In summary, Kriging surrogate model and GP surrogate model are widely applied due to their excellent performance in high-precision fitting, optimization, and uncertainty analysis. They offer effective global optimization, interpretability, and flexibility, making them versatile choices in various domains. However, both Kriging surrogate model and GP surrogate model require complex computations to estimate functions and establish correlations. Dealing with large input spaces or extensive datasets significantly increases the computational cost. Furthermore, the fitting accuracy of the model is greatly affected when there is a small initial dataset. Additionally, different surrogate models and optimization algorithms have varying advantages and disadvantages, which necessitate the use of other evaluation methods to measure their performance and applicability. Therefore, it is essential to compare and analyze optimization algorithms based on surrogate models. In this paper, different optimization algorithms of surrogate models are compared in order to better understand the surrogate models. The specific contributions of this article are as follows.

1. This paper systematically summarizes the deficiencies of traditional proxy model optimization, particularly the adaptability issue between surrogate models and optimization problems. The paper combines the Sparse Gaussian Process (SGP) surrogate method with optimization algorithms for the first time. By employing the SGP as the surrogate model, it can estimate unknown function values a small amount of known data and provide accurate uncertainty information.

2. This paper introduces a novel surrogate model method called the Adaptive Grid-based Multi-Objective Particle Swarm Optimization algorithm (AG-MOPSO), which is based on SGP surrogate modeling. The use of an adaptive grid divides the optimization problem into multiple regions and utilizes the MOPSO algorithm to optimize particles in each region, addressing the inefficiency problems of traditional algorithms in complex scenarios.

3. To validate the proposed method, three sets of test functions and a wind farm layout optimization case study are employed for testing. Comparative analysis with different optimization algorithm models demonstrates the superior performance of the proposed method in terms of accuracy and optimization speed, particularly for multi-objective optimization problems and diverse, complex industrial systems, when compared to alternative optimization algorithms.

The remaining sections of this paper are organized as follows: Section 2 introduces multi-objective optimization problems and compares the advantages and disadvantages of commonly used surrogate model-assisted optimizations. Section 3 provides a detailed description of the implementation steps of the adaptive grid multi-objective particle swarm optimization algorithm based on the SGP surrogate (AG-MOPSO-GPS) model. Section 4 evaluates the proposed surrogate model-assisted optimization algorithm. Section 5, the algorithm proposed in this paper is applied to actual engineering case. Finally, Section 6 summarizes the contributions of this paper.

2 Optimization algorithm and surrogate model

2.1 Multi-objective optimization problem

Multi-objective optimization is a prevalent challenge in modern complex industrial systems (Cui et al., 2017). Achieving simultaneous optimization for each objective, given their conflicting nature, is often unattainable. In general, multi-objective optimization problems are composed of several objective functions, some related equations and inequality constraints. Mathematically, they can be described as follows:

$\min F\left(X\right)={\left[f_1\left(x\right),f_2\left(x\right),\dots ,f_m\left(x\right)\right]}^T(1)$

$\begin{array}{l}s.t.g_i\left(x\right)\le 0,i=1,2,3,\dots p\\ h_j\left(x\right)\le 0,i=1,2,3,\dots p\end{array}(2)$

Where, $f_i\left(x\right),\left\{i=\mathrm{1,2,3},\dots ,m\right\}$ represents the objective function; $g_i\left(x\right)$ and $h_j\left(x\right)$ are the constrained function; $x=\left\{x_1,x_2{,\dots ,x}_n\right\}$ is an n-dimensional design variable. $X=\left\{x|xϵR^n,g_i\left(x\right)\ge 0,h_j\left(x\right)=0,i=\mathrm{1,2},\dots ,p,j=\mathrm{1,2},\dots ,q\right\}$ represents the feasible range of the objective function.

In multi-objective optimization, a significant challenge arises due to the mutual constraints between various objectives. This constraint often leads to a trade-off where the improvement in one target’s performance comes at the cost of sacrificing the performance of other targets. As a result, finding a single optimal solution becomes impractical in most cases. To address this issue, multi-objective optimization problems typically use a set of non-dominated solutions to represent all optimal solutions. This set of solutions is called the Pareto optimal solution set. Finding the Pareto optimal solution set is a very important task in multi-objective optimization problems. One way to solve this task is to find as many Pareto optimal solutions as possible for the optimization problem. This means finding a set of solutions that are as close to the Pareto optimal domain as possible, and finding a set of solutions that are as different as possible. So, this problem can usually be solved through multiple relatively balanced solutions, rather than a single optimal solution. The Pareto optimal solution set is a typical solution in multi-objective optimization problems. In fact, the Pareto optimal solution set is a model that can represent future needs and user decisions, as most users pursue maximum benefits without losing any purpose. Meanwhile, solving Pareto optimality problems can help people better understand and apply multi-objective optimization problems. In summary, the Pareto optimal solution set in multi-objective optimization problems is an important form of solving the optimal solution. By finding a set of Pareto optimal solutions, people can obtain multiple equilibrium solutions to solve the problem. The Pareto optimal solution set represents a feasible solution that can help people better apply multi-objective optimization problems to practical applications. Currently, the majority of evolutionary algorithms used to address multi-objective optimization problems often require tens of thousands of fitness evaluations, incurring significant expenses in terms of time and resources. Despite numerous research efforts to create novel optimization algorithms for solving multi-objective optimization problems, the trade-off between solution speed and the number of function evaluations is rarely considered (Akinola et al., 2022). As a solution to mitigate costly function evaluations, data-driven surrogate-assisted evolutionary algorithms have emerged. This method involves constructing a surrogate model to approximate the original expensive objective function and then employing the surrogate model to evaluate a subset of candidate solutions, thereby reducing computational costs (He et al., 2023). It has become a prevalent approach to alleviate the burden of costly function evaluations in multi-objective optimization problems.

2.2 Surrogate model

The surrogate model method offers a solution to the optimization design and design analysis challenges posed by complex source models. It involves constructing substitute models for these source models. Despite requiring the expensive valuation of source models during the sampling process, this method effectively reduces the number of simulations in optimization, thereby improving its efficiency. Common surrogate models include the Kriging model, RBF model, ANN model, and others. In practice applications, these surrogate models are often combined with various optimization algorithms, such as the genetic algorithm, particle swarm algorithm, simulated annealing, among others (Nguyen et al., 2014). A comparison of several classic surrogate models is presented in Table 1. The Kriging surrogate model exhibits high fitting accuracy and can provide predicted and error values. However, constructing the Kriging model can be time-consuming, especially when dealing with numerous sample points. In contrast, the RBF surrogate model is relatively easy to construct, flexible, and adaptable to nonlinear problems. The difficulty of constructing the SVM surrogate model is determined by the number of support vectors requires. Although it can reduce the problem of “curse of dimensionality” to some extent, it cannot handle large sample sets. Meanwhile, the ANN surrogate model is widely used due to its effective learning mechanism, accuracy, and robustness, although constructing a high-precision ANN surrogate model takes a lot of time (Liu et al., 2021).

TABLE 1

TABLE 1. Advantages and disadvantages of different surrogate model.

Despite the strengths exhibited by different surrogate models, choosing the most appropriate model remains a challenging problem in practice. Multiple criteria need to be considered simultaneously, such as model accuracy, construction time, and computational efficiency. Moreover, in optimization scenarios involving multiple objectives, the Pareto front (PF) of the surrogate model must be sufficiently sampled to accurately capture the entire set of solutions. In conclusion, the effective construction and updating of surrogate models are essential for the successful implementation of multi-objective evolutionary algorithms, which are central to many optimization problems. Advancements in this field necessitate a better understanding of the strengths and weaknesses of different surrogate models, along with the development of more efficient algorithms capable of accommodating the diverse requirements of various optimization scenarios.

3 Surrogate assisted optimization framework

Utilizing surrogate models in optimization is a vital technique that can significantly save computational resources, especially for multi-objective optimization problems, where evaluating each objective function can be time-consuming. The main idea behind surrogate model-assisted optimization is to use a surrogate model to approximate the true objective function, thus speeding up the optimization process and improving its efficiency. As a result, it has been extensively studied. In this section, we address the adaptability deficiencies of existing surrogate models in optimization problems and propose the AG-MOPSO-GPS optimization algorithm. We will provide a detailed introduction to the SGP surrogate model, the AG-MOPSO Optimization algorithm, and the framework of surrogate model-assisted optimization.

3.1 SGP surrogate models

As a type of machine learning algorithm, the GP model integrates the advantages of traditional statistical theory and Bayesian theorem (Zhang et al., 2019). Compared to linear regression, the advantages of the GP model lie in its ability to solve complex problems with small samples, high dimensions, and non-linearity (Giovanis and Shields, 2020). Compared to black box algorithms such as artificial neural networks, its advantages include ease of implementation, adaptive hyperparameters, and interpretability (Ghahramani, 2015). The GP model allows prior distributions to be placed over the entire function for inference, rather than just learning model parameters. Consider a dataset $\mathcal{D}={\left\{\left(x^i,f\left(x^i\right)\right)\right\}}_{i=1}^{N_I}$ of N samples. The GP model is defined as a set of random variables $\left(x_1\right),f\left(x_2\right),\dots ,f\left(x_i\right)$ , with any finite-dimensional distribution being a joint normal distribution.

$f\left(x\right)\sim GP\left(m\left(x\right),K\left(x,x^{\prime }\right)\right)(3)$

Where, $m\left(x\right)$ represents mean function, $K\left(x,x^{\prime }\right)$ is kernel function.

The goal of the GP model is to model data as the output of certain functions. However, in normal cases, the measured data contains noise. To model this noise, a Gaussian noise model is commonly used:

$y=f\left(x\right)+\epsilon \epsilon \sim N\left(0,{\sigma }_n^2\right)(4)$

The prior distribution of the training samples follows the distribution given below:

$y\sim N\left(m\left(\mathit{x}\right),K+{\sigma }_n^2\mathit{I}\right)$

Where, $I$ represents the $n$-order identity matrix.

In order to make predictions, it is important to evaluate the joint prior distribution between the training and testing data:

$\left[\begin{array}{c}\mathit{y}\\ y*\end{array}\right]\sim N\left(\left[\begin{array}{c}m\left(x\right)\\ m\left(x*\right)\end{array}\right],\left[\begin{array}{ll}{K}_{xx}+{\sigma }_n^2\mathit{I}& K_{xx*}\\ K_{x*x}& K_{x*x}+{\sigma }_n^2\mathit{I}\end{array}\right]\right)(5)$

Where, $x^{*}$ represents the new input dataset. $y^{*}$ represents the predicted new potential unknown output. $K_{xx}=K\left(x,x\right)$, $K_{x{x}^{*}}=K\left(x,x^{*}\right)$, $K_{x^{*}x}=K\left(x^{*},x\right)$, $K_{x^{*}{x}^{*}}=K\left(x^{*},x^{*}\right)$.

By applying the properties of multivariate Gaussian distribution, it is known that each conditional distribution is also a Gaussian distribution. Utilizing this standard result, we can express the posterior distribution of $y^{*}$ as:

$\begin{array}{l}p\left(y*\left|\mathit{x}*,\mathit{x},\mathit{y}\right.\right)\sim N\left(\mu *,\sum *\right)\\ \mu *=m\left(x*\right)+K_{x*x}{\left(K_{xx}+{\sigma }_n^2\mathit{I}\right)}^{-1}\left(\mathit{y}-m\left(\mathit{x}\right)\right)\\ \sum *=K_{x*x}-K_{xx*}{\left(K_{xx}+{\sigma }_n^2\mathit{I}\right)}^{-1}{K}_{x*x}+{\sigma }_n^2\end{array}(6)$

The kernel function is the most important component of the GP model, as it is used during the training phase to map the relationship between the input and output, and also for predicting new points through interpolation. The radial basis function kernel is commonly used, which has a small number of hyperparameters. These hyperparameters control the behavior of the kernel function, and in order to utilize the GP model to analyze the non-linear relationship between different features, these hyperparameters must be determined. Typically, the second type of maximum likelihood method is used to determine the hyperparameters of the kernel function. The objective of this method is to maximize the marginal likelihood (also known as model evidence) of the model. This optimization finds the minimum complex model using the Bayesian Occam’s razor, given the training set D. For convenience and numerical stability, this optimization is usually performed by minimizing the negative logarithm of the marginal likelihood. Therefore, the estimated value of the hyperparameters $\widehat{\theta }$ is obtained through the following optimization.

$\begin{array}{l}\widehat{\theta }=\underset{\theta }{\arg \min }\left\{-\log p\left(\mathit{y}\left|\mathit{x},\theta \right.\right)\right\}\\ -\log p\left(\mathit{y}\left|\mathit{x},\theta \right.\right)=-\log N\left(m\left(x\right),K_{xx}+{\sigma }_n^2\mathit{I}\right)\\ =\frac{N}{2}\log \left(2\pi \right)+\frac{1}{2}\log \left|K_{xx}+{\sigma }_n^2\mathit{I}\right|\\ =\frac{1}{2}\left[{\left(\mathit{y}-m\left(\mathit{x}\right)\right)}^T{\left(K_{xx}+{\sigma }_n^2\mathit{I}\right)}^{-1}\left(\mathit{y}-m\left(\mathit{x}\right)\right)\right]\end{array}(7)$

However, in the GP model, to determine hyperparameters or make predictions, it is necessary to evaluate the inverse of the covariance matrix with noise ${\left(k_{\mathit{x}\mathit{x}}+{\sigma }_n^2\mathit{I}\right)}^{-1}$.This computation step carries a time complexity and memory storage of $O\left(N^3\right)$, which means that using the GP model to establish non-linear regression models for big datasets is not feasible. To address this issue, inducing data sets are constructed by randomly selecting $M\left(M\ll N\right)$ data points from the training set to learn the hyperparameters, and such models are known as SGPR. Introducing these inducing points reduces the complexity of the GP model from $O\left(N^3\right)$ to $O\left(M{N}^2\right)$. Utilizing the inducing points $\left\{Z,\mathit{u}\right\}$ (where $Z$ represents the locations of inducing points and $\mathit{u}$ denotes the latent function values at these points) forms a variational approximation to the full posterior of the model. Then, the expected distribution value is obtained by minimizing the Kulback-Leibler divergence between the variational distribution and the posterior distribution:

$q\left(y*\left|\mathit{x}*,\mathit{x},\mathit{y},\mathit{u}\right.\right)=N\left(\mu *,\sum *\right)(8)$

$\mu *=Q_{x*x}{\left(Q_{xx}+{\sigma }_n^2\mathit{I}\right)}^{-1}y(9)$

$\sum *=K_{x*x*}-Q_{x*x}{\left(Q_{xx}+{\sigma }_n^2\mathit{I}\right)}^{-1}{Q}_{xx*}(10)$

Where, $Q_{\mathit{x}\mathit{x}}$ represents the approximate Covariance matrix, and the expression is as follows:

$Q_{xx}=K_{xu}{K}_{uu}^{-1}{K}_{ux}(11)$

3.2 Adaptive grid multi-objective particle swarm optimization algorithm (AG-MOPSO)

Particle swarm optimization (PSO) is a heuristic algorithm inspired by the foraging behavior of birds. Due to its high convergence rate and simple search principle, PSO has been widely used and developed in the field of multi-objective optimization (Coello et al., 2004). However, traditional MOPSO have some shortcomings, such as insufficient consideration of the selection of global optimal particles, the lack of pruning of the non-dominated set, and the imbalance between global and local search capabilities. Therefore, in this paper, an AG-MOPSO is selected to optimize multi-objective problems (Yang et al., 2008).

The AG-MOPSO algorithm adopts a dual-population approach, where one population is a standard PSO group, while the other population is used to store the non-dominated solutions discovered by the group after each iteration, known as the Archive set. The main part of AG-MOPSO is similar to the general PSO algorithm. It initializes a swarm of particles in an n-dimensional search space, where the movement of each particle depends on its position and velocity vector. The velocity and position vectors can be represented by the following equations:

$\begin{array}{l}\overrightarrow{X}=\left[{\overrightarrow{x}}_1,{\overrightarrow{x}}_2,{\overrightarrow{x}}_3,\cdots ,{\overrightarrow{x}}_n\right]\\ \overrightarrow{V}=\left[{\overrightarrow{v}}_1,{\overrightarrow{v}}_2,{\overrightarrow{v}}_3,\cdots ,{\overrightarrow{v}}_n\right]\end{array}(12)$

In the search space, particles adjust their position and speed vectors according to global searching experience $g_{best}$ and individual local searching experience $P_{best}$ with certain rules. In this way, global searching ability and local searching ability can be included simultaneously. The update formula is:

$\begin{array}{l}{v}_i^n\left(k+1\right)=\underset{previousvelocity}{\underbrace{\omega \left(k\right)·v_i^{n}k}}+\underset{personalcomponent}{\underbrace{{\mathrm{\Lambda }}_1{\varpi }_1\left(p_{best,i}^n-x_i^n\left(k\right)\right)}}\\ +\underset{globalcomponent}{\underbrace{{\mathrm{\Lambda }}_2{\varpi }_2\left(g_{best,i}^n-x_i^n\left(k\right)\right)}}\\ x_i^n\left(k+1\right)=x_i^n\left(k\right)+v_i^n\left(k+1\right)\end{array}(13)$

Where, $v_i^n\left(k+1\right)$ and $v_i^n\left(k\right)$ represent the ith particle in the nth dimension from the velocity vector of this iteration versus the last iteration, respectively. $x_i^n\left(k+1\right)$ and $x_i^n\left(k\right)$ represent the ith particle in the nth dimension the position vector from this iteration to the last iteration, respectively. $g_{best,i}^n$ and $p_{best,i}^n$ represent the position vectors of the global and historical best particles for the first particle i n the nth dimension, respectively. ${\mathrm{\Lambda }}_1$ and ${\mathrm{\Lambda }}_2$ are local and global learning factors, respectively. ${\varpi }_1$ and ${\varpi }_2$ are random numbers between [0,1]. $\omega \left(k\right)$ is the inertial factor given by the following formula:

$\omega \left(k\right)={\omega }_{\max }-\frac{k}{k_{\max }}·\left({\omega }_{\max }-{\omega }_{\min }\right)(14)$

Where, $k_{\mathit{max}}$ represents the total number of iterations. The variable inertia weight can guarantee the global search ability in the early search period and the local search ability in the late search period.

AG-MOPSO has three innovations: adaptive grid (AG) algorithm, global optimal particle selection, and Archive set truncation. Taking the two-dimensional target space optimization problem as an example, AG first calculates the search range of the target space after $K$ iterations [min $Y_1^k$, max $Y_1^k$] and [min $Y_2^k$, max $Y_2^k$]. Then the grid model is calculated according to the following formula:

$\begin{array}{l}\Delta Y_1^k=\left(\max Y_1^k-\min Y_1^k\right)/M\\ \Delta Y_2^k=\left(\max Y_2^k-\min Y_2^k\right)/M\end{array}(15)$

Where, $M$ represents dividing the target space into $M\times M$'s Grid, $Y_1$ and $Y_2$ are the values of the fitness function. In the early stage of evolution, $M$ selects a smaller value to save computational costs. As the algorithm continues to iterate, $M$ will adaptively increase to improve the resolution in the later stage of evolution, obtaining a more realistic optimal result. After obtaining the modulus of the grid, traverse the population to calculate the grid numbers of each particle. For particle $i$, its grid number is given by the following equation:

$\left(x_1^i,x_2^i\right)=\left(\mathit{Int}\left(\frac{Y_1^i-\min Y_1^k}{\Delta Y_1^k}\right)+1,\mathit{Int}\left(\frac{Y_2^i-\min Y_2^k}{\Delta Y_2^k}\right)+1\right)(16)$

Where $\mathit{Int}$ means integer. After calculating the grid number of each particle, the grid density information of each grid can be obtained. After calculating the grid number of each particle, the grid density information of each grid can be obtained. The selection of the globally optimal particles is based on the density information. Specifically, the lower the density of the grid in which the particles in the Archive are located, the greater the probability of being selected as $g_{best}$. The more particles in Archive concentration are better than the number of particles in the group, the greater the search potential of the particles. Based on this idea, the $g_{best}$ corresponding to each particle in the group is determined using the following formula:

$g_{best,j}^k=\left\{i\left|\min \left\{F\left(i\right),i\in S_j^k\right\}\right.\right\}(17)$

Where, $F\left(i\right)$ indicates the number of particles in the grid in which ith particles reside. $S_j^k$ indicates that the Archive set is better than the set of group particles $j$, which is determined by the following formula.

$S_j^k=\left\{i\left|\forall i\succ j\right.\right\}(18)$

Where, $\succ$ indicates the Pareto dominance. When there are more than one particle that meets the criteria, select the Archive set particle with the greatest potential as $g_{best}$:

$g_{best,j}^k=\left\{g_{best,j}^k\left|\max \left\{i\left|i\prec j,i\in P_k\right.\right\}\right.\right\}(19)$

Where, $P_k$ represents the population of particles in the kth iteration. This formula represents the particle that dominates the maximum number of particles in the population of all the particles satisfying formula (17). When there are particles with the same potential, choose one randomly. The Archive Set truncation technique used by AG-MOPSO is based on the density information of the mesh. When the density of the mesh exceeds the threshold, the smallest potential particles are deleted according to (19), which ensures that Archive Set always has excellent search potential throughout the iteration process.

In the AG-MOPSO algorithm, the grid quantity is adaptively adjusted based on the particle distribution and changes during the optimization process. The variation of grid quantity depends on the current particle distribution and grid density, aiming to better adapt to the characteristics and changes of the problem during the optimization process. In the initial stage of the algorithm, the grid quantity may be relatively small because there is not enough information to accurately partition the optimization space. As the optimization progresses and particles move and search, the optimization space is gradually divided into more grids. By adaptively adjusting the grid quantity, the AG-MOPSO algorithm can efficiently search for the global optimal solution or the Pareto optimal solution set in different optimization stages and features of the optimization space. This flexible variation of grid quantity allows the algorithm to better adapt to complex multi-objective optimization problems and maintain good performance throughout the optimization process.

3.3 Optimization framework and process description

When dealing with expensive multi-objective optimization problems, it is important to consider the significant uncertainty in the approximation of individuals by surrogate models (Zheng et al., 2022). This means that the areas surrounding these solutions may not be effectively explored, resulting in a high probability of finding suboptimal solutions. Furthermore, evaluating these solutions and using them to update the surrogate model could be the most effective way to enhance the accuracy of the model. Bayesian theory is an effective tool for considering the uncertainty in optimization problems. It can predict the objective function value and improve the variance of this predicted value. The optimization framework assisted by surrogate models takes a similar approach to Bayesian optimization, integrating the construction and optimization processes of surrogate models. In this study, we used an optimization framework assisted by a GP surrogate model (Sun et al., 2017), whose pseudocode is shown in Algorithm 1. The framework combines outer and inner optimization, where the outer layer updates the training set from the solution output by the inner layer, iteratively updating the surrogate model, while the inner layer iteratively optimizes to obtain the function and then the sampling points to output to the outer layer.

Algorithm 1.GP surrogate models

An optimization framework assisted by GP surrogate models

Input: Relevant parameters: $\mathcal{D}$, iteration number ${Iter}_{\mathit{max}}$, Set the initial number of samples n

Output: Optimal sample set { $f\left(x\right)$}, current optimal solution $\underset{x\in \mathcal{D}}{\min }f\left(x\right)$

Initialization:

  Using experimental design methods to initialize a solution set $\mathcal{X}\leftarrow {\left\{x^i\right\}}_{i=1}^{N_I}$ from $\mathrm{\Omega }$ and evaluate their objective function values $\mathcal{Y}\leftarrow {\left\{f\left(x^i\right)\right\}}_{i=1}^{N_I}$. Set initial training set $\mathcal{D}\leftarrow {\left\{\left(x^i,f\left(x^i\right)\right)\right\}}_{i=1}^{N_I}$. Set initial $Iter\leftarrow 0$

While $Iter<{Iter}_{\mathit{max}}$ do

  Build a GP model based on $\mathcal{D}$.

  Use optimization algorithms to optimize the acquisition function and obtain candidate solutions $x^{*}$.

  Evaluate the objective function value of $x^{*}$ and set $\mathcal{D}\leftarrow {\mathcal{D}\cup \left\{\left(x^i,f\left(x^i\right)\right)\right\}}_{i=1}^{N_I}$.

end

Return $\underset{x\in \mathcal{D}}{\min }f\left(x\right)$

Considering the limitations of traditional optimization algorithms in dealing with industrial system optimization problems, this paper proposes a new optimization algorithm called AG-MOPSO-GPS model, as shown in Figure 1. In contrast to the conventional GP surrogate model framework, we introduce the SGP model and propose the SGP-assisted optimization method. The SGP surrogate model effectively addresses the computational overhead caused by the large data requirements of traditional gaussian process model. Moreover, we divide the optimization problem into multiple regions through grid partitioning and utilize the MOPSO algorithm to optimize particles in each region. By combining the advantages of grid partitioning and particle swarm optimization, we overcome the limitations of traditional optimization algorithms in dealing with multi-objective problems. At each iteration, the surrogate model is used to predict the next optimal solution and integrated into the subsequent iteration. The advantage of this algorithm lies in its ability to reduce computation costs by avoiding frequent evaluations of the true objective function. The AG-MOPSO-GPS algorithm is characterized by a broad search range and fast search speed due to the use of MOPSO. Additionally, it can effectively handle high-dimensional non-linear problems, and is supported by mathematical theory. The specific implementation of the algorithm includes key steps, such as selecting specific hyperparameters, sampling strategies, and stopping conditions, to improve accuracy and efficiency.

FIGURE 1

FIGURE 1. The AG-MOPSO-GPS model optimization framework.

The steps of the AG-MOPSO-GPS model are as follows:

Step 1: Initialize population: randomly generate a certain number of particles and assign initial position and velocity values.

Step 2: Particle fitness evaluation: substitute the particles in the population into the objective function and calculate the fitness value of each particle.

Step 3: Update non-dominated solution set: determine whether the solution of the objective function is a non-dominated solution based on the dominance relation and update the non-dominated solution set.

Step 4: Grid division: according to the distribution of the non-dominated solution set, divide the grid and optimize each grid as a region.

Step 5: Update particle velocity and position based on non-dominated solution set: use the best particle position in each grid as a reference point, calculate the velocity and position of each particle based on the reference point, and update the position and velocity of the particles. At the same time, use the SGP surrogate method to optimize the reference point and predict the reference point for the next iteration.

Step 6: Record the best historical solution: record the best particle position and corresponding fitness value in each grid.

Step 7: Determine stopping criteria: determine whether the defined stopping criteria such as maximum number of iterations are met.

Step 8: Output results: output the final optimization results, including the position of particles in the population and their corresponding fitness values.

4 Results and discussions

In this section, we make a series of comparisons of surrogate model assisted multi-objective optimization algorithms to explore the advantages of surrogate model applied to complex optimization problems. Firstly, we compared the evaluation accuracy of surrogate models on different functions. Secondly, we applied the surrogate models to multi-objective optimization algorithms and compared the performance of surrogate model-assisted optimization algorithms.

4.1 Testing of multi-objective optimization algorithm based on surrogate model

The Zitzler Deb Thiele (ZDT) test function set is a classic set of test functions for assessing the performance of multi-objective optimization algorithms (Lim et al., 2015). Developed by Zitzler and his colleagues at the University of Barcelona in 2000, the ZDT function set features independent objective functions with varying feasible range widths and non-linear characteristics. These properties make it suitable for evaluating the performance of PF search algorithms. The ZDT function set includes several functions, with ZDT1 and ZDT2 being the most widely used ones.

The problem definition of ZDT1 is as follows:

$\begin{array}{l}{f}_1\left(x\right)=x_1\\ f_2\left(x\right)=g\left(x\right)\left[1-\sqrt{\frac{f_1\left(x\right)}{g\left(x\right)}}\right]\end{array}(20)$

Where, $g\left(x\right)=1+\left(9{\sum }_{i=2}^n{x}_i\right)/\left(n-1\right)$; $x={\left(x_1,\dots ,x_n\right)}^T\in {\left[\mathrm{0,1}\right]}^n$, which PF is convex.

The problem definition of ZDT2 is as follows:

$\begin{array}{l}{f}_1\left(x\right)=x_1\\ f_2\left(x\right)=g\left(x\right)\left[1-{\left(\frac{f_1\left(x\right)}{g\left(x\right)}\right)}^2\right]\end{array}(21)$

Where, ZDT2 takes values of $g\left(x\right)$ and $x$ that both range and dimensionally are the same as ZDT1.

The problem definition of ZDT3 is as follows:

$\begin{array}{l}{f}_1\left(x\right)=x_1\\ f_2\left(x\right)=g\left(x\right)\left[1-\sqrt{\frac{f_1\left(x\right)}{g\left(x\right)}}-\frac{f_1\left(x\right)}{g\left(x\right)}\sin \left(10\pi x_1\right)\right]\end{array}(22)$

Where, the PF of ZDT3 is discontinuous and has different scales on both targets.

DTLZ is another classic test function set used for testing multi-objective optimization algorithms (Li et al., 2015). Developed by Kalyanmoy Deb and others in 2002, this function set consists of a range of multi-objective test functions. Its features include high dimensions and interaction design in the objective functions, making it suitable for assessing the high-dimensional PF search capability.

The problem definition of DTLZ1 is as follows:

$\begin{array}{l}{f}_1\left(x\right)=\frac{1}{2}{\displaystyle \prod _{i=1}^{m-j}}{x}_i\left(1+g\left(x_{m:n}\right)\right)\\ f_{j=2:m-1}\left(x\right)=\frac{1}{2}\left({\displaystyle \prod _{i=1}^{m-j}}{x}_i\right)\left(1-x_{m-j+1}\right)\left(1+g\left(x_{m:n}\right)\right)\\ f_m\left(x\right)=\frac{1}{2}\left(1-x_1\right)\left(1+g\left(x_{m:n}\right)\right)\end{array}(23)$

Where, $g\left(x_{m:n}\right)=100\left[k+{\sum }_{i=n}^m\left({\left(x_i-0.5\right)}^2-\cos \left(20\pi \left(x_i-0.5\right)\right)\right)\right]$, $x={\left(x_1,\dots ,x_n\right)}^T\in {\left[\mathrm{0,1}\right]}^n$, DTLZ1 has many local optimal solutions, and the PF of DTLZ1 satisfies $\left({\sum }_{i=1}^m{f}_i\left(x\right)\right)=0.5$.

The problem definition of DTLZ2 is as follows:

$\begin{array}{l}{f}_1\left(x\right)=\left(1+g\left(x_{m:n}\right)\right){\displaystyle \prod _{i=1}^{m-1}}\cos \left(\frac{x_i\pi }{2}\right)\\ f_{j=2:m-1}\left(x\right)=\left(1+g\left(x_{m:n}\right)\right){\displaystyle \prod _{i=1}^{m-1}}\cos \left(\frac{x_i\pi }{2}\right)\sin \left(\frac{x_{m-j+1}\pi }{2}\right)\\ f_m\left(x\right)=\left(1+g\left(x_{m:n}\right)\right)\sin \left(\frac{x_1\pi }{2}\right)\end{array}(24)$

Where, $g\left(x\right)={\sum }_{i=n}^m\left({\left(x_i-0.5\right)}^2\right.$, The PF of DTLZ2 satisfies $\left({\sum }_{i=1}^m{f_i\left(x\right)}^2\right)=1$.

DTLZ7 problem has $2^{m-1}$ disconnected optimal Pareto regions, used to test the algorithm’s ability to maintain subpopulations within different Pareto regions. The problem definition of DTLZ7 is as follows:

$\begin{array}{l}{f}_{j=1:m-1}\left(x\right)=x_j\\ f_m\left(x\right)=\left(1+g\left(x_{m:n}\right)\right)\left(m-{\displaystyle \sum _{i=1}^{m-1}}\left[\frac{1}{1+g\left(x_{m:n}\right)}\left(1+\sin \left(3\pi f_i\left(x\right)\right)\right)\right]\right)\end{array}(25)$

Where, $g\left(x_{m:n}\right)=1+9{\sum }_{i-m}^n{x}_i/k,x={\left(x_1,\dots ,x_n\right)}^T\in {\left[\mathrm{0,1}\right]}^n$.

The WFG test functions encompass a total of 9 test problems, with each function exhibiting distinct characteristics and complexities to cover various types of multi-objective optimization problems (Li et al., 2018). In this paper, we have selected three test functions: WFG1, known for its plateau-like preference characteristics; WFG2, which demonstrates multi-modal non-continuity properties; and WFG3, a deceptive problem.

In general, ZDT test functions are suitable for evaluating relatively simple multi-objective optimization algorithms. Compared to ZDT, DTLZ test functions are more suitable for exploring algorithm performance under different problem characteristics, while WFG test functions offer more diversity and complexity, making them closer to real-world multi-objective optimization problems.

4.2 Evaluation criteria

The purpose of multi-objective optimization algorithm is to quickly search and approximate the true PF and distribute uniformly on it. Therefore, evaluating the performance of a multi-objective optimization algorithm requires consideration of its convergence, distribution, coverage, and search speed, among others. This paper tested the performance of the algorithm, mainly using two comprehensive indicators: inverted generational distance (IGD) and hypervolume (HV).

In multi-objective optimization, the IGD is an important indicator to evaluate the convergence, distribution uniformity, and generality of solutions. It is the inverse mapping of the generational distance (GD) and calculates the average distance from each individual in the true Pareto optimal solution set to the non-dominated solution set obtained by the algorithm. The smaller the IGD value of the solution set, the better the performance. In addition to reflecting the convergence of solution sets, IGD can also reflect the distribution uniformity and generality of solution sets. A smaller IGD value indicates better diversity and convergence.

$IGD\left(F*,F\right)=\frac{{\sum }_{x\in F*}\min dis\left(x,F\right)}{\left|F*\right|}(26)$

Where, $F$ is the non-dominated solution set obtained by the algorithm, $F^{*}$ represents the true Pareto non-dominated solution set, $d\left(.\right)$ represents the Euclidean distance, $\left|F^{*}\right|$ represents the number of solutions in the true PF.

The HV indicator is one of the metrics used to evaluate the comprehensive performance of multi-objective optimization algorithms. It represents the volume of the region in the objective space between the non-dominated solution set obtained by the algorithm and a reference point. HV can be used to measure the degree of closeness between the solution set and the optimal solution set and partially reflects the distribution of solutions in the objective space. A larger HV value indicates better comprehensive performance of the algorithm.

$HV\left(P\right)=VOL\left(\underset{x\in P}{\cup }\left[x_1,z_1^r\right]\times \dots \times \left[x_m,z_m^r\right]\right)(27)$

Where, $VOL$ represents the Lebesgue measure, $m$ represents the number of non-dominated solutions, $x_m$ represents the $mth$ objective component of the solution set $x$ in $P$, $z_m^r$ represents the $rth$ objective component of the reference point in the objective space.

4.3 Validation analysis of AG-MOPSO-GPS

Before validating the surrogate-assisted optimization algorithm, it is necessary to set the parameters of the optimization algorithm and the test functions. For the ZDT series test functions, the decision variables of ZDT1 and ZDT2 are set to 30. For the DTLZ series test functions, DTLZ1 has 7 decision variables and 3 dimensions, while DTLZ2 has 12 decision variables and 4 dimensions. For the WFG test function, the effectiveness of the method proposed in this article is verified by testing under three and four objectives. And for the three objectives problem, set 100 reference vectors, and for the four objectives problem, set 120 reference vectors with a vector dimension of 11 dimensions. The population size of the AG-MOPSO algorithm is set to 100, the number of partitions of each dimension in the adaptive grid-based method is set to 30, the inertia weight w is set to 0.4, the learning factor is set to 2.0, and the mutation parameter is set to 0.5. All the simulation experiments in this section were performed on a computer with 2.90 GHz AMD Ryzen 74800H with Radeon Graphics and 16 GB of memory using MATLAB_R2021b. The AG-MOPSO-GPS algorithm was run 30 times on uniformly random test functions to reduce accidental results, and the IGD measure and HV measure were calculated for each run. The data in Tables 1, 2 show the IGD and HV measures of the proposed algorithm and four other multi-objective algorithms, where “Mean” represents the average value of IGD and HV obtained by running the algorithm 30 times, and “Std” represents the standard deviation.

TABLE 2

TABLE 2. Comparative analysis of IGB measures for test functions.

To better validate the superiority of the proposed algorithm in this paper, we selected two basic optimization algorithms, AG-MOPSO and NSGA-II, as well as two agent-based optimization algorithms, AG-MOPSO-ANNS and AG-MOPSO-SVMS, for comparative research. Tables 2, 3 present the statistical results of these five optimization algorithms on three sets of test functions concerning IGD and HV measures. The best average results are highlighted in bold. As revealed in Table 3, the proposed algorithm exhibits the best IGD results among different test functions, except for the ZDT3 test function. Particularly, in the ZDT1 and ZDT2 test functions, the IGD has a significant advantage of one order of magnitude. Although the IGD measure is not optimal for ZDT3, the proposed method still outperforms traditional optimization algorithms and SVM agent models, indicating that the AG-MOPSO-GPS algorithm can obtain an optimal solution set closer to the true PF, further validating its effectiveness. Furthermore, the proposed algorithm shows significant improvement in the DTLZ type of test functions compared to other optimization algorithms, demonstrating its generalization and robustness for three-objective high-complexity optimization problems. Additionally, in WFG test functions with varying numbers of objectives, the proposed algorithm achieves the smallest IGD values, indicating that the adaptive grid strategy and SGP agent model of AG-MOPSO-GPS can achieve more stable optimization results while ensuring convergence and diversity, making it suitable for solving high-dimensional multi-objective optimization problems. Interestingly, regardless of whether the ANN or SVM agent model is used in different test functions, the IGD measure is superior to AG-MOPSO and NSGA-II, indicating the effectiveness of the agent-assisted optimization method in solving multi-objective optimization problems.

TABLE 3

TABLE 3. Comparative analysis of HV measures for test functions.

Meanwhile, the average and standard deviation of the approximate PF obtained using different optimization algorithms for ZDT, DTLZ and WFG test functions based on the HV measure are shown in Table 3. The HV measure can reflect the comprehensive performance in terms of convergence and diversity. It can be observed that the average values obtained by the AG-MOPSO-GPS algorithm are better than those of the other four algorithms, while the standard deviation is better than that of the other algorithms in most cases. This indicates that the proposed algorithm can obtain an optimal solution set that is closer to the true PF in various forms of multi-objective optimization problems, effectively avoiding the algorithm from getting stuck in local optima and demonstrating good global search and local development performance.

To more intuitively compare the convergence and distribution of different algorithms, Figures 2–4 present the PF obtained by different optimization algorithms on different test functions. It can be clearly seen that for the ZDT test function, the non-dominated solutions generated by the AG-MOPSO and NSGA-II algorithms are still far from the true PF of ZDT. However, the agent-assisted multi-objective optimization algorithm can converge to the true PF of the ZDT test function. As for DTLZ1, it has multiple local PFs, and the agent-assisted multi-objective optimization algorithm can escape from local optima, which successfully finds solutions on the true PF. Comparing the ANN and SVM agent models with GP agent model-assisted multi-objective optimization algorithms, it can be found that the GP agent-assisted algorithm can converge better to the true PF. In addition, the PF obtained by the proposed algorithm on the DTLZ2 test function is well-distributed in the objective space, and there are more solutions, which is better than the other compared algorithms. Similarly, for the DTLZ7 test function with a discontinuous PF, the proposed method also achieves a well-distributed and abundant set of non-dominated solutions. In the WFG test functions, the proposed algorithm not only attains a smaller IGD value but also demonstrates better convergence and diversity in the obtained solution set. Overall, the other algorithms’ non-dominated solution sets still fail to fit the true PF effectively, requiring more iterations and incurring higher computational costs.

FIGURE 2

FIGURE 2. The PF effects of five different algorithms on three ZDT test functions.

FIGURE 3

FIGURE 3. The PF effects of five different algorithms on three DTLZ test functions.

FIGURE 4

FIGURE 4. The PF effects of five different algorithms on three WFG test functions.

5 Engineering cases

With the increasingly serious global climate problems, the development of renewable energy sources is urgent, among which wind power is the main force of clean energy. Therefore, improving the capacity of wind farms is a pressing issue. Generally, for wind farms that have been built, improving the power generation efficiency can be achieved by improving the control strategy of wind turbines, while for planned wind farms, the power generation capacity can be improved by optimizing the layout of wind turbines. Wind farm layout optimization is a highly complex multi-objective optimization problem, involving multiple fields. This paper applies the agent-based optimization framework to wind farm layout optimization and proves its superiority in complex optimization problems by comparing with different optimization algorithms.

The wind farm layout optimization problem aims to obtain a wind turbine layout that minimizes the annual energy cost $f_1$, minimizing the overall wind farm area $f_2$, reducing wake loss and maximizing efficiency $f_3$ through a multi-objective strategy. The specific expressions are as follows:

$\begin{array}{l}F\left(\mathit{x},\mathit{y}\right)=\min \left(f_1,f_2,-f_3\right)\\ f_1=\frac{N_t\left(\frac{2}{3}+\frac{1}{3}{e}^{-0.00174N_t^2}\right)}{8760{\displaystyle \sum _{k=0}^{360}}{\displaystyle \sum _{i=1}^{N_t}}{f}_k\left(\theta \right)P_i\left(U_i\left(\theta \right)\right)}\\ f_2=\frac{1}{2}abs\left({\displaystyle \sum _{i=1}^{n-1}}{x}_i{y}_{i-1}+x_n{y}_1-{\displaystyle \sum _{i=1}^{n-1}}{x}_{i+1}{y}_i+x_1{y}_n\right)\\ f_3=100\frac{{\displaystyle \sum _{k=0}^{360}}{\displaystyle \sum _{i=1}^{N_t}}{f}_k\left(\theta \right)P_i\left(U_i\left(\theta \right)\right)}{{\displaystyle \sum _{k=0}^{360}}{\displaystyle \sum _{i=1}^{N_t}}{f}_k\left(\theta \right)P_i\left(U_0\left(\theta \right)\right)}\end{array}(28)$

Subject to

$\begin{array}{l}{\displaystyle \sum _{i=1}^{N_t}}{\displaystyle \sum _{j=1}^{N_t}}\max \left(\left(6D_{wt}-\sqrt{{\left(x_i-x_j\right)}^2+\left(y_i-y_j\right)}\right),0\right)\le 0\\ x_i\ge D_{wt};x_i\le 2000-D_{wt}\\ y_i\ge D_{wt};y_i\le 2000-D_{wt}\\ x_i\ne x_j;y_i\ne y_j\end{array}(29)$

Where, $N_t$ is the total number of wind turbines; $f_k\left(\theta \right)$ represents the wind speed probability in the $\theta$ direction, with a division step of 10°. $U_i\left(\theta \right)$ is the wind speed in the $\theta$ direction. $P_i\left(U_i\left(\theta \right)\right)$ represents the power output under $U_i\left(\theta \right)$. $P_i\left(U_0\left(\theta \right)\right)$ represents the power output obtained in the free stream wind speed (excluding wake loss). $D_{wt}$ is the diameter of the wind turbine. The specific details of wind farm parameters and experimental settings can be found in ref. (Moreno et al., 2021).

The proposed algorithm in this paper is applied to case 1 in Moreno et al. (2021), and compared with the best-performing algorithm MO-LSA under the fixed wind speed (12 m/s) condition, and the best-performing algorithm NSGA-II under the variable wind speed (8, 12, 17 m/s) condition. The performance is shown in Table 4, and it can be observed that the AG-MOPSO-GPS algorithm proposed in this paper is superior in terms of IGD and HV measures in all cases.

TABLE 4

TABLE 4. Metrics performance for constant wind speed and variable wind speed, with $N_t$ = 15.

Figures 5, 6 are optimization comparison diagrams between the optimal method in the literature and the method proposed in this paper. It can be found that the method proposed in this paper obtains more non-dominated solutions and a more uniform distribution when optimizing the layout of wind farms. The optimal solutions of each optimization algorithm were substituted into the objective function to obtain the annual energy cost, wind farm area, and wind turbine efficiency, as shown in Table 5. Compared with the method proposed in this paper, MO-LSA reduced the energy cost by 5.48%, decreased the wind farm area by 10.12%, and improved the wind turbine efficiency by 3.02% under the constant wind speed condition. NSGA-II algorithm reduced the energy cost by 2.71%, decreased the wind farm area by 3.93%, and improved the wind turbine efficiency by 1.60% compared with the method proposed in this paper under the variable wind speed condition. This indicates that developing high-performance optimization algorithms is important for improving efficiency and reducing costs in multi-objective optimization problems in practical industrial applications.

FIGURE 5

FIGURE 5. Layout with the best trade-off. [(A) constant wind speed, (B) variable wind speed].

FIGURE 6

FIGURE 6. The PF of different optimization algorithms. [(A) constant wind speed, (B) variable wind speed].

TABLE 5

TABLE 5. The optimal solutions of different optimization algorithms.

6 Conclusion and future plans

With the continuous improvement of industrial demand, the complexities arising from the interrelatedness of different optimization objectives are increasing. Therefore, the development of efficient methods to solve complex optimization problems is of paramount importance in the realm of multi-objective optimization. In this paper, we thoroughly analyze and study the problems of the multi-objective optimization problems requiring a large number of function evaluations and propose a new method to solve complex optimization problems by combining the AG-MOPSO algorithm with surrogate models. Through comparative research, the main findings are as follows.

1. Validation on different types of test functions demonstrates that the proposed AG-MOPSO-GPS achieves satisfactory results in various metrics. This indicates that the proposed method in this paper can obtain a more realistic PF. Specifically, in the WFG test functions, the ICD and HV metrics outperform other algorithms significantly. This suggests that when multi-objective optimization problems have multiple local optima, the proposed method in this paper effectively assists particles in capturing and maintaining these multiple Pareto optimal solution sets.

2. Compared with the traditional optimization models AG-MOPSO and NSGA-II, the surrogate model assisted multi-objective optimization algorithm can obtain non-dominated solutions and the distribution of solutions is more uniform, which indicates that the surrogate model can provide a more efficient, comprehensive and robust solution for multi-objective optimization problems, which helps to accelerate the optimization process and obtain a better approximate PF.

3. From the PF and solutions distribution, the SGP surrogate model can provide uncertainty estimates of unknown target function values, helping the optimization algorithm be more robust when dealing with noise or uncertainty.

4. In the wind farm layout optimization, this method can effectively reduce the land use area, reduce costs, and improve power generation efficiency. This indicates that for complex problems with multiple conflicting objectives, the proposed method in this paper can efficiently search for the Pareto optimal solution set, making it suitable for tackling complex multi-objective optimization problems.

In generally, the proposed approach in this paper can quickly solve multi-objective optimization problems and is suitable for various industrial requirements, demonstrating strong robustness. Future research plans include exploring online updating techniques for agent models to adapt to dynamic characteristics of optimization problems that change over time or parameters and to timely reflect the regularity of objective functions, in order to improve the real-time and accuracy of agent models.

Data availability statement

The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author.

Author contributions

YC: Methodology, software, writing-original draft, visualization LW: Methodology, Validation, writing-review and editing HH: Validation, writing-review and editing. All authors contributed to the article and approved the submitted version.

Funding

This work is supported by the Hunan Provincial Education Planning Project (XJK23CGD053).

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Akinola, O. O., Ezugwu, A. E., Agushaka, J. O., Zitar, R. A., and Abualigah, L. (2022). Multiclass feature selection with metaheuristic optimization algorithms: a review. Neural Comput. Appl. 34 (22), 19751–19790. doi:10.1007/s00521-022-07705-4

PubMed Abstract | CrossRef Full Text | Google Scholar

Avendaño-Valencia, L. D., Abdallah, I., and Chatzi, E. (2021). Virtual fatigue diagnostics of wake-affected wind turbine via Gaussian Process Regression. Renew. Energy 170, 539–561. doi:10.1016/j.renene.2021.02.003

CrossRef Full Text | Google Scholar

Chen, G., Zhang, K., Xue, X., Zhang, L., Yao, C., Wang, J., et al. (2022). A radial basis function surrogate model assisted evolutionary algorithm for high-dimensional expensive optimization problems. Appl. Soft Comput. 116, 108353. doi:10.1016/j.asoc.2021.108353

CrossRef Full Text | Google Scholar

Ciccazzo, A., Di Pillo, G., and Latorre, V. (2015). A SVM surrogate model-based method for parametric yield optimization. IEEE Trans. Computer-Aided Des. Integr. Circuits Syst. 35 (7), 1224–1228. doi:10.1109/tcad.2015.2501307

CrossRef Full Text | Google Scholar

Coello, C. A. C., Pulido, G. T., and Lechuga, M. S. (2004). Handling multiple objectives with particle swarm optimization. IEEE Trans. Evol. Comput. 8 (3), 256–279. doi:10.1109/tevc.2004.826067

CrossRef Full Text | Google Scholar

Cui, Y., Geng, Z., Zhu, Q., and Han, Y. (2017). Review: multi-objective optimization methods and application in energy saving. Energy 125, 681–704. doi:10.1016/j.energy.2017.02.174

CrossRef Full Text | Google Scholar

Currin, C., Mitchell, T., Morris, M., and Ylvisaker, D. (1991). Bayesian prediction of deterministic functions, with applications to the design and analysis of computer experiments. J. Am. Stat. Assoc. 86 (416), 953–963. doi:10.1080/01621459.1991.10475138

CrossRef Full Text | Google Scholar

Ghahramani, Z. (2015). Probabilistic machine learning and artificial intelligence. Nature 521 (7553), 452–459. doi:10.1038/nature14541

PubMed Abstract | CrossRef Full Text | Google Scholar

Giovanis, D. G., and Shields, M. D. (2020). Data-driven surrogates for high dimensional models using Gaussian process regression on the Grassmann manifold. Comput. Methods Appl. Mech. Eng. 370, 113269. doi:10.1016/j.cma.2020.113269

CrossRef Full Text | Google Scholar

Golparvar, B., Papadopoulos, P., Ezzat, A. A., and Wang, R-Q. (2021). A surrogate-model-based approach for estimating the first and second-order moments of offshore wind power. Appl. Energy 299, 117286. doi:10.1016/j.apenergy.2021.117286

CrossRef Full Text | Google Scholar

Grimstad, B., Foss, B., Heddle, R., and Woodman, M. (2016). Global optimization of multiphase flow networks using spline surrogate models. Comput. Chem. Eng. 84, 237–254. doi:10.1016/j.compchemeng.2015.08.022

CrossRef Full Text | Google Scholar

Gu, Q., Wang, Q., Li, X., and Li, X. (2021). A surrogate-assisted multi-objective particle swarm optimization of expensive constrained combinatorial optimization problems. Knowledge-Based Syst. 223, 107049. doi:10.1016/j.knosys.2021.107049

CrossRef Full Text | Google Scholar

Han, Z-H., and Zhang, K-S. (2012). Surrogate-based optimization. Real-world Appl. Genet. algorithms 343. doi:10.48550/arXiv.2105.03893

CrossRef Full Text | Google Scholar

He, C., Zhang, Y., Gong, D., and Ji, X. (2023). A review of surrogate-assisted evolutionary algorithms for expensive optimization problems. Expert Syst. Appl. 217, 119495. doi:10.1016/j.eswa.2022.119495

CrossRef Full Text | Google Scholar

Jeong, S., Murayama, M., and Yamamoto, K. (2005). Efficient optimization design method using kriging model. J. Aircr. 42 (2), 1375–1420. doi:10.2514/1.c10485e

CrossRef Full Text | Google Scholar

Jones, D. R., Schonlau, M., and Welch, W. J. (1998). Efficient global optimization of expensive black-box functions. J. Glob. Optim. 13, 455–492. doi:10.1023/a:1008306431147

CrossRef Full Text | Google Scholar

Joseph, V. R., Hung, Y., and Sudjianto, A. (2008). Blind kriging: A new method for developing metamodels. New York City, Unirted States: ASME.

Google Scholar

Kudela, J., and Matousek, R. (2022). Recent advances and applications of surrogate models for finite element method computations: A review. Soft Comput. 26 (24), 13709–13733. doi:10.1007/s00500-022-07362-8

CrossRef Full Text | Google Scholar

Li, K., Wang, R., Zhang, T., and Ishibuchi, H. (2018). Evolutionary many-objective optimization: A comparative study of the state-of-the-art. IEEE Access 6, 26194–26214. doi:10.1109/access.2018.2832181

CrossRef Full Text | Google Scholar

Li, Y., Liu, H., Xie, K., and Yu, X. (2015). “A method for distributing reference points uniformly along the Pareto front of DTLZ test functions in many-objective evolutionary optimization,” in 2015 5th International Conference on Information Science and Technology (ICIST), Changsha, China, 24-26 April 2015, 541–546.

CrossRef Full Text | Google Scholar

Lim, W. J., Jambek, A. B., and Neoh, S. C. (2015). Kursawe and ZDT functions optimization using hybrid micro genetic algorithm (HMGA). Soft Comput. 19, 3571–3580. doi:10.1007/s00500-015-1767-5

CrossRef Full Text | Google Scholar

Ling, C., Kuo, W., and Xie, M. (2022). Complementary and alternative medicine during COVID-19 pandemic: what we have done. IEEE Trans. Reliab. 20, 1–3. doi:10.1016/j.joim.2021.11.008

CrossRef Full Text | Google Scholar

Liu, B., Koziel, S., and Zhang, Q. (2016). A multi-fidelity surrogate-model-assisted evolutionary algorithm for computationally expensive optimization problems. J. Comput. Sci. 12, 28–37. doi:10.1016/j.jocs.2015.11.004

CrossRef Full Text | Google Scholar

Liu, B., Zhang, Q., and Gielen, G. G. (2013). A Gaussian process surrogate model assisted evolutionary algorithm for medium scale expensive optimization problems. IEEE Trans. Evol. Comput. 18 (2), 180–192. doi:10.1109/tevc.2013.2248012

CrossRef Full Text | Google Scholar

Liu, F., Zhang, Q., and Han, Z. (2023b). MOEA/D with gradient-enhanced kriging for expensive multiobjective optimization. Nat. Comput. 22 (2), 329–339. doi:10.1007/s11047-022-09907-0

CrossRef Full Text | Google Scholar

Liu, J., Wang, Y., Sun, G., and Pang, T. (2023a). Solving highly expensive optimization problems via evolutionary expected improvement. IEEE Trans. Syst. Man, Cybern. Syst. 53, 4843–4855. doi:10.1109/tsmc.2023.3257030

CrossRef Full Text | Google Scholar

Liu, J-Q., Feng, Y-W., Xue, X-F., and Lu, C. (2021). Intelligent extremum surrogate modeling framework for dynamic probabilistic analysis of complex mechanism. Math. Problems Eng. 2021, 1–12. doi:10.1155/2021/6681489

CrossRef Full Text | Google Scholar

Liu, X., Zhao, W., and Wan, D. (2022). Multi-fidelity Co-Kriging surrogate model for ship hull form optimization. Ocean. Eng. 243, 110239. doi:10.1016/j.oceaneng.2021.110239

CrossRef Full Text | Google Scholar

Lystad, T. M., Fenerci, A., and Øiseth, O. (2023). Full long-term extreme buffeting response calculations using sequential Gaussian process surrogate modeling. Eng. Struct. 292, 116495. doi:10.1016/j.engstruct.2023.116495

CrossRef Full Text | Google Scholar

Mavrovouniotis, M., Li, C., and Yang, S. (2017). A survey of swarm intelligence for dynamic optimization: algorithms and applications. Swarm Evol. Comput. 33, 1–17. doi:10.1016/j.swevo.2016.12.005

CrossRef Full Text | Google Scholar

Moreno, S. R., Pierezan, J., dos Santos Coelho, L., and Mariani, V. C. (2021). Multi-objective lightning search algorithm applied to wind farm layout optimization. Energy 216, 119214. doi:10.1016/j.energy.2020.119214

CrossRef Full Text | Google Scholar

Nguyen, A-T., Reiter, S., and Rigo, P. (2014). A review on simulation-based optimization methods applied to building performance analysis. Appl. energy 113, 1043–1058. doi:10.1016/j.apenergy.2013.08.061

CrossRef Full Text | Google Scholar

Palmer, S. (2019). Evolutionary algorithms and computational methods for derivatives pricing: Ucl. London: University College London.

Google Scholar

Parsopoulos, K. E., and Vrahatis, M. N. (2008). Multi-objective optimization in computational intelligence: Theory and practice. Pennsylvania, United States: IGI global, 20–42.

Google Scholar

Preen, R. J., Bull, L., and Adamatzky, A. (2019). Towards an evolvable cancer treatment simulator. Biosystems 182, 1–7. doi:10.1016/j.biosystems.2019.05.005

PubMed Abstract | CrossRef Full Text | Google Scholar

Satria Palar, P., Rizki Zuhal, L., and Shimoyama, K. (2020). Gaussian process surrogate model with composite kernel learning for engineering design. AIAA J. 58 (4), 1864–1880. doi:10.2514/1.j058807

CrossRef Full Text | Google Scholar

Shadab, S., Hozefa, J., Sonam, K., Wagh, S., and Singh, N. M. (2022). Gaussian process surrogate model for an effective life assessment of transformer considering model and measurement uncertainties. Int. J. Electr. Power and Energy Syst. 134, 107401. doi:10.1016/j.ijepes.2021.107401

CrossRef Full Text | Google Scholar

Su, G., Peng, L., and Hu, L. (2017). A Gaussian process-based dynamic surrogate model for complex engineering structural reliability analysis. Struct. Saf. 68, 97–109. doi:10.1016/j.strusafe.2017.06.003

CrossRef Full Text | Google Scholar

Sun, C., Jin, Y., Cheng, R., Ding, J., and Zeng, J. (2017). Surrogate-assisted cooperative swarm optimization of high-dimensional expensive problems. IEEE Trans. Evol. Comput. 21 (4), 644–660. doi:10.1109/tevc.2017.2675628

CrossRef Full Text | Google Scholar

Talgorn, B., Audet, C., Le Digabel, S., and Kokkolaras, M. (2018). Locally weighted regression models for surrogate-assisted design optimization. Optim. Eng. 19, 213–238. doi:10.1007/s11081-017-9370-5

CrossRef Full Text | Google Scholar

Vafadar, A., Guzzomi, F., Rassau, A., and Hayward, K. (2021). Advances in metal additive manufacturing: A review of common processes, industrial applications, and current challenges. Appl. Sci. 11 (3), 1213. doi:10.3390/app11031213

CrossRef Full Text | Google Scholar

Wang, C., Qiang, X., Xu, M., and Wu, T. (2022b). Recent advances in surrogate modeling methods for uncertainty quantification and propagation. Symmetry 14 (6), 1219. doi:10.3390/sym14061219

CrossRef Full Text | Google Scholar

Wang, Z., Sun, S., and Ding, Y. (2022a). Fatigue optimization of structural parameters for orthotropic steel bridge decks using RSM and NSGA-II. Math. Problems Eng. 2022, 1–12. doi:10.1155/2022/4179898

CrossRef Full Text | Google Scholar

Yang, J.-J., Zhou, J.-Z., Fang, R.-C., Li, Y.-H., and Liu, L. (2008). Multi-objective particle swarm optimization based on adaptive grid algorithms. J. Syst. Simul. 20 (21), 5843–5847.

Google Scholar

Zhang, Z., Ye, L., Qin, H., Liu, Y., Wang, C., Yu, X., et al. (2019). Wind speed prediction method using shared weight long short-term memory network and Gaussian process regression. Appl. energy 247, 270–284. doi:10.1016/j.apenergy.2019.04.047

CrossRef Full Text | Google Scholar

Zhao, H., Gao, Z-H., and Xia, L. (2022). Efficient aerodynamic analysis and optimization under uncertainty using multi-fidelity polynomial chaos-Kriging surrogate model. Comput. Fluids 246, 105643. doi:10.1016/j.compfluid.2022.105643

CrossRef Full Text | Google Scholar

Zheng, N., Wang, H., and Yuan, B. (2022). An adaptive model switch-based surrogate-assisted evolutionary algorithm for noisy expensive multi-objective optimization. Complex and Intelligent Syst. 8 (5), 4339–4356. doi:10.1007/s40747-022-00717-6

CrossRef Full Text | Google Scholar

Zhou, Z., Ong, Y. S., Nair, P. B., Keane, A. J., and Lum, K. Y. (2006). Combining global and local surrogate models to accelerate evolutionary optimization. IEEE Trans. Syst. Man, Cybern. Part C Appl. Rev. 37 (1), 66–76. doi:10.1109/tsmcc.2005.855506

CrossRef Full Text | Google Scholar