State assessment of 110–220 kV intelligent substation based on multisensor fusion algorithm control and image vision

With the development of modern information technology, intelligent substation technology has been widely used, which greatly promotes the development of power grid. The information integration platform of intelligent substations realizes panoramic data collection and data sharing of substations. With the continuous development of the economy and society, it is crucial to improve the power regulation ability of intelligent substations, make scientific use of intelligent technology, and optimize the patrol level of intelligent substation patrol robots to better meet the operation needs of substations. In the actual substation construction, the flexibility of equipment utilization in intelligent substations should be fully considered. In addition, the number of substations should be controlled to improve the utilization efficiency of patrol robots. This paper mainly summarizes the monitoring technology of intelligent substations, the positioning technology of inspection robots, and the multi-sensor control technology to lay a foundation for the follow-up inspection work of intelligent substations.

Introduction

In recent years, with the development of power transmission and transformation technology around the world, the scale of the power grid has been continuously expanded, and the voltage level of power transmission and transformation has been continuously improved, which inevitably increases the number and complexity of power equipment, and the maintenance cost of equipment accounts for the power grid. The proportion of the total operating cost is increasing day by day, and the maintenance workload of relay protection equipment is increasing sharply (Zheng et al., 2022). In order to ensure the stable and reliable operation of the power grid in the case of such a large number of equipment and high equipment complexity, and at the same time control maintenance costs to reduce costs and reduce the workload of relay inspection, the maintenance strategy of power equipment has changed from regular maintenance in the past to state Overhaul development. Both periodic maintenance and condition maintenance are preventive maintenance (Yang et al., 2021). The disadvantage of regular maintenance is that the maintenance cycle is relatively fixed, and the maintenance cannot be carried out in time according to the health status of the equipment, which is easy to cause problems such as insufficient maintenance and excessive maintenance. Condition-based maintenance is to obtain various state parameters of the equipment during operation through advanced state monitoring methods, identify the early signs of failures according to the actual working conditions of the equipment reflected by these state parameters, and make judgments on the fault location, fault severity, and development trend, to determine the optimal maintenance time of the equipment as a maintenance method (Shin et al., 2021).

With the development of the power industry around the world, the data of the power system has increased dramatically and has now reached the petabyte (PB) level. This means that the current power industry is entering the era of big data (Wang et al., 2021). Therefore, it is imperative to develop a maintenance strategy that can stably guarantee the operation of the power grid based on massive data. The maintenance of substation status is the basic work of power supply guarantee and power security, but it usually has great uncertainty and uncontrollability, so it puts forward higher requirements for the maintenance of substations. At present, there are many strategies applied to substation power grid maintenance, such as time-based maintenance strategy, corrective maintenance strategy, and condition-based maintenance (CBM) strategy. Compared with the first two methods, CBM is widely used because it can reduce fault loss and prolong the service life of equipment (Xu et al., 2021).

At present, it is crucial to vigorously develop the key equipment and promote the construction of intelligent substation. Inspiringly, the intelligent inspection robot based on image vision and multi-sensor fusion has been applied to the status monitoring and intelligent maintenance of substations, and the cloud platform based on a large number of monitoring data has also been embedded in substations, called intelligent substations. Compared with the traditional substation, the intelligent substation currently under construction has a compact structure design, centralized secondary equipment, more reasonable layout, and small footprint. Intelligent substation is a high-level substation that adopts advanced intelligent equipment and can automatically complete basic functions such as information collection, measurement, control, protection, measurement, and monitoring with digitalization and standardization of information sharing of the whole station as the basic requirements. In particular, Figure 1 clearly shows the hardware structure and overall layout of the smart substation.

FIGURE 1

FIGURE 1. Overall layout diagram and hardware system structure of online monitoring system of smart substation.

The intelligent substation evolved from the digital substation. After 10 years of development, the technology has become increasingly perfect. Compared with other links, intelligent substation has reached the conditions for large-scale promotion. The intelligent substation is mainly composed of equipment layer and system layer. The biggest difference between the intelligent substation and the traditional substation is reflected in three aspects: the intelligence of primary equipment, the state of equipment maintenance, and the networking of secondary equipment.

After 2011, all newly-built substations will be constructed in accordance with the technical standards for intelligent substations, and the hub and central substation will be focused on intelligent transformation. As of the sixth batch of bidding in 2013, the bidding volume of smart substations has reached about 3600, which is basically in line with the planning expectation. However, the proportion of bidding for recent batches of smart substations has decreased compared with the previous period. As the pilot project of a new generation of intelligent substation is gradually put into operation, the proportion of intelligent substation bidding will continue to increase, and the construction progress will continue to accelerate.

Condition monitoring of smart substations can provide an important guarantee for maintenance and protection. In literature (Gao et al., 2022), a new state maintenance scheme is proposed and a big data architecture is designed for the state maintenance of power big data substation equipment to achieve the data collection and application integration functions of multi-source substation equipment. The intelligent detection system can obtain the operation data information of substation equipment, including substation, circuit breaker, and capacitor equipment. The intelligent monitoring system improves the traveling wave positioning ability by establishing a big data multi-classification fusion positioning model. Work (Xu, 2019) effectively monitors the intelligent operation and maintenance monitoring platform of the substation by deeply integrating the three modules of intelligent patrol monitoring, intelligent auxiliary equipment monitoring, and personnel management intelligent monitoring to achieve multi-source information fusion, which can be multi-dimensional, comprehensive, and visualized. Real-time monitoring of the operation status of main/auxiliary equipment and personnel safety of substations, to achieve comprehensive management and control of operation and maintenance equipment and personnel, and to improve operation and maintenance quality and efficiency. Reference (Fei et al., 2016) uses multi-source information fusion technology to automatically identify the operation status of high-voltage circuit breakers in substations. It has the characteristics of fast real-time response, strong anti-interference ability, and convenient maintenance. It is also convenient for operation management and monitoring, improves the safety performance of relay protection devices in power plants, and effectively ensures the stable operation of units. Reference (Pan and Zhang, 2013) introduces a substation auxiliary system management and control platform based on multi-sensor networks to meet the development requirements of intelligent substations. This method improves the reliability and stability of real-time monitoring of substation information, and identifies and classifies fault targets. In addition, Table 1 summarizes the requirements for intelligent equipment of conventional 110–220 kV substations.

TABLE 1

TABLE 1. Functional requirements of intelligent equipment for 110–220 kV substations.

The rest of this paper is arranged as follows: The second part introduces the substation control technology and substation fault diagnosis technology based on neural network, the third part introduces the positioning function of substation patrol robot based on multi-source data fusion, the fourth part introduces the substation patrol robot based on multi-sensor fusion control technology, and the last part is the conclusion and prospects for future research.

Substation monitoring and control technology

Substation control technology based on machine vision

As a comprehensive technology including digital image processing and machine vision, substation monitoring technology has become an important part of ensuring the operation and maintenance of substations. The substation monitoring task is mainly realized through the Pan/Tilt/Zoom (PTZ) system, which is mainly composed of PTZ and testing equipment. A stable PTZ is the hardware prerequisite for obtaining high-quality information, and the real-time robust visual processing algorithm is the software guarantee for efficient detection. To realize the stable operation of the substation inspection work, the gimbal needs to meet the requirements of high control accuracy, good dynamic performance, and easy secondary development and can obtain equipment inspection information stably and quickly in the substation site with strong electricity and magnetism.

In addition to high-performance PTZ control technology and corresponding detection equipment, substation monitoring technology also requires stable and fast image detection algorithms. Visual detection technology is one of the key research directions at this stage, the essence of which is to detect the target position from the complex background environment, mark it with a rectangular frame, and perform coordinate acquisition and target classification. In particular, target detection techniques mainly include traditional processing methods and deep learning methods.

The double cascade parameter regression normalizes the target image, which realizes the registration of the instrument image and greatly reduces the false detection rate of the instrument (Fang, 2017). With the development of hardware equipment and the research of related algorithms, the target monitoring technology based on deep learning has greatly improved in terms of processing speed and monitoring accuracy, including Two-Stage and One-Stage (Duan et al., 2020; Liu and Meng, 2020; Lu et al., 2020). Two-Stage includes two parts: candidate area generation, classification, and position trimming. The target monitoring accuracy is high but the monitoring speed is slow, which cannot meet the real-time requirements. One-Stage is mainly based on regression, does not require pre-generated candidate frames, has a simple structure, and has high monitoring speed. Since the introduction of convolutional neural networks, target monitoring algorithms have developed towards real-time performance and high precision. In 2015, HeK et al. proposed the SPP-Net spatial pyramid pooling network based on the RCNN algorithm (Purkait et al., 2015), which broke through the limitation of image input size and improved the monitoring speed. In 2017, Girshick improved the R-CNN algorithm and proposed Fast-R-CNN (Shaoqing et al., 2015) and Faster-R-CNN (Redmon et al., 2016), which simplified the network structure and improved the efficiency of region matching respectively and greatly improved the accuracy and speed., but still can not meet the needs of real-time monitoring. In 2017, RedMon et al. proposed YOLO9000, which realizes real-time monitoring of more than 9000 categories by combining classification and monitoring tasks (Beaudry and Poirier, 2012; Redmon and Farhadi, 2017). In 2018, RedMon et al. proposed the YOLOv3 algorithm (Redmon and Farhadi, 2018), which introduced a residual network structure, improved network performance, and improved monitoring accuracy and small target monitoring capabilities. In 2020, Alexey B combined various optimization strategies in the field of deep learning and proposed YOLOv4 based on the YOLOv3 framework (Bochkovskiy et al., 2020), which achieved efficient and accurate target monitoring. To sum up, substation monitoring mainly uses the PTZ equipped with visible light, infrared, and other multi-source visual devices to collect information about the substation equipment and judge the operating status through the target monitoring and identification algorithm. Traditional monitoring algorithms have good monitoring performance in specific scenarios but have poor scalability and low intelligence, which cannot meet the equipment monitoring requirements in the complex environment of substations. The monitoring algorithm based on deep learning is trained through the data set, and the neural network automatically extracts the feature of the image target according to the label to complete the monitoring and identification work. However, it is not widely used in the field of substation equipment monitoring, and the general monitoring model is not yet clear. At present, substation monitoring mainly adopts traditional monitoring methods, and the method of deep learning cannot perform simultaneous monitoring of visible light and infrared, and the real-time monitoring is poor, which needs further development.

Substation fault diagnosis technology based on parallel convolutional neural network

At present, the research on substation fault diagnosis is based on the fault mechanism model of HVDC transmission, and the other part is based on the method of line signal analysis, which does not simplify the circuit model. With the continuous development of artificial intelligence algorithms, artificial intelligence algorithms have been widely used in substation system fault diagnosis. In particular, the common fault types of 110–220 kV substation equipment are clearly listed in Figure 2. At present, both fault diagnosis methods based on mechanism model and signal analysis have the problem that the fault threshold is not easy to determine. Encouragingly, a substation fault diagnosis method with both rapidity and accuracy is proposed. Firstly, the amplitude and frequency characteristics of substation line fault signal waveform are analyzed, and the fault amplitude feature extraction method based on the amplitude change of signal waveform and the fault frequency feature extraction method based on wavelet packet analysis are studied to form a fault diagnosis method based on amplitude frequency characteristics. Secondly, a dual branch convolutional neural network with fault classification branch and fault location branch-parallel convolutional neural network is constructed, and a training method based on transfer learning is proposed.

FIGURE 2

FIGURE 2. Fault identification of secondary equipment in 110–220 kV substation. (A) Arrester monitor pointer meter fault, (B) Auxiliary transformer bypass knife switch fault, (C) Air Switch of Emergency Lighting Inverter Panel fault, and (D) Opening-Closing Identification of Transformer fault.

As the current and voltage will fluctuate during the fault and finally stabilize to the new value, if the new stable value is different from the stable value during normal operation, the electrical quantity will change, so the fault type can be distinguished. The amplitude change characteristics before and after the new signal waveform are extracted as follows:

$x\left(i\right)=\left[{x_{T-1},x_{T-1+\Delta t},x_{T-1+2\Delta t},\dots ,x_{T-1+m\Delta t},x}_T,x_{T+\Delta t},x_{T+2\Delta t},\dots ,x_{T+m\Delta t}\right](1)$

where: $x\left(i\right)$ is the data after synthesis, T is the time of failure, ΔT is the time interval of the sampling point, ΔT = 10-4s, m = 19, indicating that the number of sampling points m+1 is 20.

When the short circuit fault occurs in the substation system, the line current and voltage will produce high-frequency components, and the fault classification and location can be achieved by detecting these information. Eqs 2, 3 are wavelet packet decomposition algorithm and wavelet packet reconstruction algorithm respectively.

$\{\begin{array}{c}{\mathit{d}}_{j,\mathrm{1,2}n}=\sqrt{2}{\displaystyle \sum }_k{h}_{0,k-2l}{\mathit{d}}_{j+1,k,n}\\ {\mathit{d}}_{j,\mathrm{1,2}n+1}=\sqrt{2}{\displaystyle \sum }{h}_{1,k-2l}{\mathit{d}}_{j+1,k,n}\end{array}(2)$

${\mathit{d}}_{j+\mathrm{1,1},n}={\displaystyle \sum }_k{h}_{0,k-2l}{\mathit{d}}_{j,\mathrm{1,2}n}+h_{1,k-2l}{\mathit{d}}_{j,\mathrm{1,2}n+1}(3)$

where ${\mathit{d}}_{j+1,k,n}$ is the results of wavelet packet decomposition of the previous layer, ${\mathit{d}}_{j,\mathrm{1,2}n}$ and ${\mathit{d}}_{j,\mathrm{1,2}n+1}$ are the wavelet packet decomposition results of the next level, j is the scale index, n is the frequency index, l is the location index, k is a variable, $h_{0,k-2l}$ and $h_{1,k-2l}$ are the $k-2l$ multi-resolution filter coefficients used for decomposition.

Substation fault diagnosis method based on improved genetic algorithm

Aiming at the problems of premature, low accuracy and slow convergence of traditional optimization algorithm in solving the analytical model of substation fault diagnosis, a substation fault identification method based on improved genetic algorithm was developed. First of all, in view of the shortcomings of traditional genetic algorithms such as slow convergence and insufficient global search ability due to the fixed crossover and mutation probability, the crossover probability related to evolutionary algebra and mutation probability related to individual fitness are designed, and the existing adaptive genetic algorithms are improved to improve the global search ability and convergence speed of the algorithm, Secondly, the analytic model of power grid fault identification is constructed according to the topology and protection configuration of the power grid. Finally, the improved genetic algorithm is applied to solve the above fault diagnosis model. The basic steps of traditional grid fault identification algorithm based on mathematical analytic model are as follows:

Step 1: generate fault hypothesis randomly.

$S=\left[\begin{array}{c}\mathit{D}\mathit{C}\mathit{R}\end{array}\right](4)$

where, D, C, R are respectively AC equipment, circuit breaker, and protected line vectors, and D is the corresponding equipment in fault or normal state, taking 0 or 1, C means that the corresponding circuit breaker is in the active or inactive state, and the value is 0 or 1, R means that the corresponding protection is in the active or inactive state, and the value is 0 or 1.

Step 2: describe the action logic of all circuit breakers and equipment protection of the power grid through strict mathematical equations, and calculate their expected values.

$r_m^{*}=d_k\times {\overline{r}}_i\times {\overline{r}}_j+{\displaystyle \sum }_{d_l\in D\left(R_i\right)}\left(d_l\times \prod _{c_p\in C\left(R_m,D_l\right)}{\overline{c}}_p\right)(5)$

where ${\overline{r}}_i$ is the expected value of the ith backup protection equipment under the fault hypothesis, $d_k$ is the fault hypothesis value of the protected equipment D_k, ${\overline{r}}_j$ is the fault hypothesis value of the jth near backup protection equipment.

The circuit breaker failure protection action logic is as follows:

$f_{c_i}^{*}={\overline{c}}_i\times {\displaystyle \sum }_{r_x\in R\left({\mathrm{c}}_i\right)}{r}_x(6)$

where $f_{c_i}^{*}$ is the breaker failure protection corresponding to the breaker c_i, and $R\left({\mathrm{c}}_i\right)$ is the set of all protections that can make the breaker ${\mathrm{c}}_i$ act.

The action logic of the circuit breaker is as follows:

$c_k^{*}={\displaystyle \sum }_{r_i\in \mathit{R}\left(c_k\right)}{r}_i(7)$

where $c_k^{*}$ is the expected value of circuit breaker c_k under the fault hypothesis state, and $\mathit{R}\left(c_k\right)$ is the protection set that can control the action of circuit breaker c_k.

Step 3, build the differentiation function between the actual state and the expected state of the circuit breaker and protection as the objective function.

$E_{\left(S\right)}={\displaystyle \sum }_{i=1}^{n_c}\left|c_i-c_i^{\alpha }\right|+{\displaystyle \sum }_{j=1}^{n_r}\left|r_j-r_j^{\alpha }\right|+{\displaystyle \sum }_{i=1}^{n_c}\left|c_i-c_i^{*}\right|+{\displaystyle \sum }_{j=1}^{n_r}\left|r_j-r_j^{*}\right|+\alpha \left({\displaystyle \sum }_{i=1}^{n_d}{d}_i\right)(8)$

where $r_j^{\alpha }$ and $c_i^{\alpha }$ They are respectively the action status of the jth protection and circuit breaker received by the dispatching center.

Step 4: Minimize and optimize the objective function by improving the genetic optimization algorithm, and the optimal solution is the fault identification result. The cross strategy is as follows:

$P_c\left(t\right)=\{\begin{array}{cc}{P}_{c,\max }\times e^{\left(-\frac{t}{T_{\mathrm{G}\mathrm{e}\mathrm{n}}}\right)-{\left(-\frac{t}{T_{\mathrm{G}\mathrm{e}\mathrm{n}{)}^2}}\right)}^2}& \mathrm{i}\mathrm{f}\left(m_{\mathrm{t}\mathrm{m}\mathrm{p}}>P_{c,\min }\right)\\ P_{c,\min }& \mathrm{e}\mathrm{l}\mathrm{s}\mathrm{e}\end{array}(9)$

where $m_{\mathrm{t}\mathrm{m}\mathrm{p}}$ is an intermediate calculated variable, $T_{\mathrm{G}\mathrm{e}\mathrm{n}}$ is the preset maximum evolutionary algebra, t is the current evolutionary algebra, $P_{c,\min }$ and $P_{c,\max }$ are the preset minimum and maximum crossing probabilities respectively, $P_c\left(t\right)$ is the cross probability of the current population.

The mutation operation can maintain the diversity of the population and plays an important role in suppressing the premature of the algorithm, as follows:

$P_m\left(t\right)=\{\begin{array}{ll}\frac{2-\left|\frac{f_{\max }-f\left(X_i\right)}{f_{\max }}\right|}{1+{\left(t/T_{\mathrm{G}\mathrm{e}\mathrm{n}}\right)}^2+t/T_{\mathrm{G}\mathrm{e}\mathrm{n}}}{P}_{m,\max }& \mathrm{i}\mathrm{f}\left(m_{\mathrm{t}\mathrm{m}\mathrm{p}}>P_{m\cdot \min }\right)\\ P_{m\cdot \min }& \mathrm{e}\mathrm{l}\mathrm{s}\mathrm{e}\end{array}(10)$

where $P_{m,\max }$ and $P_{m\cdot \min }$ is the preset maximum and minimum probability of variation respectively, $f\left(X_i\right)$ is the fitness of individual $X_i$, $f_{\max }$ and $f_{\min }$ are the maximum and minimum fitness values of the population respectively, $P_m\left(t\right)$ is the variation probability of individual $X_i$ in the t generation population.

In particular, Figure 3 is the flow chart of substation fault identification algorithm based on the above improved genetic algorithm.

FIGURE 3

FIGURE 3. Flow chart of substation fault identification algorithm based on improved genetic algorithm.

Location technology of substation inspection robot based on multi-source fusion

Intelligent positioning of substation inspection robot

The traditional single inspection mode of substation is manual inspection. With the rapid development of economy, the scale of power system is expanding, and the requirements for system stability are increasing. The existing manual inspection mode is difficult to meet the above needs due to its high labor intensity, scattered inspection quality and severe interference from bad weather. It is difficult to make an objective, comprehensive and accurate judgment simply by relying on the sense and experience of patrol inspectors, laying hidden dangers for the safe operation of equipment. In recent years, a large number of substation patrol robots adapted to different environmental conditions have been developed, which has greatly promoted the application of intelligent patrol robots in substations, and promoted the improvement of substation patrol efficiency while saving manpower.

So far, the tracking technology of mobile robot mainly includes radar tracking, visual tracking and multi-sensor fusion tracking. Among them, radar tracking technology is the most widely used, but radar tracking is difficult to determine the type and geometric shape of the measured object, and it is difficult to achieve stable tracking in complex scenes, and the cost of radar components is high; Multi sensor fusion tracking is a relatively stable tracking method because it can obtain rich features, but it is difficult to register coordinates and has high cost; Compared with other technologies, visual tracking has a better recognition rate for the category of tracked objects, and the price of visual sensors is lower. However, this technology involves a large amount of computation, which limits its application in the field of mobile robots. In particular, work (LUO et al., 2021) analyzed the current mainstream motion model generation candidate box method of patrol robots, and used adaptive Kalman filtering and least square method to process the robot’s inertial sensor output, thus realizing the attitude estimation of the vehicle platform.

Early patrol robots only stayed in the experimental stage. In 2009, Canada developed a remote-controlled wheeled inspection robot using a handle, which can be equipped with RGB camera and infrared camera to inspect the power plant. In 2012, Massey University of New Zealand developed a robot control system and carried a vision acquisition device to monitor changes in the grid environment (Guo et al., 2009). At present, most of the existing inspection systems adopt the guidance and positioning method, and there are shortcomings in the laying process, such as excessive workload, low flexibility in path fixation, and inability to accurately anchor the specific position of the inspection robot (Salvucci et al., 2014). Therefore, studying accurate and robust navigation and positioning algorithms can lay a foundation for the stable operation of autonomous inspection robots (Zhang, 2019). Navigation and positioning is the key technology for patrol robot to realize intelligent patrol. Patrol robots generally use preset magnetic tracks and radio frequency identification (RFID) technology for positioning (Ju et al., 2020), but this method has low positioning accuracy and weak anti-interference ability. In recent years, although the sensor performance has been continuously improved, the positioning ability of a single sensor in harsh environments is still insufficient. For example, satellite navigation and positioning may fail in environments with large electromagnetic interference (Chi et al., 2021). The inertial navigation system is less affected by the environment, but the price of high-precision inertial measurement unit (IMU) is high, and low-precision IMU is easy to accumulate positioning errors (Dong et al., 2021). There are problems of unclear texture features and failure under high dynamic conditions when visual sensors are used for navigation and positioning (Budiyono, 2013), and there are problems of degradation when lidar sensors are used for positioning in open environments (Zhang et al., 2021). Document (Frosi and Matteucci, 2022) proposes a multi-sensor fusion algorithm based on Federated Kalman filtering (FKF) to integrate fault monitoring and information recovery algorithms and builds a smart grid platform based on AAA level game engine, A virtual sensor is constructed to verify the navigation and positioning algorithm of the smart grid inspection robot. In recent years, intelligent inspection robots developed around the world based on accurate positioning algorithms and advanced virtual sensors not only save manpower in substations but also improve maintenance efficiency. In particular, Figure 4 shows the mainstream substation monitoring robots around the world.

FIGURE 4

FIGURE 4. World mainstream substation intelligent inspection robot. (A) Hangzhou substation inspection robot in China, (B) Shandong Luneng intelligent maintenance robot in China, (C) Canadian substation monitoring robot, and (D) American electric power detection and operation robot.

Multi-robot joint inspection system based on multi-source data

A multi-robot system is a collection of robots that have dynamic and kinematic characteristics and can interact with other individuals. It can be predicted that more and more robots will be put into practical applications in the future, and the related research on multi-robot system will also have greater application value. The use of multi-robot linkage system in substations can show strong adaptability to complex environments. Its strong carrying capacity greatly shortens the working time and effectively improves production efficiency. In a multi-robot system, the completion of a task requires the participation of each robot, rather than completely depending on one robot. If the working environment changes or the system fails, the robot can re-plan the task to adapt to the environment through the central controller and its coordination system, so it has strong fault tolerance and robustness.

The inspection robot system is generally composed of the robot body, moving parts, background monitoring, control bus, and network, which involves the structural design of the robot and the intelligent control of the robot (Wen et al., 2022). According to the actual operation of the substation, this paper analyzes the existing technical level and functional requirements of the substation inspection robot, summarizes the information fusion technology of the inspection robot, and the dynamic planning of the inspection path into three points, as follows:

a) Dynamic planning of patrol path based on multimodal environment information interactive perception: the calibration matching optimization algorithm between three-dimensional visual image sensing recognition and Radar Real-time ranging sensors is used to establish a multimodal environment information interactive perception fusion model, and then analyze the real-time reconstruction method of the non-structural complex environment. The dynamic tracking compensation model of inspection path is constructed, and the autonomous path planning and obstacle avoidance control algorithm of substation inspection multi-robot system based on the unstructured complex environment reconstruction model is analyzed;

b) Dynamic topology optimization and ad hoc networking of multi-robot mobile node network structure: MANET Routing Protocol mechanism, dynamic topology model, and real-time dynamic update of routing table of multi-robot system group in substation inspection are adopted. Combining the path dynamic change information of each node and the dynamic topology optimization of network nodes, the model construction of information interaction and fusion between the two is analyzed, and then the research on network topology change perception, network topology connection maintenance, and multi-channel access is carried out;

c) Multi-robot complex system modeling: study the local interaction mechanism and coordination control algorithm between individual robots, and realize information perception and interaction between individual robots through controller design, sensor configuration, and multi-source information fusion. A multi-robot complex system modeling method based on the fusion of dynamic path planning, mobile network information transmission, multi-modal information interaction, and other technologies realizes the global coordinated control of multi-robot systems.

Power inspection robot based on multi-sensor fusion

Application of image vision in patrol robot

The vision localization algorithm (VLA) is usually composed of three core modules: pose tracking, local optimization, and loop closing. The pose tracking module realizes the calculation of the relative motion between camera frames. According to different types of cameras, VLA can be divided into pose tracking modules based on monocular cameras, binocular cameras, and RGBD cameras. Although these three different modules have different information processing methods, they all use the visual odometry (VO) algorithm in calculating the relative motion mode between image frames (Wen et al., 2022). Work (Caramuzza and Fraundorfer, 2011) proposed a parallel tracking and mapping method (PATM), the main idea of which is to simultaneously extract and track feature points and build a map. The tracking process tracks feature points, and the mapping process optimizes the map at the back end, this is the first time to propose the concept of visual slam front-end and back-end. In 2015, Raul Murartal in Spain proposed the orb-slam algorithm based on feature points (Klein and Murray, 2007), which adds a loop closing thread for loop monitoring based on the PATM algorithm, which achieves good results in tracking and mapping and ensures the consistency of the global map and the robot’s posture. At present, the visual slam method based on feature point method occupies the mainstream, which can work when the noise is large and the camera moves fast, but the map is sparse feature points.

In addition to using feature points, the direct method also provides another way of thinking, that is, when extracting feature points from an image, a large amount of useful information in the image is discarded, so the subsequent motion estimation does not use information other than feature points. RGBD cameras can directly obtain the depth information of the surrounding environment and reduce the calculation time of robot positioning and map building. They have been widely used and studied. Among them, the Kinect fusion algorithm developed by Microsoft has a significant impact on the tracking camera and map building (Mur-Artal et al., 2015), but the Kinect fusion method needs GPU acceleration to ensure real-time performance, and cannot ensure the accuracy of pose estimation when there are large differences in images.

Multi-sensor fusion positioning technology of inspection robot in substation

The multi-sensor fusion algorithm is a technology that fuses a variety of input sensor information to improve the output performance of the system. It performs positioning through the fusion of multiple sensors carried by the robot. The advantage is that it improves the robustness of the system and improves the accuracy of output data by combining the differences of sensors to complement each other.

Work (Newcombe et al., 2011) proposed a closed formula to predict the lidar frame line measurement, which makes feature correlation, residual error, and Graphical User Interface (GUI) display possible. With this formula, the extended Kalman filter (EKF) can be directly used to fuse the lidar and IMU data to estimate the full state of the inspection robot. When the Global Navigation Satellite Systems (GNSS) signal is masked, lidar can help reduce the uncertainty of state estimation in a certain direction. Literature (Lynen et al., 2013) proposed a general (MSF-EKF) framework, which can fuse the measurement data of different types of sensors and conduct sensor self-calibration. Work (Wu et al., 2017) integrates IMU odometer and vision, proposes a vision-aided navigation system (VINS) for wheeled robot positioning in view of the situation that there are unobservable motion constraints due to wheel slip during robot rotation, and simulates the unobservable constraints of IMU in combination with the planar motion of the odometer, improving the accuracy of the positioning system. Because the system operation needs to initialize the zero deviation of acceleration and gravity, which is not easy to do on the mobile robot platform, the effect of this algorithm is not ideal (Cecerska-Hery et al., 2022).

Conclusion

As an important part of smart grid, smart substation is the information collection and command execution link of intelligent dispatching center, and the core link of power grid equipment operation and maintenance management. Intelligent diagnosis of equipment faults is of great significance for the construction of smart grids. In particular, condition-based maintenance is a reliable and economical substation maintenance method, and the correct and effective implementation of condition-based maintenance requires advanced condition monitoring technology and accurate condition monitoring data as support. This paper discusses the concept and monitoring objects of the condition monitoring of the secondary equipment of the smart substation; summarizes the research status of the information modeling of the secondary equipment of the smart substation and the analysis and processing methods of the condition monitoring data, and analyzes the various condition monitoring methods used in the smart substation. data sources, advantages and disadvantages, and their respective applicability. As the core of substation status assessment, the substation power inspection robot is based on image vision, its development has shortcomings such as accuracy, visual positioning, and moving speed. The current research obstacles in this field and the prospects for future research are mainly summarized as follows:

a) The current research results of the visual positioning algorithm of inspection robots are rich, but the algorithm will be restricted by many environmental factors in the actual application process. For example, when the camera moves fast or the viewing angle changes greatly, the positioning effect of the algorithm is Poor; when the light intensity is low, the visual localization algorithm does not work properly. These problems are still the difficulties that need to be solved in the research and application-oriented process of visual positioning algorithm;

b) Various theoretical methods of fault diagnosis and fault early warning have been proposed in large numbers, however, potential engineers should develop corresponding products and software systems for substation condition monitoring, which has great research value for practical applications in smart substations;

c) With the rise of big data and cloud computing, it is possible to consider building a cloud platform in substations or dispatch centers in the future to improve the analysis and processing capabilities of substation condition monitoring and other systems. With the construction of cloud platforms and the introduction of data stream mining, the Construct the primary and secondary joint substation condition monitoring system and subsequent advanced applications such as fault diagnosis system, fault early warning system, and condition maintenance decision support system based on it;

d) Facing the application requirements of low cost, high precision, and high stability of the positioning system of the power inspection robot in the substation, future research should be dominated by the method of sensor fusion to obtain stable and accurate positioning results and efficient and stable operation of the inspection robot;

e) Advanced artificial intelligence algorithms should be widely used in the field of identification and positioning of patrol robots. Patrol inspection data should be compared and trend analyzed to timely find potential accidents and fault precursors in power grid operation, to effectively reduce the labor intensity of operation and maintenance personnel, and provide all-round security guarantee for substations;

f) To solve the problem that it is difficult to reliably model the target and background with a single feature in the moving scene of the substation patrol robot, it can be improved from the perspective of feature selection, and to solve the problem of high computing cost of the depth learning method, it can be improved from the perspective of network structure optimization. In order to solve the problems of less prior information of tracked objects and easy deformation of objects in mobile robot tracking, research can be carried out from the perspective of model updating strategy, and a better model updating mechanism can be developed to improve the speed and accuracy of the algorithm.

Author contributions

WZ, AC, and MF: writing the original draft and editing, LY, KC, and XG: discussion of the topic.

Acknowledgments

The authors gratefully acknowledge the support of the first batch (supplementary) science and technology projects of Yunnan Power Grid Co., Ltd. in 2022-Research and Application of the Key Technology of Live Disconnection of the Drain Wire of Disconnector in 110–220 kV Substation Bus Side under Complex Environment (059400KK52220003).

Conflicts of interest

WZ, AC, MF, LY, KC, and XG were employed by transmission branch of Yunnan Power Grid Co., Ltd.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Beaudry, J., and Poirier, S. (2012). “Vehicule teleopere pour inspection visuelle et thermographique duns les postesde transformation. Taiwan, China. Report IREQ-2012-0121.

Google Scholar

Bochkovskiy, A., Wang, C. Y., and Liao, H. Y. M. (2020). YOLOv4: Optimal speed and accuracy of object detection. Québec, QC: Comput. Vis. Pattern Recognit. doi:10.48550/arXiv.2004.10934

CrossRef Full Text | Google Scholar

Budiyono, A. (2013). Principles of GNSS, inertial, and multi-sensor integrated navigation systems. Ind. Robot. 67 (3), 191–192. doi:10.1108/ir.2012.04939caa.011

CrossRef Full Text | Google Scholar

Caramuzza, D., and Fraundorfer, F. (2011). Visual odomdry. Robotics&Automation Mag. IEEE 18 (4), 80. doi:10.1109/MRA.2011.943233

CrossRef Full Text | Google Scholar

Cecerska-Hery, E., Polikowska, A., Serwin, N., Roszak, M., Grygorcewicz, B., Heryc, R., et al. (2022). Importance of oxidative stress in the pathogenesis, diagnosis, and monitoring of patients with neuropsychiatric disorders, a review. Neurochem. Int. 153, 105269. doi:10.1016/j.neuint.2021.105269

PubMed Abstract | CrossRef Full Text | Google Scholar

Chi, Q., Wan, K. H., Yuan, F. X., Li, L. S., Wu, J. F., Li, W. H., et al. (2021). Autonomous localization algorithm of inspection robot based on particle filter and environmental label correction. Grid Analysis Study 49 (4), 101.

Google Scholar

Dong, Y., Wang, D. J., and Wu, J. (2021). MEMS-based SINS/GNSS tightly-coupled navigation aided by time-differenced carrier-phase measurements. J. Chin. Inert. Technol. 29 (4), 451.

Google Scholar

Duan, Z. J., Li, S. B., Hu, J. J., Yang, J., and Wang, Z. (2020). Review of deep learning based object detection methods and their mainstream framework. Laser & Optoelectron. Prog. 57 (12), 1. doi:10.3788/lop57.120005

CrossRef Full Text | Google Scholar

Fang, L. Industrial meter reading for patrol robot of electric power substation. Hangzhou, China: Zhejiang University. https://kns.cnki.net/kcms/detail/detail.aspx?dbcode=CMFD&dbname=CMFD201702&filename=1017169456.nh&uniplatform=NZKPT&v=YZ7C5AjX7qDGs7ZIBXadHpI47pdYcY4FFHLCaCMpkrnU3e4oh9DtHwp7s7fkRUmX.

Google Scholar

Fei, Y. L., Zhang, L. Z., Liu, Y. X., Xu, W. G., and Ruan, L. X. (2016). Study of automatic identification method of equipment action state. Based Multi-source Inf. Fusion 35 (10), 65–68. doi:10.19585/j.zjdl.2016.10.014

CrossRef Full Text | Google Scholar

Frosi, M., and Matteucci, M. (2022). ART-SLAM: Accurate real-time 6DoF LiDAR SLAM. IEEE Robot. Autom. Lett. 7 (2), 2692–2699. doi:10.1109/LRA.2022.3144795

CrossRef Full Text | Google Scholar

Gao, B., Wu, D., Yang, Z. H., and Zheng, T. (2022). Research on condition maintenance technology of substation equipment based on power big data. Microcomput. Appl. 38 (4), 84.

Google Scholar

Guo, R., Li, B. Q., Sun, Y. T., and Han, L. (2009). “A patrol robot for electric power substation,” in International Conference on Mechatronics and Automation, Changchun, China, 9-12 August 2009. doi:10.1109/ICMA.2009.5245105

CrossRef Full Text | Google Scholar

Ju, Z. L., Yang, B., Sun, H. F., Huang, X. Y., Pu, Lu., Zhao, X. F., et al. (2020). Cooperative path planning of multi-robot system for power grid intelligent inspection. Power Grid Technol. Study 48 (6), 92–97.

Google Scholar

Klein, G., and Murray, D. (2007). “Parallel tracking and mapping for small AR workspaces,” in IEEE and ACM international symposium on mixed and augmented reality, Nara, Japan, 13-16 November 2007, 225–234. doi:10.1109/ISMAR.2007.4538852

CrossRef Full Text | Google Scholar

Liu, J. M., and Meng, W. H. (2020). Review on single-stage object detection algorithm based on deep learning. AERO Weapon. 317 (3), 48–57. doi:10.12132/ISSN.1673-5048.2019.0100

CrossRef Full Text | Google Scholar

Lu, J., He, J. X., Li, Z., and Zhou, Y. R. (2020). A survey of target detection based on deep learning. Electron. Opt. Control 263 (5), 60. doi:10.3969/j.issn.1671-637X.2020.05.012

CrossRef Full Text | Google Scholar

Luo, J., Fan, Y, K., Jiang, P., He, Z., Xu, P., Li, X., et al. (2021). Vehicle platform attitude estimation method based on adaptive Kalman filter and sliding window least squares. Meas. Sci. Technol. 32 (3), 035007–035009. doi:10.10.1088/1361-6501/abc5f8

CrossRef Full Text | Google Scholar

Lynen, S., Achtelik, M. W., Weiss, S., Chli, M., and Siegwart, R. (2013). “A robust and modular multi-sensor funsion approach applied to may navigation,” in IEEE/RSJ International Conference on Intelligent Robots and Systems, Tokyo, Japan, 3-07 November 2013, 3923–3929. doi:10.1109/IROS.2013.6696917

CrossRef Full Text | Google Scholar

Mur-Artal, R., Montiel, J. M. M., and Tardos, J. D. (2015). ORB-SLAM:A versatile and accurate monocular SLAM system. IEEE Trans. Robot. 31 (5), 1147–1163. doi:10.1109/TRO.2015.2463671

CrossRef Full Text | Google Scholar

Newcombe, R. A., lzadi, S., Hilliges, O., Molyneaux, D., Kim, D., Davison, A., et al. (2011). “KinectFusion:Rea1-time dense surface mapping and tracking,” in 10th IEEE International Symposium on Mixed and Augmented Reality, Basel, Switzerland, 26-29 October 2011, 127–136. doi:10.1109/ISMAR.2011.6092378

CrossRef Full Text | Google Scholar

Pan, Z. X., and Zhang, W. S. (2013). Research and application of information fusion technology in smart substations. Telecommun. Electr. Power Syst. 34 (243), 36. doi:10.3969/j.issn.1005-7641.2013.01.008

CrossRef Full Text | Google Scholar

Purkait, P., Zach, C., and Zhao, C. (2015). SPP-Net: Deep absolute pose regression with synthetic views, arXiv:1712.03452. Comput. Sci.. doi:10.48550/arXiv.1712.03452

CrossRef Full Text | Google Scholar

Redmon, J., Diwala, S., Girshick, R., and Farhadi, A. (2016). “You only look once: Unified, real-time object detection,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, 27-30 June 2016, 779–788. doi:10.1109/CVPR.2016.91

CrossRef Full Text | Google Scholar

Redmon, J., and Farhadi, A. (2017). “YOLO9000: Better, faster, stronger,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, 21-26 July 2017, 7263–7271. doi:10.1109/CVPR.2017.690

CrossRef Full Text | Google Scholar

Redmon, J., and Farhadi, A. (2018). YOLOv3: An incremental improvement. Computer vision and pattern recognition. Salt Lake City, Utah: IEEE, 3523.

Google Scholar

Salvucci, V., Baratcart, T., and Koseki, T. (2014). Increasing isotropy of intrinsic compliance in robot arms through biarticular structure. IFAC Proc. Vol. 47 (3), 332–337. doi:10.3182/20140824-6-ZA-1003.02731

CrossRef Full Text | Google Scholar

Shaoqing, R., Kaiming, H., Girshick, R., and Sun, J. (2015). Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 39 (6), 1137–1149. doi:10.1109/TPAMI.2016.2577031

CrossRef Full Text | Google Scholar

Shin, H., Song, Y., and Kong, P. Y. (2021). Robust online overhead transmission line monitoring with cost efficiency in smart power grid. IEEE Access 9 (9), 86449–86459. doi:10.1109/ACCESS.2021.3088843

CrossRef Full Text | Google Scholar

Wang, J. Y., Zhao, Q. G., and Ye, Q. (2021). Design of remote monitoring system for substation DC power supply under the background of big data. J. Phys. Conf. Ser. 2037, 012005. doi:10.1088/1742-6596/2037/1/012005

CrossRef Full Text | Google Scholar

Wen, G., Xia, S. P. C., Zhou, F. R., Zhang, H., Pan, H., Gao, J., et al. (2022). Multi-source fusion robust positioning and high-fidelity simulation method for inspection robots. Power Grid Analysis Study 50 (4), 110.

Google Scholar

Wu, K. J., Guo, C. X., Georgiou, G., and Roumeliotis, S. (2017). “VINS on wheels,” in IEEE International Conference on Robotics and Automation (ICRA), Singapore, 29 May 2017-03 June 2017, 5155–5162. doi:10.1109/ICRA.2017.7989603

CrossRef Full Text | Google Scholar

Xu, M. H., Guo, Y., Zhang, D. W., Gao, R., Shen, Y. B., and Han, S. W. (2021). Research on high voltage online monitoring system for dielectric loss of capacitive equipment in substation. IOP Conf. Ser. Earth Environ. Sci. 769 (4), 042025. doi:10.1088/1755-1315/769/4/042025

CrossRef Full Text | Google Scholar

Xu, Q. (2019). Research on substation intelligent operation and maintenance monitoring platform based on multi-source information fusion. Mod. Archit. Electr. 3 (10), 59–62. doi:10.16618/j.cnki.1674-8417.2019.03.013

CrossRef Full Text | Google Scholar

Yang, C. K., Wu, J. F., Zhao, J. S., Zhang, X. L., and Mu, J. (2021). Research on the architecture of integrated platform of intelligent substation auxiliary monitoring system. J. Phys. Conf. Ser. 2095 (1), 012010. doi:10.1088/1742-6596/2095/1/012010

CrossRef Full Text | Google Scholar

Zhang, S. W. (2019). Research on autonomous navigation method of substation patrol robot based on fusion of machine vision and radar data. Wuhan, China: Huazhong University of Science and Technology. Available at: https://kns.cnki.net/kcms/detail/detail.aspx?dbcode=CMFD&dbname=CMFD202001&filename=1019918899.nh&uniplatform=NZKPT&v=gA9KwnaLT32j5gvO8KF93-EaXvhTWUL2AKsnrhvbuRtJOr5AckF50zKDHg0qkMEu.

Google Scholar

Zhang, X., Lin, J., Li, Q. Q., Liu, T., and Fang, Z. X. (2021). Continuous indoor visual localization using a perceptual Hash algorithm and spatial constraint. Acta Geod. Cartogr. Sinica 50 (12), 1639–1649. doi:10.11947/j.AGCS.2021.202000286

CrossRef Full Text | Google Scholar

Zheng, R. L., Niu, B., She, C. Y., and Gao, D. M. (2022). Research on improvement of substation monitoring capability based on AI algorithm. J. Phys. Conf. Ser. 2290 (1), 012016. doi:10.1088/1742-6596/2290/1/012016

CrossRef Full Text | Google Scholar

Nomenclature

CBM condition-based maintenance

EKF extended Kalman filter

FKF Federated Kalman filtering

GNSS Global Navigation Satellite Systems

GUI Graphical User Interface

IMU inertial measurement unit

MSF-EKF multi-sensor extended Kalman filter

PATM parallel tracking and mapping method

PB petabyte

PTZ Pan/Tilt/Zoom

RFID radio frequency identification

VINS vision-aided navigation system

VLA vision localization algorithm

VO visual odometry