Gabor-YOLONet: A lightweight and efficient detection network for low-voltage power lines from unmanned aerial vehicle images

Maintaining and monitoring low-voltage overhead power lines are of great importance because such lines are the key link between the power system and low-voltage power users. At present, few networks can be detected accurately on intelligent edge identification devices because of the complex backgrounds and limited characteristics in unmanned aerial vehicle images as well as the low computing abilities of hardware. In order to give consideration to accuracy and speed, a novel power line detection method was proposed, denoted by Gabor-YOLONet, used for intelligent edge identification devices available to UAV. Unlike existing methods, the proposed method uses the Gabor algorithm to extract foreground of power lines from cluttered backgrounds automatically and predict power lines and their auxiliary targets such as insulators in the foreground scene. In addition, a new inference method was introduced, which can summarize the average location and orientation of auxiliary targets by clustering to verify the rationality of the predicted results for power lines. The experiment results showed that the proposed method had the higher accuracy and consumed less computing resources; compared with other methods, the mAP of identification for power lines was 86.6% and the running time was only 25 ms, with excellent performance on intelligent edge devices.

1 Introduction

A low-voltage distribution network is the link between the power system and low-voltage power users. Low-voltage overhead power line inspection plays an essential role in the daily maintenance of power transmission systems (Zhang et al., 2021). In addition, the basic information, for example, the line direction and circuit topology of a low-voltage distribution network, is seriously missing, which makes it difficult to apply almost any modern information-based inspection method to low-voltage distribution networks (Gao et al., 2020). Statistics show that a vast majority of power outage accidents and power-related public safety accidents encountered by power users occur in the low-voltage distribution network (Lu et al., 2016). Therefore, many studies have been carried out to improve the informatization degree of a low-voltage distribution network and ensure the reliability of power supply, among which the vision-based unmanned aerial vehicle (UAV) inspection has long been regarded as one of the essential work (Oktay et al., 2018a; Oktay et al., 2018b; Kose and Oktay, 2020; Şahin et al., 2022).

Low-voltage distribution lines are divided into buried cables and overhead lines, and our study involves low-voltage overhead lines. The state-of-the-art methods used in power line detection are divided into three directions:

1. Power line detection based on a laser point cloud. For example, Cheng et al. (2014) used vehicle-borne LiDAR data to extract power lines. This method generally uses a laser to scan power lines, which is suitable for high-voltage transmission circuits. However, it is not suitable for low-voltage overhead lines because high-voltage transmission lines have special transmission corridors while low-voltage overhead lines do not. The complex background of low-voltage overhead lines, especially other stray lines, will cause serious occlusion and misjudgment of laser scanning. In addition, the problems associated with low-voltage overhead lines are prohibitively expensive to use with laser.

2. Power line detection based on binocular vision. For example, Cheng et al. (2014) and Shuai et al. (2017) proposed a binocular vision-based method for power lines extraction and distance measurement. This type of methods can obtain the distance information of power lines through visual matching, but this does not improve the accuracy of power lines, and the calculation amount of 3D convolution is much larger than that of 2D convolution, which makes it difficult to ensure the efficiency of power line extraction. In addition, the error of binocular ranging is too large, especially when the distance is greater than 10 m.

3. A single camera–based visible light power line identification method. A number of studies based on RGB images have been carried out (Jiang et al., 2017; Zhang et al., 2017). Power line extraction was regarded as the issue of line edge extraction (Zhang et al., 2022). After an edge detection algorithm was adopted to obtain the edge graph, Hough transform was used to extract line segments on the edge graph. Zhang et al. (2014) took the direction of power lines as PK and used Hough transform to extract power lines to group and fit power lines fragments. Pouliot et al. (2015) used the LSD algorithm to extract line features, pole and tower features, and associated features of the lines and towers in aerial images as model input to distinguish power lines from non-power lines. Dai et al. (2022) proposed a CODNet network to extract features of power lines from cluttered backgrounds automatically and predict centers and orientations of power lines in the scene simultaneously, as a guide for the automatic navigation of UAVs. Zhang et al. (2021) proposed an ultra-lightweight and ultra-fast abnormal target identification network for transmission line, but it could not extract the power line. Zhao et al. (2022) proposed a method combining line detection and semantic segmentation for power line extraction from unmanned aerial vehicle images; however, this method was more complicated and did not have real-time performance. Zhang et al. (2019) proposed a method with convolutional features and structured constraints, and two algorithms were used to extract feature fusion for feature fusion, although such a method is also time consuming and cannot be used for real-time UAV power line extraction. This type of methods introduced deep learning models after 2018, such as Fast-PLDN proposed by Zhu et al. (2022) and the asemantic segmentation model for power line detection. These methods perform pixel-level semantic segmentation of power lines. In recent years, related research has focused on the detection of ultra-high voltage transmission lines and railway power lines, and there are few studies on the identification of low-voltage overhead lines.

It can be seen that related research in recent years has focused on the detection of ultra-high voltage transmission lines and railway power lines, and there are few studies on the identification of low-voltage overhead lines. Because of the complex background of low-voltage overhead lines and the lack of dedicated distribution channels, laser scanning and binocular cameras are not suitable for low-voltage overhead lines. Also, no scholars have proposed a low-voltage overhead power line detection algorithm that can be directly mounted on low-power computers, and there are few related studies. Therefore, low-voltage power line detection from aerial images remains a challenging problem.

To this end, a new power line detection method was proposed to address the aforementioned challenges for UAV navigations. In order to accurately capture the small and inconspicuous overhead power lines in the low-voltage distribution network with cluttered background, power lines extraction suitable for real-time aerial photography by UAVs was investigated, and a Gabor-YOLO algorithm based on the contextual information was proposed in our study, including the following four parts:

1. Aiming at the cluttered background of low-voltage distribution network, an adaptive foreground segmentation algorithm based on the Gabor operator was proposed, employing Gabor features to detect edge and contour information, filter out most of the meaningless background noises, and provide reliable candidate regions for the next-stage algorithm.

2. Aiming at the characteristics of narrow, long, slender, and inconspicuous features of overhead lines, an improved YOLO network (Zhang et al., 2014) model based on the attention mechanism was proposed, which was used to perform accurate power lines identification and positioning and provide contextual information for power lines in the foreground area of the image.

3. In order to reduce the false positive of the algorithm, an inference algorithm based on contextual information was proposed which was used to infer and discriminate power lines through the set knowledge template after performing K-means clustering and IOU calculation according to the power lines and contextual information, which improves the overall reliability and practicability of the algorithm in this study.

4. The proposed method takes accuracy and speed into account, and it can run in real-time and be easily applied to intelligent edge devices, such as Nvidia Jetson Xavier NX.

2 Related works

The algorithm in this study mainly involves the principle of edge detection, Gabor transform, and residual convolution neural network, which are described as follows:

2.1 Edge detection

The so-called edge refers to the collection of pixels with sharp changes in the gray scale of surrounding pixels, which is the most basic feature of an image (Chen et al., 2020). Edges exist between the target, background, and region, so it is the most important basis for image segmentation. The main tool of edge detection is the edge detection template, which is used to subtract the gray value of the right adjacent point from the gray value of the left adjacent point as the gray value of the point. Common edge detection templates include the Laplacian operator, Sobel operator, and Gabor operator. The Gabor operator was used in the study.

2.2 Gabor Transform

Gabor transform is a windowed Fourier transform, and the Gabor function can extract relevant features in different scales and directions in the frequency domain (Jiang et al., 2011). The main features of Gabor transform include its ability to fully highlight the characteristics of certain aspects of the problem through transformation, localize the time (space) frequency analysis, and gradually refine the signal (function) through scaling and translation operations, with arbitrary details of the signal, for solving the difficulty of Fourier transform. The main disadvantages of Gabor transform include the basis function unable to be an orthogonal system, and a non-orthogonal redundant basis required to be used in a signal analysis or numerical calculation, resulting in a relatively large amount of calculation and storage.

2.3 Residual convolution neural network

A convolutional neural network is a kind of feedforward neural network with a deep structure and convolution computation. The parameter sharing of convolutional kernel within the hidden layer and the sparsity of interlayer connections enable a convolutional neural network to calculate complex features with a small amount of computation (Redmon et al., 2016). A residual convolutional neural network is a convolutional neural network with residual blocks proposed by Kaiming He. The residual convolutional neural network is a necessary structure for deep neural networks because it can greatly alleviate the problems of gradient disappearance and explosion. The key structure of the residual convolutional neural network is a skipping connection structure. In the residual convolutional neural network, assuming the residual block input is X, the output is H(x) and the forward branch is F(x), then:

$H\left(x\right)=F\left(x\right)+x,(1)$

where F (x) represents the forward branch, and x represents the skipped connection branch. It can be seen that even if the gradient of the weight layer disappears, the marked x can still be transferred back to the earlier layer, thus avoiding the disappearance of the overall gradient.

2.4 R-CNN

R-CNN is the combination of region proposals and CNN. It is a pioneering work that introduces CNN into the field of target detection (Girshick et al., 2010). It has important epoch-making significance and greatly improves the effect of target detection. The R-CNN series of algorithms are in the order of their development. It is divided into R-CNN, Fast RCNN, and Faster RCNN in turn. The RCNN first scans the input image with the selective search algorithm to extract candidate boxes, then scales all candidate boxes to a fixed pixel size through normalization, and then inputs them into the convolutional neural network to unify the length of the feature vector. The standard is then extracted, the feature vector in each candidate region of the SVM binary classifier is provided according to the target category to classify the corresponding number, and the target position information is obtained through regression to complete the target detection. In the RCNN algorithm, the extraction of features and the classification decision are carried out in series, and the SVM classifier is used for classification, which leads to the disadvantage of a large amount of calculation. Girshick (2015) proposed Fast RCNN on the basis of RCNN, which directly inputs the image into the convolution, and after passing through the ROI pooling layer, the generated region of interest is sent to the fully connected layer and then classification of objects is done with the help of SoftMax classifier. However, Faster RCNN (Ren et al., 2015) uses the regional candidate network instead of the selective search method and obtains the adjusted candidate box by setting anchor boxes of different scales and combining with the convolutional neural network. The network combines the extraction of target feature information with the realization of target classification, and end-to-end training is achieved for the first time.

3 Methods and models

The Gabor-YOLO algorithm in this study is composed of an adaptive foreground extraction module based on the Gabor operator, an improved YOLO network based on attention mechanism, and a reasoning module based on contextual information. The overall framework of the Gabor-YOLO algorithm is shown in Figure 1. The UAV image was divided into the image input foreground extraction module, in which the image was first preprocessed by gray scale and Gaussian filtering, then improved after performing feature extraction with the Gabor operator, and finally the foreground area was obtained in the image and input to the next module. In the improved YOLO network module, the power lines and auxiliary targets were located and identified, and the results were input to the next module. In the inference module, K-means clustering was performed on the coordinates of all auxiliary targets, and after the power distribution channel was obtained, and the IOU calculation was performed with the power line, to obtain the final power lines extraction results.

FIGURE 1

FIGURE 1. Structure of the model.

3.1 Foreground extraction algorithm

In the images of a low-voltage distribution network, there are a large number of background pixels having a great impact on the performance of the edge detection algorithm, so it is not appropriate to directly use the conventional edge detection method to extract power lines. However, the edge detection operator has the advantages of simplicity and efficiency, so it can be used to detect the edge of the image first and establish the candidate pool of lines pixel points, which can not only improve the accuracy of power lines extraction but also greatly reduce the computation of the subsequent convolutional neural network.

In this study, an adaptive foreground extraction algorithm was proposed based on edge detection, which was used to filter out background interference and obtain the foreground rapidly. The algorithm structure is shown in Figure 2.

FIGURE 2

FIGURE 2. Obtaining foreground areas from images.

The initial image collected by the UAV was an RGB color model, and the gray scale processing was performed on the target image to reduce the amount of data. The gray scale calculation formula is as follows:

$Gray\left(i,j\right)=0.299\times R\left(i,j\right)+0.578\times G\left(i,j\right)+0.114\times B\left(i,j\right),(2)$

where $\left(i,j\right)$is the horizontal and vertical coordinates of pixels in the image; $Gray$represents the gray value after gray processing; $R$represents the gray value of the red component; $G$ represents the gray value of green component; and $B$represents the gray value of the blue component.

In the process of UAV shooting and transmission, there were a lot of noises in the image, which weakened the details of the image. In order to eliminate its influence on edge extraction as much as possible, the image was processed by Gaussian filtering. The calculation formula of Gaussian filtering is as follows:

$C\left(x,y\right)={\displaystyle \sum }_{v=-c}^c{\displaystyle \sum }_{u=-c}^{c}I\left(x+u,y+v\right)\frac{1}{2\pi {\delta }^2}{e}^{-\frac{u^2+v^2}{2{\delta }^2}},(3)$

where $x,y$is the horizontal and vertical coordinates of pixels in the image; $C$is half of the length/width of the filtering template; $\sigma$is the standard deviation; $C\left(x,y\right)$ is the result of Gaussian filtering; and $x,y$are the horizontal and vertical coordinates of pixels in an image.

The core principle of the Gabor operator is the Gabor transform. In this study, the imaginary part of a two-dimensional Gabor function was used for feature extraction, with the expression of the two-dimensional Gabor function shown in Formula (Eq. 4), as follows:

$g_{\lambda }\left(x,y\right)=\frac{1}{2\pi {\lambda }^2}\exp \left(\frac{-\left(x^2+y^2\right)}{2{\lambda }^2}\right),(4)$

where $\theta$is the direction angle; $\lambda$is the width parameter of the Gaussian function; $f$ is the center frequency of the filter bandwidth, and $x,y$are the horizontal and vertical coordinates of pixels in an image.

Considering that Gabor filter is a complex function, its imaginary part is the following:

$I_{\phi }\left(x,y\right)=g_{\lambda }\left(x,y\right)\sin \left(2\pi f\left(x\cos \theta +y\sin \theta \right)\right).(5)$

In actual operation, the input is a discrete value, so discretization is performed on equation (6). Assume $D\left(x,y\right)$ is the digital image of matrix $m\times n$, so for the field window of size $W\times W\left(W=2k+1\right)$ the imaginary part of discretized convolution of $D\left(x,y\right)$ with $G_{\phi }\left(x,y\right)$ is as follows:

$G_I\left(x,y|\phi \right)={\displaystyle \sum }_{n=-k}^{\kappa }{\displaystyle \sum }_{m=-k}^{k}D\left(x+n,y+m\right)I_{\phi }\left(n,m\right).(6)$

The spectral energy at any point $\left(x,y\right)$ in the window $W\times W$ is defined as follows:

$E_{\phi }\left(x,y\right)={\left(e_{i,j}^{\phi }\right)}_{P\times Q}=\frac{1}{W^2}\left(G_R^2\left(x,y|\phi \right)+G_I^2\left(x,y|\phi \right).\right)(7)$

In this study, the spectral energy function $E_{\phi }\left(x,y\right)$ was used as the response of input image to the Gabor filter. A total of eight groups of Gabor filters were designed according to different wavelengths and bandwidths. Each Gabor filter only responded to its $E_{\phi }\left(x,y\right)$ corresponding frequency and bandwidths while the energy of other textures was suppressed, so as to extract and analyze texture features. Then the edge detection images of multiple groups of Gabor filters were fused and the foreground segmentation was performed.

3.2 Gabor-YOLONet: Improved YOLO network based on gabor operator

Most of the backgrounds have been filtered out in the image of the foreground region given by the Gabor algorithm. On this basis, the improved YOLO network in this section can identify power lines and auxiliary objects, for example, insulators and C-clamps. Combined with the fact that the power lines itself involves less high-order semantic features and the speed requirement of power lines extraction, the lightweight backbone was adopted in the study. At the same time, for improving the adaptability of lightweight backbone to low-voltage overhead lines, on the one hand, a convolutional block attention module was added that combines channel attention and spatial attention, and the feature fusion branch was adjusted to identify. The overall network structure of the algorithm is shown in Figure 3.

FIGURE 3

FIGURE 3. Improved YOLO Network based on the Gabor operator.

A convolution block attention model combining channel attention and spatial attention was added to backbone residual blocks, to further highlight the prominent feature channels in feature graphs and suppress background feature channels.

In channel attention mechanism, global average pooling and maximum pooling were adopted to extract channel weight information, for avoiding information omission caused by only using average pooling. The calculation formula of channel attention mechanism is shown in formula (Eq. 8):

$\begin{array}{l}{M}_C\left(F\right)=\sigma \left(MLP\left(AvgPool\left(F\right)\right)+MLP\left(MaxPool\left(F\right)\right)\right),\\ =\sigma \left(W_1\left(W_0\left(F_{avg}^c\right)\right)+W_1\left(W_0\left(F_{\max }^c\right)\right)\right),\end{array}(8)$

where $\sigma \left(\cdot \right)$represents sigmoid function; $MLP$represents the shared network in the module, which consists of the hidden layer and multilayer perceptron. The activation size of the hidden layer is set to $R^{\raisebox{1ex}{$C$}\!\left/ \!\raisebox{-1ex}{$r$}\right.\times 1\times 1}$. In the multilayer perceptron, its $W_0\text{and}{W}_1$ are set to $R^{\raisebox{1ex}{$C$}\!\left/ \!\raisebox{-1ex}{$r$}\right.\times C}$ and $W_1\in R^{C\times \raisebox{1ex}{$C$}\!\left/ \!\raisebox{-1ex}{$r$}\right.}$respectively. R is set to 16. $AvgPool\left(\cdot \right)$and $MaxPool\left(\cdot \right)$ represent the module performing average pooling and maximum pooling of feature map spatial information, respectively. $F_{avg}^c$and $F_{\max }^c$ represent the global average pooling and maximum average pooling operations of channel attention mechanism, respectively.

Considering that the channel attention mechanism cannot obtain the image position information well, the spatial attention mechanism was introduced to pay attention to the spatial region, and the corresponding feature images of each channel were calculated and screened. The calculation formula of spatial attention mechanism is shown in Eq. 9:

$M_S\left(F\right)=\sigma \left(f^{7\times 7}\left(\left[AvgPool\left(F\right);MaxPool\left(F\right)\right]\right)\right)=\sigma \left(f^{7\times 7}\left(F_{avg}^c;F_{\max }^c\right)\right),(9)$

where $f^{7\times 7}$denotes convolution calculation with a size of 7 × 7; $F_{avg}^c$and $F_{\max }^c$ represent global average pooling and maximum average pooling operations of spatial attention mechanism, respectively.

Aiming at the long and narrow linear structure of power lines in the image, a double feature layer extraction structure and a pyramid multi-scale fusion module are adopted. High-level semantic information can help to accurately detect generalized objects such as power distribution corridors, whereas low-level geometric details can accurately detect small objects such as power lines. Therefore, in the convolutional neural network, it is necessary to fuse features of different scales to achieve the best detection effect. The feature fusion model in this study is different from the conventional model. The feature map output by the first residual module is used as a low-order geometric feature map, and the feature map output by the third residual module is used as a high-order semantic feature map.

An inference module was proposed, with the flowchart shown in Figure 4. The inference module was applied to screen the power lines after detection of power lines and auxiliary objects. Long and narrow linear structures in the images are very common in actual scenes, such as ropes and miscellaneous lines (Zhang et al., 2014). The image features are very similar to power lines, which is easy to cause false detection. It is generally believed that only the power lines in the distribution corridor area are overhead power lines, and insulators, C-clamps and even transformers are equipment specific to the distribution corridor. Therefore, these unique devices are detected together and used to assist extraction of power lines.

FIGURE 4

FIGURE 4. Inference module.

The process of the inference module are as follows:

First, the center point is obtained of all auxiliary targets according to the prediction box given by the improved YOLO network.

Second, the K-means clustering algorithm is performed on the center point coordinates of all auxiliary objects, and the auxiliary objects are divided into K clusters, and the relative distance of the targets in the cluster is as small as possible, and the distance between the two clusters is as large as possible. If the cluster is divided into $\left(C_1,C_2,\dots ,C_n\right)$, the purpose of clustering algorithm is to minimize the square error E, which is to minimize Eq. 10:

$E={{\displaystyle \sum }_{i=1}^k{\displaystyle \sum }_{x\in C_i}\Vert x-\frac{1}{\left|C_i\right|}{\displaystyle \sum }_{x\in C_i}x\Vert }_2^2.(10)$

In the study, each cluster is the tower area, which is the stress point of the power line, and the area in the middle of the two towers is the distribution corridor. After determining the power distribution corridor area, the Intersections over Union (IOU) of power lines and power distribution corridor area were calculated, to exclude the power lines outside the power distribution corridor area. The IOUs of power lines and power distribution channel were calculated, to exclude the power lines outside the power distribution channel. If there is only one single clustering result, IOU calculation can be carried out based on the tower area and power lines, to complete the automatic power lines extraction of the algorithm. Suppose the area of the tower is $S_1$and the area of the power line is $S_2$, then

$IOU=\frac{S_1\cap S_2}{S_1\cup S_2}.(11)$

The specific values of $S_1$and $S_2$ are calculated according to the coordinates of the tower and the power line in the images.

4 Experiments and results

In order to verify the advantages and effectiveness of the algorithm in our study, actual aerial images of a real low-voltage overhead lines in Wuhan, Hubei, China were used to conduct the simulation verification experiment, with the following software and hardware platforms: Intel Core i5-10400F@2.90GHz×; 6 CPUs, NVIDIA GeForce RTX 2060, Ubuntu 16.04LTS operating system, Pytorch deep learning framework, and Nvidia Jetson Xavier NX intelligent edge device.

4.1 Dataset

The experimental data sets used in this study were collected by UAV from the actual low-voltage distribution network of a village in Wuhan, Hubei, China. As shown in Figure 5, compared with high-voltage transmission lines, overhead distribution lines of low-voltage distribution network travel in streets and residential buildings, with very serious background interference and shielding. These are typical small targets with very weak features of lines and insulators.

FIGURE 5

FIGURE 5. Images of low-voltage distribution network transmission lines.

Labeling tools was used to label images of low-voltage overhead distribution lines, including power lines, insulators, C-clamp, power towers and transformers. A total of 1,200 images were labeled. In order to increase the diversity of data samples and improve the generalization ability of network model, random rotation and pruning were carried out on the collected data sets, to expand the sample data to 2,400 pieces. 2000 images were selected as training sets and the remaining 400 images were used as test sets. The information on the dataset is shown in Table 1. According to the table, there are a total of 1,176 images including power lines, in which a total of 3,304 power lines are labeled.

TABLE 1

TABLE 1. Dataset information.

4.2 Model training

In the process of model training, the image was input into the foreground extraction module to obtain the foreground image, and the image size was uniformly adjusted to 608 × 608. Adam optimizer was adopted for stochastic gradient descent, the initial learning rate was set to 0.01 and adopted cosine annealing learning rate. In order to prevent over-fitting of the model, the weight attenuation regularization was set to 0.0005, the batch size to 64, and the iteration epoch to 300.

To achieve better training effect, transfer learning strategy and staged training method were adopted. First, after fully pre-training the convolutional neural network, connect it with the foreground extraction module, freeze the parameters of the foreground extraction module and backbone, and perform 50 rounds of model iterations, then unfreeze the parameters of the foreground extraction module and backbone, and iteratively train together until 300 rounds to get the optimal model weights.

The loss convergence curve and mean Average Precision (mAP) curve of the network model during training are shown in Figure 6. The abscissa represents the iterations of the network model, and the ordinate represents the loss value or mAP value in the training process. It can be seen from Figure (a) that the initial loss value of the model in this study was 29.93 (point A in the figure), decreased to about 6.10 (point B in the figure) in the first 50 rounds of iteration, and the loss value finally decreased to about 4.13 (point C in the figure). It can be seen that the detection model proposed in this study has achieved good training effect. As can be seen from Figure (b), the mAP value of the algorithm in this study has risen to 0.6 (point A in the figure) in the first 50 rounds, and finally stabilized at around 0.9 (point B in the figure), achieving high recognition accuracy.

FIGURE 6

FIGURE 6. Loss and mAP Curves.

4.3 Model results

The mAP values of power lines and auxiliary targets of the proposed algorithm are shown in Table 2. As seen from Table 2, the average accuracy mAP of the proposed algorithm for power lines can reach 93.4%, the average accuracy of auxiliary objects such as insulators is satisfactory, and the overall average accuracy is 86.6%, indicating that the proposed algorithm has the advantage of high accuracy. The output results of each stage of the algorithm in this study are shown in Figure 7, in which (a) is the original image taken by UAV, (b) is pretreated after image gray scale and Gaussian filter, (c) is the characteristic figure extracted with Gabor operator, (d) is the fused character figure of different Gabor features, (e) is the foreground figure, and (f) is the final results of the model. The final results are shown in Table 3.

TABLE 2

TABLE 2. MAP value of power lines and auxiliary target.

FIGURE 7

FIGURE 7. Results of each step in our model.

TABLE 3

TABLE 3. Model’s output.

As shown in Table 3, foreground image accounts for 27.4% of the original image, which greatly contributes to promoting the accuracy and the speed of the whole algorithm. The improved neural network model accurately completed the power lines and auxiliary objects’ recognition, the inference module has succeeded in selecting the true low-voltage distribution overhead line.

To verify the effectiveness of the method proposed in this study, ablation experiments were performed for recognition of power lines. A module of the algorithm was removed in turn, the experimental results were compared, and the overall mAP value comparison results were obtained, as shown in Table 4. It can be seen from Table 4 that the mAP of the complete algorithm is 93.4%. When the foreground extraction module is removed, the algorithm fails to remove a large amount of background noises, and the mAP is reduced to 77.1%. After replacing the improved backbone into the common DSPDarknet, the average accuracy is reduced to 78.8%, which verifies the effectiveness of the improved schemes such as attention mechanism and feature extraction branch proposed in this study for backbone. When the inference module is removed, the false positives of the algorithm increase significantly, and the mAP decreases to 89.8%.

TABLE 4

TABLE 4. Results of ablation experiment.

4.4 Comparison and analysis

To further verify the advantages of our method, the method was compared with three representative object detection methods, namely Faster RCNN, YOLOv3, and YOLOV3-tiny, in terms of accuracy and recognition speed for power lines recognition. The same data samples and parameters for training were used for all the object detection methods, and the detection results are shown in Table 5.

TABLE 5

TABLE 5. Results of contract experiment.

According to the comparison results in Table 5, our method has the highest mAP value (93.4%). Thanks to the rapid-speed of Gabor transform, the recognition speed of our method can reach 40 frames/s, only slower than that of YOLOV3-TINY, which can fully guarantee the real-time performance of the algorithm and is suitable for UAV. Yolov3-tiny has significantly higher detection speed than others, but its mAP value is the lowest, leading to a large number of missed detections, especially for power lines and insulators. Compared with YOLOV3-TINY, YOLOV3’s recognition speed is greatly reduced, but the improvement of mAP value is limited, which also confirms the view in this study that power line extraction is not sensitive to higher-order semantic features. Faster RCNN is a two-stage recognition algorithm, and its mAP value is significantly higher than YOLOV3 algorithm, but its detection speed is far slower than other algorithms, so it is obviously not suitable for the UAV.

The comparison is shown in Figure 8. Our method can accurately identify the power distribution corridor and extract the power lines in the power distribution corridor, with the highest detection accuracy. The YOLOv3-tiny algorithm fails to detect a large number of power lines and insulators, and also misidentifies stray lines and pipes outside the power distribution corridor as power lines. Compared with YOLOV3-tiny, YOLOv3 algorithm adds rich high-order semantic features, but false detections and missed detections also exist. Some power lines and insulators are not detected, but stray lines outside the distribution channel and white road dividing lines are mistakenly identified as power lines.

FIGURE 8

FIGURE 8. Results of various algorithms.

4.5 Running on low-power computers

In order to verify the lightweight and practicality of the method in this paper, it is necessary to apply Gabor-YOLONet to a low-power computer. Therefore, we made a new development and successfully applied Gabor-YOLONet to low-power terminals Nvidia Jetson Xavier NX and Rockchip RK3399pro. We adopt a multi-rotor drone, which forms a drone system with the drone nest. The effect of the proposed algorithm running on Nvidia Jetson Xavier NX and RockchipRK3399pro. On Nvidia Jetson Xavier NX, the frame used is TensorRT, the running speed is 29.4, and the accuracy of power line recognition is 82.3%. The average power is 9.8w.On Rockchip RK3399pro, the frame used is RKNN, the running speed is 19.1, the accuracy of power line recognition is 79.6%, and the average power of the device is 4.3. It is worth noting that Our algorithm is no longer based on PyTorch in the low-power computers. The program running on Nvidia Jetson Xavier NX is accelerated by TensorRT, and the program running on Rockchip RK3399pro is accelerated by RKNN. Because the NPU of RK3399PRO does not support Pytorch, we have carried out a series of work on the neural network model of this paper, such as model acceleration and quantization, operator fusion, operator replacement and model transformation. In summary, the proposed method is hardware friendly It can run in real time on low-power computers and has strong practicability.

5 Conclusion

In order to accurately capture the small and inconspicuous overhead power lines in the low-voltage distribution network with cluttered background, a Gabor-YOLO algorithm based on contextual information was proposed in this study. The improved Gabor operator was used to filter out the environmental noise for foreground segmentation and the convolution block attention model combining channel attention and spatial attention was used to improve the YOLO network to make it more suitable for power lines recognition and an inference module based on contextual information was proposed to determine the power line. A method for accurate and real-time detection of overhead lines of low-voltage distribution network has already been realized.

The experiment was carried out on the overhead lines of a low-voltage distribution network in Wuhan, Hubei, and it has proved that the algorithm proposed in this study can still take speed and accuracy into account under harsh conditions such as complex background environment.

In the future studies, the proposed algorithm can be fused with real-time geographic information and transplanted into edge devices carried by UAVs to realize all aspect automatic recognition of route direction based on UAV aerial photography, effectively improving the basic data of low-voltage distribution network and reducing the burden of operators.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Author contributions

LZ: conceptualization and methodology. LF: data curation, writing—original draft preparation, and software. ZG: visualization and investigation. RZu: supervision. LL: writing–reviewing and editing.

Funding

The study was funded by the National Natural Science Foundation of China (51277134).

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors, and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fenrg.2022.960842/full#supplementary-material

References

Chen, S., Meng, Q., and Li, D. (2020). Remote sensing image edge detection combined with ImageEnhancement and improved canny. J. Henan Univ. Nat. Sci. 50 (05), 623–630. doi:10.15991/j.cnki.411100.2020.05.014

CrossRef Full Text | Google Scholar

Cheng, L., Tong, L., Wang, Y., and Li, M. (2014). Extraction of urban power lines from vehicle-borne LiDAR data. Remote Sens. 6 (4), 3302–3320. doi:10.3390/rs6043302

CrossRef Full Text | Google Scholar

Dai, Z., Yi, J., Zhang, H., Wang, D., Huang, X., and Ma, C. (2022). CODNet: A center and orientation detection network for power line following navigation. IEEE Geosci. Remote Sens. Lett. 19, 1–5. doi:10.1109/lgrs.2021.3092399

CrossRef Full Text | Google Scholar

Gao, Z., Zhao, Y., Yu, Y., Luo, Y., Xu, Z., and Zhang, L. (2020). Low-voltage distribution network topology identification method based on knowledge graph. Power Syst. Prot. Control 48 (02), 34–43. doi:10.19783/j.cnki.pspc.190379

CrossRef Full Text | Google Scholar

Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2010). “Rich feature hierarchies for accurate object detection and semantic segmentation,” in Proceeding of the Computer Vision and Pattern Recognition, June 2014, Columbus, OH, USA, 580–587.

Google Scholar

Girshick, R. (2015). “Fast RCNN,” in Proceedings of the IEEE International Conference on Computer Vision, December 2015, Santiago, Chile, 1440–1448.

CrossRef Full Text | Google Scholar

Jiang, X., Xia, Y., Zhang, Z., Hu, Q., and Hu, J. (2011). Image detection for broken strand faults of transmission conductor based on optimized gabor filter. Autom. Electr. Power Syst. 35, 78–83.

Google Scholar

Jiang, S., Jiang, W., Huang, W., and Yang, L. (2017). UAV-based oblique photogrammetry for outdoor data acquisition and offsite visual inspection of transmission line. Remote Sens. 9 (3), 278. doi:10.3390/rs9030278

CrossRef Full Text | Google Scholar

Kose, O., and Oktay, T. (2020). Simultaneous quadrotor autopilot system and collective morphing system design. Aircr. Eng. Aerosp. Technol. 92 (7), 1093–1100. doi:10.1108/aeat-01-2020-0026

CrossRef Full Text | Google Scholar

Lu, J., Zhou, T., Wu, C., and Li, B. (2016). Fault statistics and analysis of 220kV and above power transmission line in province-level power grid. High. Volt. Eng. 42 (1), 200–207. doi:10.13336/j.1003-6520.hve.2016.01.026

CrossRef Full Text | Google Scholar

Oktay, T., Celik, H., and Turkmen, I. (2018a). Maximizing autonomous performance of fixed-wing unmanned aerial vehicle to reduce motion blur in taken images. Proc. Inst. Mech. Eng. Part I J. Syst. Control Eng. 232 (7), 857–868. doi:10.1177/0959651818765027

CrossRef Full Text | Google Scholar

Oktay, T., Celik, H., and Turkmen, I. (2018b). Constrained control of helicopter vibration to reduce motion blur. Aircr. Eng. Aerosp. Technol. 90 (9), 1326–1336. doi:10.1108/aeat-02-2017-0068

CrossRef Full Text | Google Scholar

Pouliot, N., Richard, P. L., and Montambault, S. (2015). LineScout technology opens the way to robotic inspection and maintenance of high-voltage power lines. IEEE Power Energy Technol. Syst. J. 2 (1), 1–11. doi:10.1109/jpets.2015.2395388

CrossRef Full Text | Google Scholar

Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016). “You only look once:unified, real-time object detection,” in Proceeding of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 779–788.

CrossRef Full Text | Google Scholar

Ren, S., He, K., Girshick, R., and Sun, J. (2015). Faster RCNN: Towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 39 (6), 1137–1149. doi:10.1109/tpami.2016.2577031

PubMed Abstract | CrossRef Full Text | Google Scholar

Şahin, H., Kose, O., and Oktay, T. (2022). Simultaneous autonomous system and powerplant design for morphing quadrotors. Aircr. Eng. Aerosp. Technol. 94, 1228–1241. doi:10.1108/aeat-06-2021-0180

CrossRef Full Text | Google Scholar

Shuai, C., Wang, H., Zhang, G., Kou, Z., and Zhang, W. (2017). "Power lines extraction and distance measurement from binocular aerial images for power lines inspection using UAV,” in Proceeding of the 2017 9th International Conference on Intelligent Human-Machine Systems and Cybernetics (IHMSC), August 2017, Hangzhou, China, 69–74. doi:10.1109/IHMSC.2017.131

CrossRef Full Text | Google Scholar

Zhang, J., Shan, H., Cao, X., Yan, P., and Li, X. (2014). Pylon line spatial correlation assisted transmission line detection. IEEE Trans. Aerosp. Electron. Syst. 50 (4), 2890–2905. doi:10.1109/taes.2014.120732

CrossRef Full Text | Google Scholar

Zhang, Y., Yuan, X., Li, W., and Chen, S. (2017). Automatic power line inspection using UAV images. Remote Sens. 9 (8), 824. doi:10.3390/rs9080824

CrossRef Full Text | Google Scholar

Zhang, H., Yang, W., Yu, H., and Xia, G. S. (2019). Detecting power lines in UAV images with convolutional features and structured constraints. Remote Sens. 11 (11), 1342. doi:10.3390/rs11111342

CrossRef Full Text | Google Scholar

Zhang, J., Wang, J., and Zhang, S. (2021). An ultra-lightweight and ultra-fast abnormal target identification network for transmission line. IEEE Sens. J. 21 (20), 23325–23334. doi:10.1109/jsen.2021.3110878

CrossRef Full Text | Google Scholar

Zhang, Y., Wang, W., and Zhao, S. (2022). Research on automatic extraction of railway catenary power line under complex background based on RBCT algorithm. High. Volt. Eng. 48 (06), 2234–2243. doi:10.13336/j.1003-6520.hve.20210661

CrossRef Full Text | Google Scholar

Zhao, W., Dong, Q., and Zuo, Z. (2022). A method combining line detection and semantic segmentation for power line extraction from unmanned aerial vehicle images. Remote Sens. 14 (6), 1367. doi:10.3390/rs14061367

CrossRef Full Text | Google Scholar

Zhu, K., Xu, C., Wei, Y., and Cai, G. (2022). Fast-PLDN: fast power line detection network. J. Real. Time. Image Process. 19, 3–13. doi:10.1007/s11554-021-01154-3

CrossRef Full Text | Google Scholar