- 1State Grid Hebei Electric Power Company, Shijiazhuang, China
- 2Henan XJ Metering Co., Ltd., Xuchang, China
With the fast development of large-scale Photovoltaic (PV) plants, the automatic PV fault identification and positioning have become an important task for the PV intelligent systems, aiming to guarantee the safety, reliability, and productivity of large-scale PV plants. In this paper, we propose a residual learning-based robotic (UAV) image analysis model for low-voltage distributed PV fault identification and positioning. In our target scenario, the unmanned aerial vehicles (UAVs) are deployed to acquire moving images of low-voltage distributed PV power plants. To get desired robustness and accuracy of PV image detection, we integrate residual learning with attention mechanism into the UAV image analysis model based on you only look once v4 (YOLOv4) network. Then, we design the sophisticated multi-scale spatial pyramid fusion and use it to optimize the YOLOv4 network for the nuanced task of fault localization within PV arrays, where the Complete-IOU loss is incorporated in the predictive modeling phase, significantly enhancing the accuracy and efficiency of fault detection. A series of experimental comparisons in terms of the accuracy of fault positioning are conducted, and the experimental results verify the feasibility and effectiveness of the proposed model in dealing with the safety and reliability maintenance of low-voltage distributed PV systems.
1 Introduction
In recent years, the Photovoltaic (PV) energy has experienced a fast development process, and increasingly, it plays an important role in our daily life due to the advantages of easy installation, cleaning, reasonable return period, and short construction period (Akram et al., 2020; Maka et al., 2021). Compared to conventional power generation approaches, PV power generation shows more superiorities such as safety, reliability, noiselessness, environmentally friendliness, resource distribution, and has a high energy quality and short construction period (Ali et al., 2020; Stiubiener et al., 2020). Especially, with the increasing energy demand, large-scale PV installations have received a surge of attentions, and this leads to the establishment of a mature PV market and technological innovation in the PV industry (Ali et al., 2020; Ma et al., 2022b). With the rapid development of large-scale PV power plants, automatic identification and localization of PV system faults are of critical importance to improving the safety, reliability, and productivity of PV systems. Faults in PV systems can reduce system efficiency and pose potential safety hazards, necessitating the proactive detection of potential faults and the implementation of appropriate corrective measures.
Typically, a large-scale PV system contain massive solar modules, which are variously interconnected with each other. Once one of them fail to work well, the efficiency of the PV system will be largely degraded. Therefore, it is urgently needed to actively detect any potential faults such that suitable corrective measures can be employed before the disruptions happen. To reach this goal, various artificial intelligence (AI) and robotic techniques have been utilized in the fault positioning/diagnosis of PVPP to enhance the intelligent processing capability (Ma et al., 2022b; Wang et al., 2024). Especially, due to the advantages of mobility, flexibility, programmability and large-area coverage, the unmanned aerial vehicles (UAVs) have been widely employed in the PV plant to capture moving images of various PV modules in the distributed power plants. Through intelligent image analysis, we can inspect specific PV faults in an efficient way, which is key to improve the operation and maintenance (O&M) level of the power plant (Atsu et al., 2020; Chen et al., 2020; Abubakar et al., 2021; Navid et al., 2021; Cui et al., 2022).
In fact, many deep learning models have been utilized for the improvement of PV-system fault diagnosis, such as the residual neural network, convolutional neural network, and semi-supervised ladder network (Atsu et al., 2020; Aziz et al., 2020). However, how to develop an effective AI-based approach, which can fully exploit useful fault feature information in UAV images, still is a challenging issue to enhance the fault diagnosis performance.
Regarding the on-site deployment of UAVs for inspection purposes, data used for inspection should be collected and processed first, including geographic information, environmental features, and specific requirements of the expected task. According to the characteristics of the target scene and the requirements of the task, the flight route, inspection frequency and data collection method of the UAV should be designed and implemented. The quality of PV images is challenging to guarantee, and issues such as under-exposure, high noise, and blurred details are frequently encountered. Moreover, the diversity of PV faults and the significant difference in image features among different faults pose significant challenges to the performance of image analysis algorithms.
In this paper, we propose a residual learning-based UAV image analysis model for low-voltage distributed PV fault identification and positioning. In our target PV power plants, the UAVs are deployed to acquire moving images of low-voltage distributed PV products. Based on YOLOv4 network, we integrate residual learning with attention mechanism into the UAV image analysis model, aiming to improve the robustness and accuracy of PV image detection. Then, we propose a sophisticated multi-scale spatial pyramid fusion method and use it to optimize the YOLOv4 network for the nuanced task of fault localization within PV arrays, where the Complete-IOU loss is used in the predictive modeling phase, which is able to significantly enhance the accuracy and efficiency of fault detection. To augment the size of the dataset, this paper adopts data augmentation techniques to flip and adjust the original samples. The image data used comes from three real power plants. The experimental results on a series of datasets verify the effectiveness of the proposed model.
The contributions of this paper are as follows:
1. Novel residual learning-based UAV image analysis model: A residual learning-based UAV image analysis model is proposed for PV fault recognition, where the residual learning network is constructed based on attention mechanism. This model is able to well exploit useful fault feature from UAV moving images, and then significantly boosts the accuracy and efficiency of PV fault identification and localization.
2. Enhanced YOLOv4 optimization for precise fault localization: An improved optimization method is designed to optimize YOLOv4 network, where a sophisticated multi-scale spatial pyramid fusion aims to optimize the model for the nuanced task of fault localization within PV arrays, while the Complete-IOU loss is used in the predictive modeling phase to enhance the accuracy of fault diagnosis in PV systems.
3. Extensive validation on real-world datasets: The proposed model has been trained and tested on a set of datasets to detect its application capability of PV fault detection and diagnosis.
The rest of the paper is organized as follows: section 2 reviews the related work on PV identification and fault positioning research. Section 3 describes the proposed methods in detail. Section 4 reports the experimental result and analysis. Section 5 represents the conclusion and future work.
2 Related work
Currently, many deep learning techniques have been successfully applied to real-world scenarios and effectively solved various challenging problems. For example, Jin et al. (2024) proposed a physical-informed neural network model with model predictive control controller and a perturbation-resistant neural dynamics controller equipped with the noise-suppression ability (Liufu et al., 2024). These two models addressed the challenges faced by autonomous vehicle control systems when dealing with unpredictable external environments and internal system noise disturbances. In the field of PV identification and fault localization, deep learning also has many applications.
Photovoltaic identification and fault localization are important components of PV O&M. It aims to timely discover and eliminate various faults in PV systems, thus improving the performance and safety of systems. Possible faults in PV systems include open-circuit faults, short-circuit faults, hot spot faults, etc. PV faults can be classified according to their occurrence time, severity, persistence, and cause, such as infant failures, midlife failures, wear-out failures, acute failures, chronic failures, permanent failures, and temporary failures (Hong and Pula, 2022). These faults will cause the output power of PV systems to decrease, and even lead to serious consequences such as fire. Therefore, effective methods are needed to diagnose and locate faults in PV systems.
Fault detection methods based on electrical data are a type of method that uses historical or real-time data of voltage, current, power, etc. in PV systems to extract feature information through data mining, machine learning and artificial intelligence technologies. This feature information is then used to establish fault identification and location classifiers or regressors. Some researchers used data analysis techniques such as feature extraction, clustering, classification, and regression to estimate the faults location and severity of PV arrays based on the voltage and current data in PV systems (Alajmi and Abdel-Qader, 2016; Chen and Wang, 2017; Fadhel et al., 2019; Patil and Hinge, 2019). Fadhel et al. proposed a framework for PV faults detection and localization based on hybrid data, which combines voltage–current sensor network and environmental sensors, such as temperature, humidity, and irradiance. It improves the accuracy and robustness of PV faults detection and localization (Alajmi et al., 2018). These methods have the advantages of being independent of model parameters and adaptable to complex environmental conditions. However, it also has the disadvantages of requiring additional sensors, instruments and circuits, and being dependent on environmental conditions.
Computer vision-based fault detection methods use artificial intelligence technologies such as deep learning to process and interpret large amounts of visible or infrared images in PV systems, thus achieving the automation and intelligence of PV fault identification and localization. Some researchers proposed a PV fault detection and localization method based on convolutional neural networks. This method used the thermal imaging data of PV systems to achieve the detection and localization of open circuit faults, short circuit faults, hot spot faults, and shadow faults in PV arrays through image processing, feature extraction, and classifier training (Ali et al., 2020; Herraiz et al., 2020; Alves et al., 2021). Korkmaz et al. proposed a PV fault classification method based on transfer learning and multi-scale convolutional neural networks. This method used the visible light image data of PV systems to achieve the classification of open circuit faults, short circuit faults, hot spot faults, and shadow faults of PV modules through image preprocessing, feature extraction, and classifier training (Korkmaz and Acikgoz, 2022). These methods have the advantages of being able to handle complex nonlinear problems, as well as having self-adaptation and learning ability. However, they also have the disadvantages of requiring a lot of training data, computing resources and algorithm optimization.
This section provides an overview of research concerning photovoltaic identification and fault localization. It primarily explores fault types, diagnostic methods, localization techniques and deep learning models related to photovoltaic modules. However, existing fault Identification and positioning techniques for PV systems based on current location suffer from poor robustness, especially when environmental conditions change. Computer vision-based fault Identification and positioning methods may not achieve the expected accuracy when the data are insufficient or the training is inadequate. In addition, the processing range of image processing methods is narrow, which may not cover all types of faults in PV systems. In response to these limitations, we propose methods of photovoltaic identification and fault localization in low-voltage distributed PVPP.
3 Methods
This section elaborates on the methods of photovoltaic identification and fault localization in low-voltage distributed PVPP, emphasizing the integration and optimization of deep learning technology, especially in the context of drone imaging. The structure of the method presented in this paper is shown in Figure 1.
The primary aim of this framework is to enhance photovoltaic identification and fault localization using deep learning techniques, in order to improve the operational efficiency and safety of photovoltaic power plants. First, the relevant information of low-voltage distributed photovoltaic power plants is acquired. Then, the deep learning model undergoes analysis, and the identification model earmarked for optimization is determined. The model is improved by combining residual U-shaped module composed of dilated convolution and residual network, achieving accurate identification of the photovoltaic area. Finally, the photovoltaic fault localization technique is refined using multi-scale spatial pyramid fusion, Complete-IOU loss and self-attention mechanism, thereby achieving precise fault positioning in photovoltaic power plants.
3.1 PV identification and fault positioning of low-voltage distributed PVPP
A PVPP is a facility that uses light energy to convert it into electricity. It is composed of solar panels and inverters. Solar panels convert sunlight into DC electricity. Inverters convert DC energy into alternating current energy, which is fed into the grid. PVPPs can be divided into centralized and distributed according to their nature. Distributed PVPPs are often installed in factories, residential roofs, fish ponds, and other small ground or building areas, generally connected to the grid through 380 V voltage. It operates flexibly and independently of the grid under the right conditions (Gallardo-Saavedra et al., 2020). The advantages of PVPPs include no emissions, low maintenance costs, and long life. It can effectively reduce environmental pollution and energy consumption. However, the disadvantage is that it depends on the weather and light intensity, and the construction and maintenance costs are relatively high (Gao et al., 2021). As a PV system with rapid growth in installed capacity in the next few years, more and more PV power distribution and installation urgently need efficient and low-cost PVPP health inspection methods to detect the function of PV modules and ensure the normal operation of the system (Jie et al., 2020). Figure 2 shows the actual picture of the low-voltage distributed PVPP.
In a complete PV-EG system, after connecting several PV modules in series, a PV string with DC output is formed, which is called a string. The PV brackets used in PVPP can be divided into fixed and tracking brackets. The tracking brackets can automatically adjust the direction to maximize production capacity (Khalil et al., 2020). In the traditional PV bracket unit design, the PV modules are arranged in vertical double rows or horizontal three and four rows. A bracket unit is usually equipped with one or two strings. The number of modules is determined by the number of modules connected in series in the PV string. PV modules as the core EG module, from top to bottom: glass, upper encapsulant, cell, lower encapsulant, backplane, frame, and junction box (Li et al., 2021).
When a hot spot fails on the cell, highlighted areas appear in the infrared image. Therefore, for defect detection, infrared image defect diagnosis is one of the most widely used defect detection methods. Infrared thermal imaging cameras can detect faults caused by internal defects in PV modules and judge the severity of the fault based on the temperature in a non-invasive way. Inspection UAVs can obtain information, including altitude and temperature, by integrating UAVs, sensors and infrared cameras (Liang et al., 2020a). The detection process is shown in Figure 3.
3.2 PV area identification based on DL
Photovoltaic identification identifies important information, such as the brand and model of the solar panel, by extracting and classifying the features of the solar panel. PV identification plays an important role in the assessment, supervision, and management of PVPPs. Traditional PV identification identifies the transmitted back image through UAV. For most infrared images obtained by PVPP, the PV area absorbs more heat and has a more obvious temperature difference from the ground. The grayscale histogram will have obvious peaks. The PV area can be obtained using a suitable gray threshold method (Liang et al., 2020b). For single-peaked prominent pictures, simple pre-and post-scene separation is done. The maximum inter-class variance threshold segmentation algorithm is considered to find the best threshold for segmentation (Liu et al., 2020). The principle is to convolve the original image through a convolution check of a specific shape and size. When a point pixel of the original image and its surrounding pixels can form a convolution kernel shape, it is retained. If not, it is deleted. The expansion operation, on the contrary, can be used to enlarge the objects in the image or recombine the separated pixels (Lyu et al., 2020).
For traditional image processing algorithms, the process is intuitive, and the calculation is simple and effective. However, the processing scenario is single, and the robustness is low. When the ambient temperature and the PV area temperature are similar, or there is interference in the environment, it is difficult to determine the threshold by the maximum inter-class variance threshold segmentation algorithm in the grayscale histogram. The algorithm directly fails (Manno et al., 2021). The PV region in the infrared image has a typical visual salience, which can be regarded as a binary pixel classification problem in semantic segmentation. The purpose of the semantic segmentation task is to correctly classify each pixel of an image. Each pixel can be classified into a corresponding category by manually determining the semantic label of each pixel and learning (Marqusee et al., 2021). In the PV area visual recognition task, the pixels in the infrared image can be divided into two categories. The target pixel is the PV area pixel, and all the remaining pixels are classified as the background. So, the U-Network (U-Net) semantic segmentation model is introduced. Figure 4 displays the structure of the U-Net model.
The left side of Figure 4 is the downsampling part. It is found that the feature map decreases, and the number of channels increases. The same convolution kernel on the right side is upsampled using the bilinear interpolation method. At present, the U-Net model is affected by the size of the convolution kernel, resulting in small receptive fields and incomplete information capture. Therefore, the receptive field is increased by pooling and dilated convolution. The pooling operation is accompanied by decreased resolution while increasing the receptive field. Dilated convolution can increase the receptive field without increasing the number of parameters and reducing the convolution kernel.
In Equations (1, 2), is the convolution kernel size, and is the dilated convolution kernel size. is the dilation rate hyper-parameter. is the receptive field of the previous layer. is the current receptive field. is the stride product, and i is the number of layers. As the network deepens, degradation and gradient vanishing problems may occur, resulting in deep neural networks that are difficult to train. Although many studies have optimized network architectures, the problem has not been fully solved yet (Li et al., 2023). So, residual learning is introduced. Residual learning is easier than direct learning, so errors can be calculated through residual learning.
In Equation (3), is the input information, is the result of residual learning, and is the result of direct learning. Deeper structures under reasonable calculation can be constructed through the residual U-shaped module composed of dilated convolution and residual network to obtain multi-scale features. The residual U-shaped module retrieves edge features through feature joining, alleviating the problem that the receptive field of simple convolution is too small to capture global information. The overall structure of the optimized model is shown in Figure 5.
Its main architecture is a classical U-Net network similar to Encoder-Decoder, a simple and effective saliency object detection network that can be used for the semantic segmentation of two classifications. Each stage comprises residual U-shaped modules, which can more effectively capture the global features of multi-scale features. Specific residual U-shaped modules are used in different encoder and decoder stages. From the first layer to the seventh layer, the information structure of the picture has been analyzed and deconstructed. Residual learning enables the model to construct deeper structures and extract multi-scale features with reasonable computation, which facilitates effective visual identification of PV areas under various power plant scenarios and environmental interferences. It also empowers the model with better adaptability and learning ability when dealing with complex non-linear problems. The advantage of this is that more and more feature information will be extracted. The key components of the optimized model is shown in Table 1.
The process of the proposed optimized semantic segmentation model can be as follows:
Step 1: The input image is fed into the input layer of the network for feature extraction.
Step 2: In encoder stage 1, the residual U-shaped module performs a preliminary feature analysis on the image, extracting low-level edge and texture information.
Step 3: In encoder stage 2, the enhanced residual U-shaped module conducts a deeper feature analysis on the image, extracting high-level semantic and structural information.
Step 4: At the intermediate layer, a complex feature integration method combines features at different scales and resolutions, resulting in a global feature representation.
Step 5: In decoder stage 1, bilinear interpolation upsampling and convolution operations reconstruct the features, while concatenation operations connect the features from encoder stage 1 with the features from decoder stage 1, achieving global information transfer.
Step 6: In decoder stage 2, bilinear interpolation upsampling and convolution operations further reconstruct the features, while concatenation operations connect the features from the input layer with the features from decoder stage 2, achieving detail information recovery.
Step 7: At the output layer, convolution operations transform the features into a binary semantic segmentation image, representing the analyzed and processed features.
3.3 Fault positioning technology and model of low-voltage distributed PVPP
For the infrared image fault diagnosis system of PVPP based on UAV inspection, the health of the components is the most concerned item of the power plant after obtaining infrared images. The corresponding PV module information can be used to guide subsequent O&M after making judgments in infrared fault diagnosis (Mellit and Kalogirou, 2021). In the UAV inspection proposed here, the shooting height can be set in the path planning stage. The field of view of the infrared picture can be locked by collecting the original power plant information and camera angle. The main PV mounts are adjusted horizontally to ensure m complete brackets in one infrared image or one complete bracket in n continuous images (Qais et al., 2020). For the resulting profile composed of several points, the vertex information can be obtained by quadrilateral fitting by the Ramer-Douglas-Peucker algorithm (Quiles et al., 2020). This method, also known as the iterative endpoint fitting algorithm, is an algorithm that approximates the curve as a series of points and reduces the number of points. The contour points obtained are a subset of the original contour (Ramadan et al., 2020).
The Line Segment Detector method is a linear detection method with low time complexity. It forms a horizontal line field by first calculating the horizontal line angle within the eight neighborhoods of each pixel. The vertical angle of the gradient direction of this pixel is the horizontal line angle, and the horizontal line field is a matrix corresponding to the points in the image one by one (Ridha et al., 2020). After obtaining the horizontal line field, the area growth method is used to generate several connected domains according to the horizontal line angle. The threshold t is set to 22.5 degrees, and the horizontal line angle change of all pixels in each connected domain cannot exceed t. The connected domain obtained at this time is called the line support area. Each line support area is a candidate for segment detection. Each line support area corresponds to a matrix, represented by its smallest circumscribed rectangle (Ridha et al., 2021). The line support area’s spindle direction is the matrix’s major axis direction, and the rectangle covers the entire area. The smallest external rectangle represents the straight-line information. The confirmation line can be obtained by filtering the fitted rectangle information. The complete single PV module can be obtained by simple area extraction (Romero-Fiances et al., 2022).
Common fault positioning classification models are convolutional neural networks (CNNs) and Transformer, and the dominant CNNs benefit from their inductive bias. For example, the adjacent area has adjacent and translation invariance characteristics. The implied visual prior knowledge can be effectively used to extract information. Therefore, there is good ingestion even for small data, although many studies have been conducted to optimize the CNN architecture (Ma et al., 2023a), which may still lead to performance degradation (Tsanakas et al., 2020). In sequence-related tasks in natural language processing (NLP) neighborhoods, recurrent neural networks (RNNs) with fixed structure memory units simplify the difficulty of long-distance learning and significantly outperform other RNNs (Veerasamy et al., 2021). However, with the introduction of the attention mechanism, it broke through the field of traditional NLP and became the model with the best performance. Figure 6 demonstrates the structure of multi-headed self-attention mechanism.
Compared with CNNs, calculating the association between two positions in the AM only requires calculating the association weight between the two pairs. In contrast, multi-layer convolution is required in CNNs to obtain the relationship between distant positions. However, the model based on the self-attention mechanism needs to calculate more parameters and lacks the inductive bias brought by convolution. CNN has more advantages when there is less data. The target detection network is the dual information determination task of target position and category information (Yang et al., 2021). Traditional detection modes use a combination of candidate boxes and feature extraction. For example, manually designed features are extracted in each window by sliding windows. Then, the features are obtained with a simple classifier (Zghaibeh et al., 2022; Ma et al., 2022a, 2023b). Here, the you only look once v4 (YOLOv4) network is needed. Its overall loss function consists of regression box loss, confidence loss, and classification loss. A threshold is set for confidence. For lower thresholds, no classification loss occurs. This method can solve the problem of sample imbalance well, which is suitable for the detection target required here. The formula of Complete-IOU loss for YOLOv4 is shown below:
In Equations (4–6), is the overlap between the predicted bounding box and the ground truth bounding box. is the Euclidean distance between the centers of the predicted bounding box () and the ground truth bounding box (), normalized by the diagonal length of the smallest enclosing box covering both boxes. is the diagonal length of the smallest enclosing box covering both the predicted and ground truth bounding boxes. is a value that represents the aspect ratio difference between the predicted bounding box and the ground truth bounding box. The optimized YOLOv4 network structure is given in Figure 7.
Based on the optimized structure of YOLOv4 shown above, a PV positioning method of low-voltage distributed PVPP is proposed, and its process is as follows:
Step 1: The feature map obtained from the backbone network is fed into the prediction network. The CBL includes three components: convolution (Conv), batch normalization (Batch Norm), and leaky rectified linear unit (Leaky ReLU).
Step 2: Introduce additional layers in the prediction network to optimize the algorithm performance, enhance the feature detection capabilities, and increase the sensitivity to the nuances of the PV array images.
Step 3: After using a multi-scale fusion of spatial pyramid pooling (SPP), YOLOv4 improves the feature extraction capability through the feature pyramid and path aggregation network fusion structure. The feature pyramid network, in conjunction with the path aggregation network, forms a fusion structure that effectively consolidates features at various scales and resolutions.
Step 4: Complete-IOU loss is used in the prediction, which integrates the prediction box boundary non-coincidence, center distance information, and aspect ratio information so that the regression operation of the prediction box obtains fast speed and high accuracy. The utilization of Complete-IOU loss results in a more nuanced and detailed regression operation of the prediction box.
This enhances both the speed and accuracy of the algorithm, allowing for rapid processing without compromising the quality of fault detection. The accurate detection of boundaries, combined with the precise localization of faults, makes our optimized YOLOv4 algorithm particularly effective for PV fault positioning. This level of accuracy is critical in the context of PV maintenance, where the timely and precise identification of faults can significantly impact the efficiency and longevity of PV installations.
4 Experimental result and analysis
4.1 Comparative analysis of experimental results of PV area identification
The image data used in the experimental training set comes from three real power plants. They represent three types of power plants: mountain PV, fishery-solar complementary, and agro-solar complementary. They are named training set 1, training set 2, and training set 3. The test data set is also selected from three real power plants, called test set 1, test set 2, and test set 3. The original samples of the dataset are flipped and adjusted for broadening. We set the ratio of the training sets to the tests set at 8:2. The specific sample number of the dataset is shown in Table 2.
The models participating in the comparative experiment include Otsu, Grabcut, U-Net, Recurrent Residual CNN-based U-Net (RES U-Net), U2-Net, and the optimized model proposed here. The structure and hyperparameter settings of the algorithm used in the experiment are shown in Table 3.
There are three experimental evaluation indicators: accuracy, recall, and Intersection over Union (IoU). Accuracy represents the proportion of correctly identified PV area pixels to the total pixels. Recall represents the proportion of correctly identified PV area pixels to the actual PV area pixels. IoU represents the ratio of the intersection of correctly identified PV area pixels and actual PV area pixels to their union. Figure 8 reveals the results.
From Figure 8, in the comparative experiments of accuracy and IoU, the corresponding data of the optimized model are the highest, 96.5 and 95.07%, respectively. This indicates that the model can effectively distinguish between the PV area and the background area, and has a high degree of matching with the real PV area. Regarding recall, the optimized model has the highest recall rate of 98.46%, except that it is not as high as the Grabcut model of 98.54%. Its recall rate is only 0.08% lower than the Grabcut model. This indicates that the model can cover most of the real PV areas, but there are also a few cases of missed detection. Comparative experiments can show that the optimized model has high adaptability to different power plant scenarios and environmental disturbances. Additionally, it has a high degree of attention to the data at the edge of the PV power plant, which can effectively carry out visual identification of the PV area from the image.
The optimized U-Network semantic segmentation model proposed here uses residual U-shaped module composed of dilated convolution and residual network, which can build deeper structures and obtain multi-scale features under reasonable computation. The model extract edge features through feature joining, avoiding the problems of gradient disappearance and degradation in neural network, and alleviating the problem of small receptive field and inability to capture global information in simple convolution. This can effectively achieve accurate identification of the photovoltaic area. The introduction of residual learning enhances feature extraction ability of the model, enabling the model to adapt to different power station scenarios and environmental disturbances, thus effectively performing visual recognition of the PV area.
Based on the above improvements, the proposed algorithm not only effectively addresses the limitations of limited receptive fields and inability to capture global information inherent in simple convolutions. More importantly, it exhibits remarkable adaptability in handling PV area identification tasks under diverse power plant scenarios and environmental interferences. This is the reason why the optimized model proposed in this aper can achieve good performance.
4.2 Experimental analysis of fault positioning in PV area
The dataset of the fault positioning experiment uses the dataset in PV identification, from which 2,000 sheets are extracted to construct the training set. The PV images in datasets are shown in Figure 9.
The number of the fault and normal components is classified. The data of the target detection dataset is presented in Table 4.
The evaluation indicators are precision, recall, true positive (TP), false positive (FP), false negative (FN) and average precision. The experimental results are plotted in Figure 10.
From Figure 10, the optimized YOLOv4 has played a very high performance. The recall rate of normal components has reached 100%, and the precision has gained 99.9% by testing normal components. When testing hot spot fault components, it is found that the recall rate of hot spot fault components also reaches 99.53%, and the precision rate reaches 98.73%. In the identification of normal components, the data of FN is 0. In addition to high precision and recall, the proposed model has great advantages in terms of operation time. Models chosen for comparison include single shot multibox detector (SSD), Faster region based convolutional neural network (Faster R-CNN), YOLO, YOLOv2, YOLOv3, and YOLOv4. The experimental results are shown in Figure 11.
Through the comparison of Figure 11, the average accuracy (Map) of the optimized model is the highest, at 90%. The memory requirement is 1.5G, and its operation time is only 0.022 s. This shows that the optimized YOLOv4 model has lower computational complexity and higher running speed, which can quickly process a large number of UAV infrared images, saving O&M costs and time. Through horizontal comparison, the prediction time of the proposed model is shorter than that of other traditional models, and the memory requirement is small. This shows that the optimized YOLOv4 model occupies less resources and is more suitable for deployment and running on devices with limited resources. Combined with the previous experiments, the overall accuracy and recall of the optimized model are much higher than those of the traditional model for PV identification and fault positioning. Therefore, the rationality and effectiveness of the proposed model can be verified by comparative experiments.
From the above analysis, it is evident that the optimized YOLOv4 model has a significant advantage in the fault positioning task of PV components, not only performing well in terms of precision and recall, but also having obvious optimization in prediction time and memory requirement. The optimized YOLOv4 proposed here improves the robustness and generalization ability of the model by effectively integrating features of different scales and resolutions through multi-scale fusion of spatial pyramids. This can make the model to recognize both macro and micro features in PV array images. The Complete-IOU loss function takes into account the non-overlap, center distance and aspect ratio of prediction box. This can refine the regression process for prediction boxes, and enhance localization precision and recall rates, and facilitating precise fault localization in PV systems. These improvements enable the optimized YOLOv4 model to better capture and analyze the details and features of the PV component images, thereby enhancing the performance of fault localization.
Experimental results show that the proposed PV identification model has high accuracy and IoU. In PV identification, the accuracy rate of the optimized model can reach 96.5%, and the IoU is 95.07%. Among the six models compared, the IoU is also the highest. Moreover, the recall rate of the optimized model is only 0.08% lower than that of the Grabcut model, which has the highest recall rate. It verifies the effectiveness of the PV identification model proposed here. In the experiment of fault positioning, precision, recall, TP, FP, FN, and average precision are used as performance indicators. The fault positioning model achieves 100% recall and precision of 99.9% in testing normal components. For the hot spot fault component test, the recall rate of the hot spot fault component also reaches 99.53%, and the precision rate reaches 98.73%. The model performs well. Meanwhile, a horizontal comparison is added at the end of the experiment. Compared with the traditional model, the average accuracy value of the optimized model is the highest, 90%. The memory requirement is 1.5G, and its operation time is only 0.022 s, significantly exceeding the operation time of other models.
The experimental results demonstrate that the proposed model has high practical value and effectiveness in the O&M of real PV power plants. It can provide strong support for the safe, reliable and efficient operation of PV power plants. Specifically, the model can accurately locate faults in PV modules, helping O&M personnel to timely discover and solve problems, thereby improving the operation efficiency and long-term stability of PV power plants. For example, in the actual application of a certain PV power plant, the model successfully identified a batch of aging PV modules and replaced them in time, avoiding accidents at the power plant.
5 Conclusion
As the installed capacity of PV power generation increases rapidly, how to detect abnormalities and faults of PV modules in an efficient manner has become a key challenge to maintain the safety, reliability, and productivity of large-scale PV plants. With the consideration that all the fault information of the PV module exists in the moving images of UAVs, we propose an improved residual learning model to extract useful fault feature from the UAV moving images, and then use it for low-voltage distributed PV fault identification and positioning. This way works in an end-to-end way, and it can not only detect single faults, but also identify the existence of hybrid PV faults. First, we integrate residual learning with attention mechanism into the UAV image analysis model, aiming to improve the robustness and accuracy of PV image detection. Then, we propose a sophisticated multi-scale spatial pyramid fusion method for the optimization of the YOLOv4 network, targeting at the nuanced task of fault localization within PV arrays, where the Complete-IOU loss is used in the predictive modeling phase, significantly enhancing the accuracy and efficiency of fault detection. The proposed novel residual learning model and optimized YOLOv4 network were applied to fault identification and localization in low-voltage distributed PV systems. The models were trained and tested on a real-world dataset, demonstrating their application potential in fault detection and diagnosis of low-voltage distributed PV systems. This research is of great significance to ensure the safety, reliability and productivity of large-scale PV power plants.
The current training dataset is restricted to a limited number of fault categories, hindering direct applicability to fault identification and positioning of equipment in other types or domains. To enhance generalization capability and robustness of the proposed algorithm, we will focus on expanding the dataset to encompass a broader spectrum of fault types in future research. Moreover, considering that many novel and effective computer vision and deep learning methods emerge rapidly, we will adopt state-of-the-art intelligent models rather than the original neural network models in the target PV fault identification and positioning. In addition, we attempt to optimize the scheduling problem after fault localization using multi-objective optimization algorithms.
Data availability statement
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.
Author contributions
XZ: Conceptualization, Formal analysis, Methodology, Validation, Writing – original draft, Writing – review & editing. YG: Formal analysis, Methodology, Validation, Writing – review & editing. YW: Data curation, Formal analysis, Validation, Writing – review & editing. JW: Formal analysis, Methodology, Validation, Writing – review & editing. WW: Data curation, Validation, Writing – review & editing. LL: Data curation, Validation, Writing – review & editing.
Funding
The author(s) declare that no financial support was received for the research, authorship, and/or publication of this article.
Conflict of interest
XZ, YG, and YW were employed by State Grid Hebei Electric Power Company. JW, WW, and LL were employed by Henan XJ Metering Co., Ltd.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
Abubakar, A., Almeida, C. F. M., and Gemignani, M. (2021). Review of artificial intelligence-based failure detection and diagnosis methods for solar photovoltaic systems. Mach. Des. 9:328. doi: 10.3390/machines9120328
Akram, M. W., Li, G., Jin, Y., Chen, X., Zhu, C., and Ahmad, A. (2020). Automatic detection of photovoltaic module defects in infrared images with isolated and develop-model transfer deep learning. Sol. Energy 198, 175–186. doi: 10.1016/j.solener.2020.01.055
Alajmi, M., and Abdel-Qader, I. (2016). Fault detection and localization in solar PV arrays using the current-voltage sensing framework[C]//2016 IEEE international conference on electro information technology (EIT). IEEE, 0307–0312. doi: 10.1109/EIT.2016.7535257
Alajmi, M., Aljasem, O., Ali, N., Alqurashi, A., Abdel-Qader, I., et al. (2018). Fault detection and localization in solar PV arrays framework: hybrid methods of data-analysis and a network of voltage-current sensors[C]//2018 IEEE international conference on electro/information technology (EIT). IEEE, 0404–0410. doi: 10.1109/EIT.2018.8500264
Ali, M. U., Khan, H. F., Masud, M., Kallu, K. D., and Zafar, A. (2020). A machine learning framework to identify the hotspot in photovoltaic module using infrared thermography. Sol. Energy 208, 643–651. doi: 10.1016/j.solener.2020.08.027
Alves, R. H. F., de Deus, J. G. A., Marra, E. G., et al. (2021). Automatic fault classification in PV modules using convolutional neural networks. Renew. Energy 179, 502–516. doi: 10.1016/j.renene.2021.07.070
Atsu, D., Seres, I., Aghaei, M., and Farkas, I. (2020). Analysis of long-term performance and reliability of PV modules under tropical climatic conditions in sub-Saharan. Renew. Energy 162, 285–295. doi: 10.1016/j.renene.2020.08.021
Aziz, F., Haq, A. U., Ahmad, S., Mahmoud, Y., Jalal, M., and Ali, U. (2020). A novel convolutional neural network-based approach for fault classification in photovoltaic arrays. IEEE Access 8, 41889–41904. doi: 10.1109/ACCESS.2020.2977116
Chen, L., and Wang, X. (2017). Adaptive fault localization in PV systems. IEEE Trans. Smart Grid 9, 6752–6763. doi: 10.1109/TSG.2017.2722821
Chen, S. Q., Yang, G. J., Gao, W., and Guo, M. F. (2020). Photovoltaic fault diagnosis via semisupervised ladder network with string voltage and current measures. IEEE J. Photovolt. 11, 219–231. doi: 10.1109/JPHOTOV.2020.3038335
Cui, F., Tu, Y., and Gao, W. (2022). A photovoltaic system fault identification method based on improved deep residual shrinkage networks. Energies 15:3961. doi: 10.3390/en15113961
Fadhel, S., Delpha, C., Diallo, D., Bahri, I., Migan, A., Trabelsi, M., et al. (2019). PV shading fault detection and classification based on IV curve using principal component analysis: application to isolated PV system. Sol. Energy 179, 1–10. doi: 10.1016/j.solener.2018.12.048
Gallardo-Saavedra, S., Hernández-Callejo, L., del Carmen, A.-G. M., Morales-Aragonés, J. I., Alonso-Gómez, V., and Martínez-Sacristán, O. (2020). Nondestructive characterization of solar PV cells defects by means of electroluminescence, infrared thermography, I–V curves and visual tests: experimental study and comparison. Energy 205:117930. doi: 10.1016/j.energy.2020.117930
Gao, S., Wang, K., Tao, S., Jin, T., Dai, H., and Cheng, J. (2021). A state-of-the-art differential evolution algorithm for parameter estimation of solar photovoltaic models. Energy Convers. Manag. 230:113784. doi: 10.1016/j.enconman.2020.113784
Herraiz, Á. H., Marugán, A. P., and Márquez, F. P. G. (2020). PV plant condition monitoring using thermal images analysis by convolutional neural network-based structure. Renew. Energy 153, 334–348. doi: 10.1016/j.renene.2020.01.148
Hong, Y. Y., and Pula, R. A. (2022). Methods of PV fault detection and classification: a review. Energy Rep. 8, 5898–5929. doi: 10.1016/j.egyr.2022.04.043
Jie, Y., Ji, X., Yue, A., Deng, Y., Chen, J., and Zhang, Y. (2020). Combined multi-layer feature fusion and edge detection method for distributed photovoltaic power station identification. Energies 13:6742. doi: 10.3390/en13246742
Jin, L., Liu, L., Wang, X., Shang, M., and Wang, F. Y. (2024). Physical-informed neural network for MPC-based trajectory tracking of vehicles with noise considered. IEEE Trans. Intellig. Vehicles, 1–10. doi: 10.1109/TIV.2024.3358229
Khalil, I. U., Ul-Haq, A., Mahmoud, Y., Aamir, M., Ahsan, M. U., and Mehmood, K. (2020). Comparative analysis of photovoltaic faults and performance evaluation of its detection techniques. IEEE Access 8, 26676–26700. doi: 10.1109/ACCESS.2020.2970531
Korkmaz, D., and Acikgoz, H. (2022). An efficient fault classification method in solar PV modules using transfer learning and multi-scale convolutional neural network. Eng. Appl. Artif. Intell. 113:104959. doi: 10.1016/j.engappai.2022.104959
Li, N., Ma, L., Guo, Y., Xue, B., Zhang, M., and Jin, Y. (2023). Survey on evolutionary deep learning: principles, algorithms, applications and open issues. ACM Comput. Surv. 56, 1–34. doi: 10.1145/3603704
Li, K., Yan, J., Hu, L., Wang, F., and Zhang, N. (2021). Two-stage decoupled estimation approach of aggregated baseline load under high penetration of behind-the-meter PV system. IEEE Trans. Smart Grid 12, 4876–4885. doi: 10.1109/TSG.2021.3105747
Liang, J., Ge, S., Qu, B., Liu, F., Yang, H., and Li, Z. (2020a). Classified perturbation mutation base particle swarm optimization algorithm for parameters extraction of photovoltaic models. Energy Convers. Manag. 203:112138. doi: 10.1016/j.enconman.2019.112138
Liang, J., Qiao, K., Yu, K., Qu, B., Xu, R., and Li, K. (2020b). Parameters estimation of solar photovoltaic models via a self-adaptive ensemble-based differential evolution. Sol. Energy 207, 336–346. doi: 10.1016/j.solener.2020.06.100
Liu, Y., Chong, G., Heidari, A. A., Liang, G., Ye, X., and Wang, M. (2020). Horizontal and vertical crossover of Harris hawk optimizer with Nelder-Mead simplex for parameter estimation of photovoltaic models. Energy Convers. Manag. 223:113211. doi: 10.1016/j.enconman.2020.113211
Liufu, Y., Jin, L., Shang, M., Wang, X., and Wang, F. Y. (2024). ACP-incorporated perturbation-resistant neural dynamics controller for autonomous vehicles. IEEE Trans. Intellig. Vehicles, 1–12. doi: 10.1109/TIV.2023.3348632
Lyu, Y., Fairbrother, A., Gong, M., Gu, X., Kempe, M., and Boyce, K. (2020). Impact of environmental variables on the degradation of photovoltaic components and perspectives for the reliability assessment methodology. Sol. Energy 199, 425–436. doi: 10.1016/j.solener.2020.02.020
Ma, L., Huang, M., Yang, S., Wang, R., and Wang, X. (2022a). An adaptive localized decision variable analysis approach to large-scale multiobjective and many-objective optimization. IEEE Trans. Cybernet. 52, 6684–6696. doi: 10.1109/TCYB.2020.3041212
Ma, L., Li, N., Guo, Y., Geng, X., Cheng, S., Wang, X., et al. (2023a). Pareto-wise ranking classifier for multi-objective evolutionary neural architecture search. IEEE Trans. Evol. Comput. :1. doi: 10.1109/TEVC.2023.3314766
Ma, L., Li, N., Guo, Y., Huang, M., Yang, S., Wang, X., et al. (2022b). Learning to optimize: reference vector reinforcement learning adaption to constrained many-objective optimization of industrial copper burdening system. IEEE Trans. Cybernet. 52, 12698–12711. doi: 10.1109/TCYB.2021.3086501
Ma, L., Liu, Y., Guo, Y., Wang, X., Mo, H., Wang, G., et al. (2023b). Decomposition-based multiobjective optimization for variable-length mixed-variable Pareto optimization and its application in cloud service allocation. IEEE Trans. Syst. Man. Cybernet. Syst. 53, 7138–7151. doi: 10.1109/TSMC.2023.3295371
Maka, A. O. M., Salem, S., and Mehmood, M. (2021). Solar photovoltaic (PV) applications in Libya: challenges, potential, opportunities and future perspectives. Clean. Eng. Technol. 5:100267. doi: 10.1016/j.clet.2021.100267
Manno, D., Cipriani, G., Ciulla, G., Di, D. V., Guarino, S., and Brano, V. L. (2021). Deep learning strategies for automatic fault diagnosis in photovoltaic systems by thermographic images. Energy Convers. Manag. 241:114315. doi: 10.1016/j.enconman.2021.114315
Marqusee, J., Becker, W., and Ericson, S. (2021). Resilience and economics of microgrids with PV, battery storage, and networked diesel generators. Adv. Appl. Energy 3:100049. doi: 10.1016/j.adapen.2021.100049
Mellit, A., and Kalogirou, S. (2021). Artificial intelligence and internet of things to improve efficacy of diagnosis and remote sensing of solar photovoltaic systems: challenges, recommendations and future directions. Renew. Sust. Energ. Rev. 143:110889. doi: 10.1016/j.rser.2021.110889
Navid, Q., Hassan, A., Fardoun, A. A., Ramzan, R., and Alraeesi, A. (2021). Fault diagnostic methodologies for utility-scale photovoltaic power plants: a state of the art review. Sustain. For. 13:1629. doi: 10.3390/su13041629
Patil, M., and Hinge, T. (2019). Improved fault detection and location scheme for PV system[C]//2019 innovations in power and advanced computing technologies (i-PACT). IEEE 1, 1–6. doi: 10.1109/i-PACT44901.2019.8960246
Qais, M. H., Hasanien, H. M., and Alghuwainem, S. (2020). Parameters extraction of three-diode photovoltaic model using computation and Harris hawks optimization. Energy 195:117040. doi: 10.1016/j.energy.2020.117040
Quiles, E., Roldán-Blay, C., Escrivá-Escrivá, G., and Roldán-Porta, C. (2020). Accurate sizing of residential stand-alone photovoltaic systems considering system reliability. Sustain. For. 12:1274. doi: 10.3390/su12031274
Ramadan, A., Kamel, S., Korashy, A., and Yu, J. (2020). Photovoltaic cells parameter estimation using an enhanced teaching–learning-based optimization algorithm. Iran. J. Sci. Technol. Trans. Electr. Eng. 44, 767–779. doi: 10.1007/s40998-019-00257-9
Ridha, H. M., Heidari, A. A., Wang, M., and Chen, H. (2020). Boosted mutation-based Harris hawks optimizer for parameters identification of single-diode solar cell models. Energy Convers. Manag. 209:112660. doi: 10.1016/j.enconman.2020.112660
Ridha, H. M., Hizam, H., Gomes, C., Chen, H., Ahmadipour, M., and Alghrairi, M. (2021). Parameters extraction of three diode photovoltaic models using boosted LSHADE algorithm and Newton Raphson method. Energy 224:120136. doi: 10.1016/j.energy.2021.120136
Romero-Fiances, I., Livera, A., Theristis, M., Stein, S., Nofuentes, G., and Georghiou, G. E. (2022). Impact of duration and missing data on the long-term photovoltaic degradation rate estimation. Renew. Energy 181, 738–748. doi: 10.1016/j.renene.2021.09.078
Stiubiener, U., Carneiro da Silva, T., Trigoso, F. B. M., Benedito, R. S., and Teixeira, J. C. (2020). PV power generation on hydro dam’s reservoirs in Brazil: a way to improve operational flexibility. Renew. Energy 150, 765–776. doi: 10.1016/j.renene.2020.01.003
Tsanakas, J. A., van der Heide, A., Radavičius, T., Lemaire, E., Wang, K., and Voroshazi, E. (2020). Towards a circular supply chain for PV modules: review of today's challenges in PV recycling, refurbishment and re-certification. Prog. Photovolt. Res. Appl. 28, 454–464. doi: 10.1002/pip.3193
Veerasamy, V., Wahab, N. I. B., Othman, M. L., Sekar, K., Ramachandran, R., and Islam, M. Z. (2021). LSTM recurrent neural network classifier for high impedance fault detection in solar PV integrated power system. IEEE Access 9, 32672–32687. doi: 10.1109/ACCESS.2021.3060800
Wang, H., Liu, R., Ding, S. X., Hu, Q., Li, Z., and Zhou, H. (2024). Causal-trivial attention graph neural network for fault diagnosis of complex industrial processes. IEEE Trans. Industr. Inform. 20, 1987–1996. doi: 10.1109/TII.2023.3282979
Yang, Y., Ma, C., Lian, C., Zhang, Y., and Pang, X. (2021). Optimal power reallocation of large-scale grid-connected photovoltaic power station integrated with hydrogen production. J. Clean. Prod. 298, 126830–126835. doi: 10.1016/j.jclepro.2021.126830
Zghaibeh, M., Okonkwo, P. C., Belgacem, I. B., Beitelmal, W. H., and Mansir, I. B. (2022). Analytical model for a techno-economic assessment of green hydrogen production in photovoltaic power station case study Salalah city-Oman. Int. J. Hydrog. Energy 47, 14171–14179. doi: 10.1016/j.ijhydene.2022.02.180
Keywords: low-voltage distributed photovoltaics, photovoltaic identification, positioning technology, unmanned aerial vehicle imagery, horizontal comparison experiment
Citation: Zhang X, Ge Y, Wang Y, Wang J, Wang W and Lu L (2024) Residual learning-based robotic image analysis model for low-voltage distributed photovoltaic fault identification and positioning. Front. Neurorobot. 18:1396979. doi: 10.3389/fnbot.2024.1396979
Edited by:
Long Jin, Lanzhou University, ChinaReviewed by:
Ming Wan, Liaoning University, ChinaYan Pei, University of Aizu, Japan
Shangce Gao, University of Toyama, Japan
Copyright © 2024 Zhang, Ge, Wang, Wang, Wang and Lu. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Jun Wang, tasi0923@163.com