Forest fire monitoring via uncrewed aerial vehicle image processing based on a modified machine learning algorithm

Forests are indispensable links in the ecological chain and important ecosystems in nature. The destruction of forests seriously influences the ecological environment of the Earth. Forest protection plays an important role in human sustainable development, and the most important aspect of forest protection is preventing forest fires. Fire affects the structure and dynamics of forests and also climate and geochemical cycles. Using various technologies to monitor the occurrence of forest fires, quickly finding the source of forest fires, and conducting early intervention are of great significance to reducing the damage caused by forest fires. An improved forest fire risk identification algorithm is established based on a deep learning algorithm to accurately identify forest fire risk in a complex natural environment. First, image enhancement and morphological preprocessing are performed on a forest fire risk image. Second, the suspected forest fire area is segmented. The color segmentation results are compared using the HAF and MCC methods, and the suspected forest fire area features are extracted. Finally, the forest fire risk image recognition processing is conducted. A forest fire risk dataset is constructed to compare different classification methods to predict the occurrence of forest fire risk to improve the backpropagation (BP) neural network forest fire identification algorithm. An improved machine learning algorithm is used to evaluate the classification accuracy. The results reveal that the algorithm changes the learning rate between 0.1 and 0.8, consistent with the cross-index verification of the 10x sampling algorithm. In the combined improved BP neural network and support vector machine (SVM) classifier, forest fire risk is recognized based on feature extraction and the BP network. In total, 1,450 images are used as the training set. The experimental results reveal that in image preprocessing, image enhancement technology using the frequency and spatial domain methods can enhance the useful information of the image and improve its clarity. In the image segmentation stage, MCC is used to evaluate the segmentationresults. The accuracy of this algorithm is high compared with other algorithms, up to 92.73%. Therefore, the improved forest fire risk identification algorithm can accurately identify forest fire risk in the natural environment and contribute to forest protection.

Introduction

Forests are vital links in the ecological chain and essential ecosystems in nature. Damaged forests seriously affect the ecological environment of the Earth (Han et al., 2018). Protecting the forest environment plants plays a critical role in sustainable human development, and the most important aspect of protecting forest plants is preventing forest fires. If a forest fire is not monitored and warned of in time, it burns down trees, reduces forest accumulation, and causes soil erosion and vegetation damage (Ryan et al., 2021). Forest fires can also destroy understory plant resources and cause irreparable losses to rich wild plant resources. Developing diversified forest fire risk monitoring methods is necessary to reduce the adverse effects of forest fires (Cai et al., 2018).

Developed countries have conducted substantial research on monitoring forest fire disasters to reduce the losses caused by forest fires. For example, Canada has a vast territory with an average of thousands of forest fires daily, but no major fire has occurred in recent decades. Japan has also made some achievements in forest fire disaster monitoring, and there have been no major fires. These are inseparable from the rapid development of forest fire monitoring technology (Van Hoang et al., 2020). Currently, there are two main monitoring methods for forest fire risk at home and abroad. The first is through ground monitoring, including infrared, video, and radar monitoring, and the other is air monitoring, primarily through satellite, microwave, and infrared monitoring (Hu et al., 2022).

The recognition and monitoring of forest fire risk images using a machine learning algorithm have been successfully performed in recent years to realize the timely monitoring of forest fires. First, to extract the features of the forest fire risk image, we segment the image and then classify it according to the features.

Thach et al. (2018) analyzed the spatial pattern of tropical forest fire risk using random forest and the multilayer perceptron neural network and used the Pearson correlation method to evaluate the correlation between the variables and forest fire. In total, three forest fire risk models, the support vector machine (SVM) classifier, random forest, and multilayer perceptron neural network, were trained and verified (Thach et al., 2018).

Lim et al. (2019) constructed two forest fire risk prediction models based on satellite fire data and medium-resolution imaging spectrometry data monitored by the Korean Forestry Administration. They analyzed the spatial autocorrelation between the fire frequency and intensity of the two data types using a semivariogram. The accuracy and performance of the model are good (Lim et al., 2019).

Balling improved the processing and analyzability of the forest image returned by the uncrewed aerial vehicle (UAV) by designing the UAV forest fire prevention system, forest fire image monitoring algorithm, and intelligent landing gear system. This method realized the real-time monitoring of forest fire and improved the digitization and automation of forest fire early warning and prevention (Balling et al., 2021).

Devotta et al. (2021) proposed an improved recognition and positioning algorithm based on the color index. Combined with the forest fire monitoring recognition algorithm of the UAV, it can process the video image data returned during the flight of the aircraft in real time, which can monitor and recognize forest fire risk and accurately judge its location (Devotta et al., 2021).

However, image processing technology based on machine learning can extract and analyze the image features of forest fire risk and effectively identify the risk of a forest fire. Zhang et al. (2019) built a forest fire prediction model based on the convolutional neural network structure suitable for forest fire sensitivity prediction.

Moayedi et al. (2020) adopted a hybrid evolutionary algorithm to realize the approximate and reasonable task of this forest fire environmental threat. A total of three fuzzy meta-heuristic algorithms, the genetic algorithm, particle swarm optimization algorithm, and differential evolution algorithm, were used to construct a sensitive area model of the forest fire. The results reveal that the optimized structure can replace the traditional forest fire prediction model (Moayedi et al., 2020).

Ghali et al. (2021) used deep learning technology to establish a deep learning convolutional transfer learning feature extraction network. Ghali also explored the correlation of the popular allocation standard of subspace learning and designed the deep convolution and domain adaptive sample classification algorithm. The experimental effect was good (Ghali et al., 2021).

The aforementioned methods obtain forest canopy image information using UAVs or video surveillance; analyze the complex characteristics of smoke, flame, and other images by processing forest fire risk images; and build a forest fire risk monitoring and early warning model. The model can predict the time of fire risk, direction of fire spread, and fire intensity. However, few researchers have studied high-precision and lightweight backpropagation neural network (BPNN) models. The BPNN and SVM algorithm are combined to build the forest fire risk identification algorithm MD-BPNN based on an improved BPNN to improve the efficiency of forest fire risk identification.

The organizational structure of this article is as follows: In section “Improvement of forest fire identification algorithm,” we recognize forest fire risk images. In section “Forest fire risk image recognition,” we propose a color segmentation model of forest fire insurance. Section “Color segmentation model of forest fire insurance” introduces the improvement of a BP neural network forest fire identification algorithm and makes an experimental analysis. Finally, section “Conclusion” summarizes the conclusions.

Improvement of forest fire identification algorithm

Steps of the algorithm

Michael et al. (2021) proposed that the BPNN primarily comprises forward signal propagation and reverse error signal propagation. During the forward propagation of the signal, the difference between the output signal and expected output value is calculated. The error signal is transmitted in reverse through the output when a large error occurs, and the value of each layer is modified to make the actual output close to the expected output.

In this study, the image processing steps through the layers are illustrated in Figure 1, and the output of the image recognition result is provided. When the input layer has a reverse propagation method, it may affect the input and output of other layers (Al-Zebda et al., 2021). The image enters the input, hidden, and output layers and is processed by the BPNN. Finally, the processed image is generated in the output layer. The MapReduce fusion deep learning neural network based on BP comprises the input, hidden, and output layers, as depicted in Figure 1. It is trained in two ways: the forward propagation deep learning method and BP deep learning process. The input of the former method affects the other layers. The possible errors in the input data of the output layer are corrected through BP.

FIGURE 1

Figure 1. Modified machine learning algorithm network model structure.

According to the weight vector space, gradient descent technology enhances the search technology and reduces the error rate. In the hidden layer, N is the number of neurons. The hidden layer can be regarded as the input of the output layer, and whether the result is correct can be observed in its evaluation. The series N = [1, 2, 3, ⋯⋯, 10] indicates the largest prediction result found. The series [x₁, x₂, ⋯⋯, x_n] represents input variables. The series [w₁, w₂, ⋯⋯w_n] denotes the weight between the hidden and input layers. Finally, the series [v₁, v₂, ⋯⋯, v_n] is the weight assigned to hidden and output layers. In addition, _Y(t) is a predictable output, and the transfer function is used to describe the nonlinear problem, expressed as follows:

$y=f\left(x\right)=\frac{\mathit{1}}{\mathit{1}+e^{-x}}(1)$

The hidden layer z is the output expressed as follows:

$z=f_{\mathit{1}}\left(\sum w_i{x}_i\right),(2)$

where_{w1,w2,w3……wn} represent the weights of the hidden and input layers:

$u=f_{\mathit{2}}\left(\sum u_k{z}_k\right).(3)$

After the forward propagation process is completed, the error signal e is formed by the tasks of u and y(t) for the purpose of

$u=\mathit{1}/\mathit{2}{\left(\sum y_i{\left(t\right)}_u-z_i\right)}^{\mathit{2}}(4)$

Modified machine learning algorithm based on backpropagation neural network

In this study, the improvement steps of the deep learning BP neural network forest fire identification algorithm are as follows (Kukuk and Kilimci, 2021).

Step 1: Set algorithm parameters, including the initial weight diversity feature (W), that is, the weight between [−w, w] ranges. The lowest initial range here is [−1, 1].

Number of hidden layers: The number is set to 1 in this study.

Number of nodes in the hidden layer: The number of nodes in the hidden layer is less than the number of training samples.

Number of training cycles: One training cycle can improve the accuracy of description by scanning the records of the training set. At the cost of time, the accuracy may be reduced, but the use time will be reduced.

Error tolerance: It specifies that the error tolerance in the restatement is low. In terms of characteristics, error tolerance is a small-cost event with a diversity from 0 to 1.

Hidden layer sigmoid: It generates each hidden node through a sigmoid function. It can transform the continuous real value of the input into the output between 0 and 1.

Serious error: It avoid serious errors in network training (Mohammed, 2022).

Step 2: The output of the active function marked in the render parameters for each layer is calculated.

Activation function: It is responsible for mapping the input of neurons to the output.

Learning rate: The logarithm is kept in the range of [0.5∼0.8] at the beginning of discovery.

Number of training cycles: This is the only scan of all images in the training set.

Step 3: Once an error is found in the output layer, the error between all the obtained outputs and the selected outputs is calculated.

Step 4: The weight extension error gradient is adjusted on each epoch.

Step 5: The deep training mode is used to obtain the output. The depth training mode is expanded from the depth training set, allowing the functions marked in the aforementioned parameter list to be activated.

Step 6: Depth fusion is performed in the estimation function.

Step 7: The incorrect information of deep fusion is ignored, and the model is trained.

Experimental results: In the cluster (about five nodes), an 8.00 GB i3 CPU and 2.8 GHz of RAM are used. A forest fire risk dataset was constructed to compare different methods to predict forest fire risk. The forest fire risk monitoring dataset includes forest fire risk images collected by UAVs on the ground and forest fire risk images searched on the internet. Figure 2 presents the calculation results of the non-sampling fusion-level depth learning using several algorithms, and Figure 3 depicts the calculation results of the fusion-level depth learning perceptron with resampling.

FIGURE 2

Figure 2. Performance comparison of all network models without sampling.

FIGURE 3

Figure 3. Performance comparison of all network models with resampling.

Figure 2 compares the algorithm performance using the network model without resampling. The recognition accuracy of the MD-BPNN algorithm is 0.947, which is the highest compared with the accuracy of other algorithms. The recall rate of the MD-BPNN algorithm is 0.884, and the F-measure is 0.984. The root mean square error (RMSE) is 0.0759. According to Figure 3, the algorithm performance is compared using the network model with resampling. The recognition accuracy of the MD-BPNN algorithm is 0.932, and the recall rate is 0.887. Furthermore, the F-measure is 0.985, and the RMSE is 0.0434.

The data in the table reveal the consistency measurement of the proposed algorithm. The algorithm changes the learning rate between 0.1 and 0.8, consistent with the cross-index verification of the 10x sampling algorithm.

Improved machine learning algorithm and support vector machine classifier

This article presents an improved machine learning algorithm and SVM classifier to recognize forest fire risk images. This method takes the color and shape of the fire image as the criterion and combines a variety of features. This method first extracts the color features of the flame according to the composition of the flame color to reduce some significant interference (Tang et al., 2022). Second, by constructing the multi-dimensional vector of color and shape, the shape characteristic parameters of the target area are calculated. The image recognition steps based on the BPNN are shown in Figure 4.

FIGURE 4

Figure 4. Image recognition steps based on backpropagation neural network.

The input layer of the forest fire risk recognition neural network based on the BPNN is composed of extracted features, including color moments, roundness ρ of the suspicious area, and the angle N of flame.

According to the different backgrounds of forest fire risk images, the distribution of y is y = [1, 0]. y = [0, 1, 0], andy = [0, 1]. According to Equation (5), the node in the hidden layer is 8.

$y=\frac{\frac{\mathit{1}}{\mathit{2}}c\times r^{\mathit{2}}+\frac{\mathit{3}}{\mathit{2}}c\times r-\mathit{1}}{c+r}(5)$

After the number of nodes in the network hidden layer is determined, the forest fire risk image recognition steps are as follows:

Initialization: The input vector is X = [x₁, ⋯, x_n], the output vector is Y = [y₁, ⋯, y_k], and the hidden neuron vector is M = [m₁, ⋯, m_j].

Forward propagation: According to $S_j={\sum }_{\mathit{1}}^i{X}_i{W}_{\mathit{ij}}$, the fitness of hidden layer is $f\left(x\right)=\mathit{tanh}\left(x\right)=\frac{e^x-e^{-x}}{e^x+e^{-x}}$, so the output function of the hidden layer is M_j = f (S _i).

According to $Q_k={\sum }_{\mathit{1}}^j{M}_j{V}_{\mathit{jk}}$, the fitness of the output layer is $f\left(x\right)=\frac{\mathit{1}}{\mathit{1}+e^{-x}}$ and Y(k) = f (Q _k).

Backpropagation: The actual output is Y(k), the expected output is O_k, the mean square error is $E_n=\frac{\mathit{1}}{\mathit{2}}{\sum }_{\mathit{1}}{\left[O\left(k\right)-Y\left(k\right)\right]}^{\mathit{2}}$, and the sum of the mean square error is E = ∑_n=1 E_n. The learning error d_k of the input layer and the learning error d_j of the hidden layer are calculated, and the value of the weight is adjusted until the error disappears and is less than the expected error.

A total of 3,845 forest fire risk images which are taken in Longshan Forest Farm of Shaoguan, Guangdong (23 ° 12′N, 113 ° 22′E), and Lingyun Mountain of Foshan city (22 ° 57′N, 112 ° 46′E), are used as datasets. According to the ratio of 7:3, the datasets are divided into training sets and test sets. The datasets contain the image information of trees, lakes, roads, etc. in the forest environment taken by UAVs from above. The image information is collected under the conditions of sufficient light and low illumination. Longshan Forest Farm is located in Lechang city, Shaoguan, Guangdong Province. The forest farm mainly grows ecological public welfare forests and experimental forests such as Chinese fir, high-fat masson pine, bald cedar, eucalyptus, and rosemary. Lingyun Mountain is located in Gaoming district, Foshan city. Its vegetation consists of mainly masson pine, which is flammable and difficult to extinguish in case of fire. The environment of coniferous forest creates favorable conditions for the generation of crown fire.

Overall, 1,440 images are selected as the training samples, and 288 images are selected as the test samples, and the image samples are collected under the same background. Table 1 shows the parameters of some training samples.

TABLE 1

Table 1. Parameters of some training samples.

Table 2 lists the processing results. The results demonstrate that the image is considered a fire image if the output value exceeds 0.8.

TABLE 2

Table 2. Training sample.

Forest fire risk image recognition

In UAV image forest fire monitoring, image preprocessing technology is an essential link. Due to the influence of environmental factors such as illumination and image background, the image acquisition process will reduce the quality and clarity of the collected image, resulting in the inability to truly reflect the details of the image (Sayad et al., 2019; Cawson et al., 2020; Hossain et al., 2020). The purpose of applying image preprocessing technology is to weaken or eliminate useless image information so as to retain and enhance useful information.

In order to better meet the training requirements of the BP neural network model, the original fire image data are converted into small images of the same size (Gaur et al., 2021; Lin et al., 2021; Wang et al., 2021). Preprocessing of image data is divided into the following five steps:

The original forest image data are randomly cropped to 256 × 256 and the image is randomly rotate at −15°∼15°.

The cropped forest fire risk image with size 224 × 224 is transformed into a tensor of 0∼1.

After converting from –1 to 1, the tensor is normalized from 0 to 1.

Image enhancement

In the process of forest fire risk image recognition, the image needs to be enhanced (Abedi Gheshlaghi et al., 2021). The processing algorithm steps are as follows: First, the images are classified and divided into non-overlapping parts.

$\left[\begin{array}{c}{\mathit{Gx}}^{\prime }\hfill \\ {\mathit{Gy}}^{\prime }\hfill \end{array}\right]=\left[\begin{array}{c}\hfill {\mathit{Gx}}^{\mathit{2}}-{\mathit{Gy}}^{\mathit{2}}\hfill \\ \hfill \mathit{2}GxGy\hfill \end{array}\right](6)$

The average gradient vector ${[\overline{{\mathit{Gx}}^{\prime }},\overline{{\mathit{Gy}}^{\prime }}]}^T$ is calculated by using the following equation:

$\left[\begin{array}{l}\overline{G{x}^{\prime }}\\ \overline{G{y}^{\prime }}\end{array}\right]=\left[\begin{array}{l}\frac{1}{w\times w}{\displaystyle {\sum }_{i=1}^w{\displaystyle {\sum }_{i=1}^w\left(G_x^2\left(i,j\right)-G_y^2\left(i,j\right)\right)}}\\ \frac{1}{w\times w}{\displaystyle {\sum }_{i=1}^w{\displaystyle {\sum }_{i=1}^w\left(2G_x\left(i,y\right)G_y\left(i,y\right)\right)}}\end{array}\right](7)$

In Equation (7), the size of the image area is w × w:

$\mathrm{\phi }=\left\{\begin{array}{c}\frac{\mathit{1}}{\mathit{2}}{\mathit{tan}}^{-\mathit{1}}\frac{\overline{G^{\prime }{}_y}}{G^{\prime }{}_x}+\frac{\mathrm{\pi }}{\mathit{2}}{\mathit{tan}}^{-\mathit{1}}\frac{\overline{G^{\prime }{}_y}}{G^{\prime }{}_x}<\mathit{0}\hfill \\ \frac{\mathit{1}}{\mathit{2}}{\mathit{tan}}^{-\mathit{1}}\frac{\overline{G^{\prime }{}_y}}{G^{\prime }{}_x}-\frac{\mathrm{\pi }}{\mathit{2}}{\mathit{tan}}^{-\mathit{1}}\frac{\overline{G^{\prime }{}_y}}{G^{\prime }{}_x}\ge \mathit{0}\hfill \end{array}\right\}(8)$

The frequency of the flame determines the filtering effect (Babu et al., 2019; Sayad et al., 2019). If the frequency is not appropriate, the filtered image will be greatly deformed, resulting in the suppression of some flame structures, so the filtered image has a blank position. The direction window is determined according to the flame direction, and each pixel in the window is projected to the baseline. In order to calculate the flame frequency of the image, it is necessary to calculate the distance between the projected crest and trough. The algorithm steps are as follows (Mao et al., 2018):

We divided the image into non-overlapping subblocks of w × w. Next, we calculated the average value of each point along the w direction, denoted as M[ K ].

Combined with the characteristics of forest fire risk images, this study has used the method of enhancement filtering to enhance the image quality, and the most commonly used enhancement image algorithm is Gabor filtering (Wang et al., 2019). This algorithm regards the image in the local area as a group of parallel, frequency, linear, and fixed direction images. The Gabor window function is used to locate and filter the flame in the image so as to enhance the flame information. It can be expressed as follows:

$G(x,y)=\frac{\mathit{1}}{\mathit{2}\pi {\pi }_x{\mathrm{\sigma }}_y}\mathit{exp}\left[-\frac{\mathit{1}}{\mathit{2}}\left(\frac{x^{\mathit{2}}}{{\mathrm{\sigma }}_x^{\mathit{2}}}+\frac{y^{\mathit{2}}}{{\mathrm{\sigma }}_y^{\mathit{2}}}\right)\right]\mathit{exp}\left(-\mathit{2}\pi \pi \mathit{fix}\right)(9)$

As can be seen from Equation (9), combined with the directional characteristics of the image rotation filter, image information can be enhanced during filtering (Sun et al., 2021).

$R(x,y)=\frac{\mathit{1}}{\mathit{2}\pi {\pi }_x{\mathrm{\sigma }}_y}\mathit{exp}\left[-\frac{\mathit{1}}{\mathit{2}}\left(\frac{x^{\mathit{2}}}{{\mathrm{\sigma }}_x^{\mathit{2}}}+\frac{y^{\mathit{2}}}{{\mathrm{\sigma }}_y^{\mathit{2}}}\right)\right]\mathit{cos}\left(\mathit{2}\pi \pi \mathit{fx}\right)(10)$

$V(x,y)=\frac{\mathit{1}}{\mathit{2}\pi {\pi }_x{\mathrm{\sigma }}_y}\mathit{exp}\left[-\frac{\mathit{1}}{\mathit{2}}\left(\frac{x^{\mathit{2}}}{{\mathrm{\sigma }}_x^{\mathit{2}}}+\frac{y^{\mathit{2}}}{{\mathrm{\sigma }}_y^{\mathit{2}}}\right)\right]\mathit{sin}\left(\mathit{2}\pi \pi \mathit{fx}\right)(11)$

$R(x,y,f,\mathrm{\phi })=\frac{\mathit{1}}{\mathit{2}\pi {\pi }_x{\mathrm{\sigma }}_y}\mathit{exp}\left[-\frac{\mathit{1}}{\mathit{2}}\left(\frac{x_{\mathrm{\phi }}^{\mathit{2}}}{{\mathrm{\sigma }}_x^{\mathit{2}}}+\frac{y_{\mathrm{\phi }}^{\mathit{2}}}{{\mathrm{\sigma }}_y^{\mathit{2}}}\right)\right]\mathit{cos}\left(\mathit{2}\pi {\mathit{fx}}_{\mathrm{\phi }}\right)(12)$

In Equation (12), $\left[\begin{array}{l}{x}_{\mathrm{\phi }}\\ y_{\mathrm{\phi }}\end{array}\right]=\left[\begin{array}{l}cos\mathrm{\phi }sin\mathrm{\phi }\\ -sincos\mathrm{\phi }\end{array}\right]\left[\begin{array}{l}x\\ y\end{array}\right]$, φ is the direction of the Gabor filter, [x_φ, y_φ] represents the angle of rotation along the axes x and y, and f is the frequency of the sine wave and plane wave (Hong et al., 2018). There are two methods to enhance image information: one is the frequency domain method, and the other is the spatial domain method, such as image gray transformation operation, image histogram correction operation, and image filtering operation (Sousa et al., 2020; Joardar et al., 2021). Figure 5 is the effect picture after Gabor filtering, mean filtering, and Gaussian filtering.

FIGURE 5

Figure 5. Effect picture after using various filtering processes: (A) Original drawing, (B) gabor filtering, (C) mean filtering, and (D) gaussian filtering.

Image morphological processing

Morphological processing of forest fire risk image data captured by UAVs can extract image components that are meaningful to the rendered area, realize further discrimination operation, and extract the edge features of the target object and the essential features of the connected area (Devotta et al., 2021).

The basic operations of morphological image processing include expansion, corrosion, open operation, and close operation. In Figure 6A is the original (Figures 6B–F) are the comparison of the effects of several image morphological processing methods, such as binarization processing, expansion processing, corrosion processing, open operation, and closed operation (Wang et al., 2022).

FIGURE 6

Figure 6. Image morphological processing results: (A) Original image, (B) binarization treatment, (C) expansion treatment, (D) corrosion treatment, (E) open operation processing, and (F) closed operation processing.

Through the morphological processing of forest fire risk images, the image noise can be removed, the image shape can be simplified, the flame feature structure can be enhanced, and the flame information can be separated from the complex background.

Color segmentation model of forest fire insurance

In order to obtain a more detailed flame image, the flame pixels can be segmented in YCbCr color space and RGB color space, and the decision conditions can be obtained according to the two color spaces (Stankevich, 2020). In case of forest fire risk, the color of forest fire is quite different from the background color of the forest environment, and the characteristics are obvious. The main performance is that the color distribution of flame from outside to inside is red, yellow, and white. The color distribution of flame is shown in Figure 7.

FIGURE 7

Figure 7. Flame color distribution.

Pixel distribution characteristics of forest fire

In the actual forest fire risk monitoring process, when the shooting time is cloudy, the collected image information is dark. For general algorithms, it is usually difficult to achieve good results after direct processing. In order to improve the contrast effect of the image, the image needs to be preprocessed in advance. As shown in Figures 8A,B, the overall brightness value of the flame image is low. The color parameters of the HSV model include brightness, hue, and saturation (Van Le et al., 2021). The pixel value of the flame image is calculated. As shown in Figures 8C,D, the processed image has an obvious contrast effect and can meet the requirements of later image processing.

FIGURE 8

Figure 8. Flame brightness value: (A) Original image 1, (B) image 1 brightness value, (C) original image 2, and (D) image 2 brightness value.

Distribution of flame pixels

In Figures 9A–D are the flame image and the image separated by YCbCr channels.

This section compares the flame pixels of YCbCr channels with the average pixel value of the channel to describe the characteristics of flame pixels. In Figures 10A–C compare the flame pixel values in YCbCr channels with the average value of the corresponding channels (Mohammed, 2022).

FIGURE 9

Figure 9. YCbCr color space separation image: (A) Original image, (B) Y channel separation, (C) Cb channel separation, and (D) Cr channel separation.

FIGURE 10

Figure 10. Pixel values of flames in different channels: (A) Flame pixel value of Y channel, (B) flame pixel value of Cb channel, and (C) flame pixel value of Cr channel.

As can be seen from Figure 10, in the three channels, the distribution rule of flame pixels can be expressed as follows (Sevinc et al., 2020; Kumar et al., 2021; Dharmawan et al., 2022):

$Y(i,j)>Y_{\mathit{mean}},Y(i,j)>Y_{\mathit{mean}}\mathit{and}\mathit{Cb}(i,j)<{\mathit{Cb}}_{\mathit{mean}}(13)$

where Y_meanCb_meanCr_mean represent the average pixel value.

As shown in Figure 11. The comparison results are as follows:

$R(i,y)>G(i,y)(14)$

FIGURE 11

Figure 11. Value of pixels in the R and G channels.

Color segmentation results and comparison

This study uses the HAF (Equation 17) and MCC (Equation 18) methods to analyze the flame image information under low-light conditions and segment the flame color in the image. Figure 12 presents the results.

FIGURE 12

Figure 12. Segmentation flame image results.

Figure 12 illustrates the segmentation results of the flame image. The left column is the original image, and the central column is the segmented flame image based on the general BPNN algorithm. The right column displays the results from the proposed MD-BPNN algorithm.

The segmentation quality of the region algorithm can be quantified through this index. We define the matching index calculation as follows:

$M=\sum j,{\mathit{max}}_i\mathit{Card}\left(R_i^{\mathit{ref}}\bigcap R_j^{\mathit{seg}}\right)\frac{\mathit{Card}\left(R_i^{\mathit{ref}}\bigcap R_j^{\mathit{seg}}\right)}{\mathit{Card}\left(R_i^{\mathit{ref}}\bigcap R_j^{\mathit{seg}}\right)}\times {\mathrm{\rho }}_j,(15)$

where Card represents the number of pixels, ρ_j denotes the weight, $R_i^{\mathit{ref}}$ represents the ith manually segmented region, and $R_j^{\mathit{seg}}$ denotes the jth region.

For the overlapping part, combined with the over-segmentation problem, the segmentation result region corresponds to the manual segmentation result region, and the following indexes are determined:

$\mathrm{\eta }=\{\begin{array}{c}\frac{{\mathit{NR}}_{\mathit{ref}}}{{\mathit{NR}}_{\mathit{seg}}},\mathit{if}{\mathit{NR}}_{\mathit{seg}}\ge {\mathit{NR}}_{\mathit{ref}}\hfill \\ \mathit{log}\left(\mathit{1}+\frac{{\mathit{NR}}_{\mathit{seg}}}{{\mathit{NR}}_{\mathit{ref}}}\right),\mathit{otherwise}\hfill \end{array}(16)$

The calculation equation of the final evaluation index HAF is as follows:

$\mathit{HAF}=\frac{M+m\times \mathrm{\eta }}{\mathit{1}+m}(17)$

In Equation (17), the weighting factor M plays an important role in judging the segmentation of the process, and its value is 0.5. In Figure 10, the HAF segmentation index comparison is shown in Figure 13.

FIGURE 13

Figure 13. HAF evaluation index results.

From the test data, it can be seen that in the case of excessive segmentation and insufficient segmentation, the average accuracy rate of the MD-BPNN algorithm evaluated by using the HAF index is 80.37%, and the segmentation result still has good anti-interference performance. Compared with other algorithms, the performance is better, which enhances the applicability of the algorithm in multiple scenarios.

The Matthews coefficient is used to evaluate results, calculated as follows (27):

$\mathit{MCC}={\frac{\left(\mathit{TP}\times \mathit{TN}\right)-\left(\mathit{FP}\times \mathit{FN}\right)}{\sqrt{\left(\mathit{TN}+\mathit{FN}\right)\left(\mathit{TN}+\mathit{FP}\right)\left(\mathit{TP}+\mathit{FN}\right)\left(\mathit{TP}+\mathit{FP}\right)}}}_{.}(18)$

In Equation (18), TP indicates a true positive, TN denotes a true negative, FP represents a false positive, and FN indicates a false negative. The MCC evaluation index results are presented in Figure 14.

FIGURE 14

Figure 14. MCC evaluation index results.

By calculating the accuracy of the algorithm, the average accuracy of the MD-BPNN is 92.73%, which is better than other authors’ algorithms.

Conclusion

According to the actual forest image recognition situation, this study improves the combination algorithm of the BP neural network algorithm and SVM classifier, models the recognition network, enhances the image and morphological processing through the training and learning of the recognition network, segments and extracts the features of the suspected forest fire area, improves the efficiency of forest fire recognition, and improves the stability of the network. The main contents are as follows:

(1) A forest fire risk dataset was constructed to compare different classification methods to predict forest fire risk. The results reveal that the algorithm changes the learning rate between 0.1 and 0.8, consistent with the cross-index verification of the 10x sampling algorithm.

(2) In the combination of an improved BP neural network and SVM classifier, forest fire risk is recognized based on feature extraction and a backpropagation network. A total of 1,450 images are used as training samples. The experimental results show that the recognition effect of fire risk images is good.

(3) By analyzing the forest fire image, the flame pixel values of the images on R and G channels are analyzed. The value of the former is higher than that of the latter.

(4) For the flame image under low lighting conditions, HAF and MCC indexes are used to evaluate the segmentation accuracy of the forest fire image. In the case of excessive segmentation and insufficient segmentation, the average accuracy of the MD-BPNN algorithm evaluated by using the HAF index is 80.37% and the segmentation result still has good anti-interference performance, thus enhancing the applicability of the algorithm in a variety of scenarios. The average accuracy of the MD-BPNN algorithm is 92.73%, which indicates that the algorithm has high accuracy.

The improved deep learning algorithm improves the efficiency of forest fire risk identification. However, there is still room to improve model performance. In future research, we will further optimize the performance of the algorithm and improve the ability of forest fire risk identification and prevention.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Author contributions

SZ and WW: conceptualization. SZ, PG, WW, and XZ: methodology and investigation. SZ and PG: software and visualization. SZ, WW, and XZ: validation and resources. SZ, PG, and XZ: formal analysis. XZ and PG: data curation. SZ: writing—original draft preparation. WW and XZ: writing—review and editing and project administration. WW, XZ, and PG: supervision. All authors read and agreed to the published version of the manuscript.

Funding

This work was supported by the Key Field R&D Program Project of Guangdong Province, China (Grant No. 2019B020223003), Research and Application of Key Technologies of Autonomous Robots for Complex Environments (Grant No. 2120001008424), and the Guangzhou Science and Technology Plan Project Innovation Platform Construction and Sharing (Grant No. 201605030013).

Acknowledgments

We thank the reviewers for their critical comments and suggestions for improving the manuscript.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Abedi Gheshlaghi, H., Feizizadeh, B., Blaschke, T., Lakes, T., and Tajbar, S. (2021). Forest fire susceptibility modeling using hybrid approaches. Trans. GIS. 25, 311–333. doi: 10.1111/tgis.12688

CrossRef Full Text | Google Scholar

Al-Zebda, A. K., Al-Kahlout, M. M., Ghaly, A. M. A., and Mudawah, D. Z. (2021). Predicting forest fires using meteorological data: An ann approach. Gaza: International Journal of Academic Information Systems Research (IJAISR).

Google Scholar

Babu, K. V. S., Kabdulova, G., and Kabzhanova, G. (2019). Developing the forest fire danger index for the country kazakhstan by using geospatial techniques. J. Environ. Inform. Lett. 1, 48–59. doi: 10.3808/jeil.201900006

CrossRef Full Text | Google Scholar

Balling, J., Verbesselt, J., De Sy, V., Herold, M., and Reiche, J. (2021). Exploring archetypes of tropical fire-related forest disturbances based on dense optical and radar satellite data and active fire alerts. Forests 12:456. doi: 10.3390/f12040456

CrossRef Full Text | Google Scholar

Cai, W. H., Liu, Z., Yang, Y. Z., and Yang, J. (2018). Does environment filtering or seed limitation determine post-fire forest recovery patterns in boreal larch forests? Front. Plant Sci. 9:1318. doi: 10.3389/fpls.2018.01318

PubMed Abstract | CrossRef Full Text | Google Scholar

Cawson, J. G., Hemming, V., Ackland, A., Anderson, W., Bowman, D., Bradstock, R., et al. (2020). Exploring the key drivers of forest flammability in wet eucalypt forests using expert-derived conceptual models. Landsc. Ecol. 35, 1775–1798. doi: 10.1007/s10980-020-01055-z

CrossRef Full Text | Google Scholar

Devotta, S., Chelani, A., and Vonsild, A. (2021). Prediction of flammability classifications of refrigerants by artificial neural network and random forest model. Int. J. Refrig. 131, 947–955. doi: 10.1016/j.ijrefrig.2021.07.021

CrossRef Full Text | Google Scholar

Dharmawan, A., Harjoko, A., and Adhinata, F. D. (2022). Region-based annotation data of fire images for intelligent surveillance system. Data Brief 41:107925. doi: 10.1016/j.dib.2022.107925

PubMed Abstract | CrossRef Full Text | Google Scholar

Gaur, S., Chaturvedi, S., and Tanwar, R. (2021). Forest fire analysis. soft computing for intelligent Systems. Berlin: Springer, 331–348. doi: 10.1007/978-981-16-1048-6_25

CrossRef Full Text | Google Scholar

Ghali, R., Akhloufi, M. A., Jmal, M., Mseddi, W. S., and Attia, R. (2021). Wildfire segmentation using deep vision transformers. Remote Sens. 13:3527. doi: 10.3390/rs13173527

PubMed Abstract | CrossRef Full Text | Google Scholar

Han, J., Shen, Z., Li, Y., Luo, C., Xu, Q., Yang, K., et al. (2018). Beta diversity patterns of post-fire forests in central Yunnan Plateau, southwest China: Disturbances intensify the priority effect in the community assembly. Front. Plant Sci. 9:1000. doi: 10.3389/fpls.2018.01000

PubMed Abstract | CrossRef Full Text | Google Scholar

Hong, H., Tsangaratos, P., Ilia, I., Liu, J., Zhu, A.-X., and Xu, C. (2018). Applying genetic algorithms to set the optimal combination of forest fire related variables and model forest fire susceptibility based on data mining models. The case of Dayu County, China. Sci. Total Environ. 630, 1044–1056. doi: 10.1016/j.scitotenv.2018.02.278

PubMed Abstract | CrossRef Full Text | Google Scholar

Hossain, F. M. A., Zhang, Y. M., and Tonima, M. A. (2020). Forest fire flame and smoke detection from UAV-captured images using fire-specific color features and multi-color space local binary pattern. J. Unmanned Veh. Syst. 8, 285–309. doi: 10.1139/juvs-2020-0009

CrossRef Full Text | Google Scholar

Hu, J., Niu, H., Carrasco, J., Lennox, B., and Arvin, F. (2022). Fault-tolerant cooperative navigation of networked UAV swarms for forest fire monitoring. Aerosp. Sci. Technol. 123,:107494. doi: 10.1016/j.ast.2022.107494

CrossRef Full Text | Google Scholar

Joardar, B. K., Doppa, J. R., Li, H., Chakrabarty, K., and Pande, P. P. (2021). Learning to train CNNs on faulty ReRAM-based manycore accelerators. ACM Trans. Embed. Comput. Syst. 20, 1–23. doi: 10.1145/3476986

CrossRef Full Text | Google Scholar

Kukuk, S. B., and Kilimci, Z. H. (2021). Comprehensive analysis of forest fire detection using deep learning models and conventional machine learning algorithms. Int. J. Comput. Exp. Sci. Eng. 7, 84–94. doi: 10.22399/ijcesen.950045

CrossRef Full Text | Google Scholar

Kumar, S. G. P. M., Ammal, M. N., Thrinath, V. S., and Rohith, G. S. (2021). Forest fire detection and estimation in aerial images for fire fighting using vision based technology. Element. Educ. Online 20, 2398–2405.

Google Scholar

Lim, C. H., Kim, Y. S., Won, M., Kim, S. J., and Lee, W. K. (2019). Can satellite-based data substitute for surveyed data to predict the spatial probability of forest fire? A geostatistical approach to forest fire in the Republic of Korea. Geomat. Nat. Hazards Risk 10, 719–739. doi: 10.1080/19475705.2018.1543210

CrossRef Full Text | Google Scholar

Lin, G., Zhu, L., Li, J., Zou, X., and Tang, Y. (2021). Collision-free path planning for a guava-harvesting robot based on recurrent deep reinforcement learning. Comput. Electron. Agric. 188:106350. doi: 10.1016/j.compag.2021.106350

CrossRef Full Text | Google Scholar

Mao, W., Wang, W., Dou, Z., and Li, Y. (2018). Fire recognition based on multi-channel convolutional neural network. Fire Technol. 54, 531–554. doi: 10.1007/s10694-017-0695-6

CrossRef Full Text | Google Scholar

Michael, Y., Helman, D., Glickman, O., Gabay, D., Brenner, S., and Lensky, I. (2021). Forecasting fire risk with machine learning and dynamic information derived from satellite vegetation index time-series. Sci. Total Environ. 764:142844. doi: 10.1016/j.scitotenv.2020.142844

PubMed Abstract | CrossRef Full Text | Google Scholar

Moayedi, H., Mehrabi, M., Bui, D. T., Pradhan, B., and Foong, L. K. (2020). Fuzzy-metaheuristic ensembles for spatial assessment of forest fire susceptibility. J. Environ. Manage. 260:109867. doi: 10.1016/j.jenvman.2019.109867

PubMed Abstract | CrossRef Full Text | Google Scholar

Mohammed, K. (2022). R A real-time forest fire and smoke detection system using deep learning. Int. J. Nonlinear Anal. Appl. 13, 2053–2063.

Google Scholar

Ryan, T., Merritt, D. J., and Miller, B. P. (2021). Environmental factors driving seed hydration status of soil seed banks and the implications for post-fire recruitment. Front. Plant Sci. 12, 795003–795003. doi: 10.3389/fpls.2021.795003

PubMed Abstract | CrossRef Full Text | Google Scholar

Sayad, Y. O., Mousannif, H., and Al Moatassime, H. (2019). Predictive modeling of wildfires: A new dataset and machine learning approach. Fire safety J. 104, 130–146. doi: 10.1016/j.firesaf.2019.01.006

CrossRef Full Text | Google Scholar

Sevinc, V., Kucuk, O., and Goltas, M. A. (2020). Bayesian network model for prediction and analysis of possible forest fire causes. Forest Ecol. Manage. 457:117723. doi: 10.1016/j.foreco.2019.117723

CrossRef Full Text | Google Scholar

Sousa, M. J., Moutinho, A., and Almeida, M. (2020). Thermal infrared sensing for near real-time data-driven fire detection and monitoring systems. Sensors 20:6803. doi: 10.3390/s20236803

PubMed Abstract | CrossRef Full Text | Google Scholar

Stankevich, T. S. (2020). Forest fire hazard assessment by clustering and using neural networks under unstability and uncertainty. Vestnik of Astrakhan State Technical University. Series Manage. Comput. Sci. Inform. 2, 56–69. doi: 10.24143/2072-9502-2020-2-56-69

CrossRef Full Text | Google Scholar

Sun, L., Xu, C., He, Y., Zhao, Y., Xu, Y., Rui, X., et al. (2021). Adaptive Forest fire spread simulation algorithm based on cellular automata. Forests 12:1431. doi: 10.3390/f12111431

CrossRef Full Text | Google Scholar

Tang, Y., Zhu, M., Chen, Z., Wu, C., Chen, B., Li, C., et al. (2022). Seismic performance evaluation of recycled aggregate concrete-filled steel tubular columns with field strain detected via a novel mark-free vision method. Structures 37, 426–441. doi: 10.1016/j.istruc.2021.12.055

CrossRef Full Text | Google Scholar

Thach, N. N., Ngo, D. B. T., Xuan-Canh, P., Hong-Thi, N., Thi, B. H., and Nhat-Duc, H. (2018). Spatial pattern assessment of tropical forest fire danger at Thuan Chau area (Vietnam) using GIS-based advanced machine learning algorithms: A comparative study. 46, 74–85. doi: 10.1016/j.ecoinf.2018.05.009

CrossRef Full Text | Google Scholar

Van Hoang, T., Chou, T. Y., Fang, Y. M., Nguyen, N. T., Nguyen, Q. H., Xuan Canh, P., et al. (2020). Mapping forest fire risk and development of early warning system for NW Vietnam using AHP and MCA/GIS methods. Appl. Sci. 10:4348. doi: 10.3390/app10124348

CrossRef Full Text | Google Scholar

Van Le, H., Hoang, D. A., Tran, C. T., Tran, C. T., Nguyen, P. Q., Tran, V. H. T., et al. (2021). A new approach of deep neural computing for spatial prediction of wildfire danger at tropical climate areas. Ecol. Inform. 63:101300. doi: 10.1016/j.ecoinf.2021.101300

CrossRef Full Text | Google Scholar

Wang, C., Luo, T., Zhao, L., Tang, Y., and Zou, X. (2019). Window zooming–based localization algorithm of fruit and vegetable for harvesting robot. IEEE Access. 7, 103639–103649. doi: 10.1109/ACCESS.2019.2925812

CrossRef Full Text | Google Scholar

Wang, H., Dong, L., Zhou, H., Luo, L., Lin, G., Wu, J., et al. (2021). YOLOv3-Litchi detection method of densely distributed litchi in large vision scenes. Math. Probl. Eng. 2021:8883015. doi: 10.1155/2021/8883015

CrossRef Full Text | Google Scholar

Wang, H., Lin, Y., Xu, X., Chen, Z., Wu, Z., and Tang, Y. (2022). A study on long–close distance coordination control strategy for litchi picking. Agronomy 12:1520. doi: 10.3390/agronomy12071520

CrossRef Full Text | Google Scholar

Zhang, G., Wang, M., and Liu, K. (2019). Forest fire susceptibility modeling using a convolutional neural network for Yunnan province of China. Int. J. Disast. Risk Sci. 10, 386–403. doi: 10.1007/s13753-019-00233-1

CrossRef Full Text | Google Scholar