A Novel Method for Extracting Time Series Information of Deformation Area of a Single Landslide Based on Improved U-Net Neural Network

Dai, Bibo; Wang, Yunmin; Ye, Chunyang; Li, Qihang; Yuan, Canming; Lu, Song; Li, Yuyang

doi:10.3389/feart.2021.785476

BRIEF RESEARCH REPORT article

Front. Earth Sci., 03 December 2021

Sec. Geohazards and Georisks

Volume 9 - 2021 | https://doi.org/10.3389/feart.2021.785476

This article is part of the Research Topic Monitoring, Early Warning and Mitigation of Natural and Engineered Slopes View all 15 articles

A Novel Method for Extracting Time Series Information of Deformation Area of a Single Landslide Based on Improved U-Net Neural Network

Bibo Dai^1,2,5

Yunmin Wang^3,5

Chunyang Ye³

Qihang Li³

Canming Yuan³

Song Lu^4,6^*

Yuyang Li³

¹School of Resources and Civil Engineering, Northeastern University, Shenyang, China
²Geomechanics Research Center, Northeastern University, Shenyang, China
³School of Resources and Environmental Engineering, Jiangxi University of Science and Technology, Ganzhou, China
⁴Quanzhou Institute of Equipment Manufacturing, Haixi Institutes, Chinese Academy of Sciences, Quanzhou, China
⁵State Key Laboratory of Safety and Health for Metal Mines, Maanshan, China
⁶Zhejiang Zhipu Engineering Technology Corporation Limited, Huzhou, China

This paper proposed an improved U-Net fully convolutional neural network to automatically extract a single landslide deformation information under time series based on the physical model experiments. This method extracts time series information for three different landslide deformation ranges. Compared to U-Net and mainstream superpixel method, evaluation indicators of DSC, VOE and RVD verify the high recognition accuracy and strong robustness of our method.

Introduction

Landslides usually cause a large number of casualties and property losses (Lee et al., 2006; Shahabi et al., 2013; Shirzadi et al., 2017), so it is very important to obtain characteristic information when landslides occur (Guzzetti et al., 1994; Malamud et al., 2004; Lee et al., 2018). Some scholars have done a lot of research on it from different angles (Kurtz et al., 2014; Tang et al., 2014; Zhang et al., 2016; Zhang et al., 2020; Li et al., 2021a; Li et al., 2021b). Computer vision technology such as image transformation algorithms improved the application of image recognition in landslide mapping (Ardizzone et al., 2007; Cheng et al., 2013; Mwaniki et al., 2017; Bui et al., 2020; Shi et al., 2020). While the image recognition of landslide deformation area in specific time deformation area of a single landslide is few considered.

In the mainstream automatic recognition of Superpixel segmentation, Xie et al., proposed a method of super pixel generation for SAR images based on significant differences and spatial distance (Xie et al., 2019). Zhu et al., proposed a region merging method (Zhu et al., 2016). Hashiba et al., uses the superpixel SLICO method to check the appropriate area size to extract the landslide area with high accuracy (Hashiba and Sonobe, 2020). Yang et al., used superpixel algorithm to realize the automatic extraction of landslide deformation information (Yang et al., 2019). But these methods are lacking in accuracy and recognition speed. Some scholars use u-net neural network to extract landslide deformation information (Ghorbanzadeh et al., 2021). Sanghoon Lee et al., used U-Net to quantitative spatial analysis on whole slide images (Lee et al., 2020). Chen Yu et al., developed a U-net like model suitable for mapping post-earthquake lanslide susceptibility (Chen et al., 2020). Unfortunately, these methods cannot achieve both high accuracy and strong robustness for recognition of landslide deformation area.

In this study, we propose an automatic recognition method for landslide deformation characteristics based on improved U-Net neural network. Compared with U-Net neural network and mainstream superpixel methods, it has higher accuracy and robustness in identifying landslide deformation features under time series. Physical simulated landslide experiments verify the reliability of our method.

Methods

The U-Net network was first proposed by Olaf et al. to segment images of biological cells. It is well known for its small training set. U-Net consists of two parts: a contraction path and an expansion path. The contraction path includes downsampling, ReLU, and pooling; the expansion path includes upsampling and ReLU (Ronneberger et al., 2015). This study improves the U-Net network by: 1) that increasing the depth of the network is conducive to extracting deeper features of the image (He et al., 2015); 2) that replacing part of the traditional convolution kernel using a hollow convolution kernel increase the receptive field to preserve more local details of the picture (Wang et al., 2016). Table 1 lists the specific operations of each layer of the improved U-Net network. The improved U-Net contains 9 improved residual modules, each of which is composed of two hole convolutions, two BNs and two ReLU functions; 19 convolutional layers, four pooling layers and four transposed convolutional layers.

TABLE 1

TABLE 1. Architectural details of the improved U-Net.

Data Set and Experimental Environment

The data used in this study was from indoor rainfall-induced landslides experiments. The physical model device is mainly composed of a rainfall simulation equipment and a set of monitoring sensors. The rainfall simulation system includes a metal frame, a metal support, 15 nozzles, and 5 flowmeters (water velocity was 25–250 ml/min for each flowmeter). The length, width, and height of the metal frame are 1.6, 0.8, and 0.8 m respectively (Figure 1). The main landslide monitoring sensor in our study is a trinocular camera (type: Point Gray Bumblebee X3, resolution: 1,280 × 960), which saves the captured images every 5 s. There are five groups of experimental models in the experiment, named S1, S2, S3, S4, and S5. The five groups of physical model experiment materials are all ionic rare earths, and each group of tests rains for 5 h, and the total rainfall is 260 mm. In addition, the back-peak rainfall method uses decreasing rainfall intensity, which are 140 mm/h, 70 mm/h, 40 mm/h, 20 mm/h, and 10 mm/h respectively. On the other hand, the front-peak rainfall method uses incremental rainfall intensity, which are 10 mm/h, 20 mm/h, 40 mm/h, 70 mm/h and 140 mm/h respectively (Li et al., 2020). In these five sets of experiments, due to the different physical model building methods and rainfall methods, the forms of slope landslide damage are also different. These experiments provide more data support for later method validation.

FIGURE 1

FIGURE 1. Rainfall device model.

Test Procedure

In this part, we use an improved U-Net method to extract landslide deformation information (Figure 2), and use currently popular evaluation indicators to evaluate the experimental segmentation results. Since there is no public dataset of continuous deformation images of landslides, we used LabelMe software to make the training set labels in the experiment. This experiment uses a neural network framework based on PyTorch to implement U-Net full convolutional networks. The data in the experiment consists of two parts consisting of 500 pieces of training set and 100 pieces of test set. Each group of tests extracts an average of 100 training sets (30 small-scale damage images, 40 medium-scale damage images, and 30 large-scale damage images) and 20 test sets (6 small-scale damage images, 8 medium-scale damage images and 6 large-scale damage images). In order to better utilize the characteristic of U-Net full convolutional network, this paper keeps the size of each image in the experiment 512 × 512. In the experiment we used i7-6,700 (CPU), NVIDIA RTX 2060s (GPU) in the Windows 10 environment, and the computer’s RAM is 16G. This paper uses the following parameters to set up the U-Net network: learning rate = 1e-4, batch size = 1 and training epoch = 100.

FIGURE 2

FIGURE 2. The flow chart of time sequence information extraction of landslide deformation area.

Evaluation Index

Currently, most of the image evaluation methods are widely used in the fields of medicine and computer vision, and have the characteristics of high accuracy (Huang et al., 2018; Sudan et al., 2019; Kromp et al., 2021). This paper uses DSC, VOE and RVD evaluation indicators to evaluate the segmentation effect of the model (Dash et al., 2019; Liu et al., 2019). DSC is used to evaluate the consistency between the segmentation results and the real results, and is defined as follows:

D S C (G, P) = \frac{2 | G \cap P |}{| G | + | P |}

(3-1)

The VOE calculation method is as follows:

V O E (G, P) = 1 - \frac{| G \cap P |}{| G \cap P |}

(3-2)

The RVD is defined as follows:

R V D (G, P) = \frac{| P | - | G |}{| G |}

(3-3)

Among them, $D S C$ represents the Dice Similariy Coefficient, $V O E$ represents the Volumetric Overlap Error, $R V D$ represents the Relative Volume Difference, $G$ represents the real result and $P$ represents the algorithm segmentation result. The closer DSC is 1, the better the segmentation result. The closer VOE and RVD is 0, the better the segmentation result.

Results

To verify the effectiveness of the proposed method, the proposed method is compared with the original U-Net network and the mainstream landslide deformation region segmentation superpixel algorithm, and the evaluation coefficients DSC, VOE, and RVD mentioned above are used to evaluate the segmentation results.

In this study, three small-scale landslide damage images, four medium-scale landslide damage images, and three large-scale landslide damage images at different times during training are randomly selected. The results of the three algorithms in the task of identifying small-scale landslide damage are shown in Figure 3A. It can be found that the performance of the improved U-Net model is better than that of the original U-Net and superpixel algorithm; The original U-Net model misidentifies the large non-destructive area as the destructive area, while the superpixel algorithm fails to distinguish the deformation area of landslide well. Figure 3B shows the results of the three algorithms in the identification of mesoscale landslide damage areas. Both the U-Net and the improved U-Net algorithms can well identify the damaged area in the landslide. However, the original U-Net method is easy to lose information in some subtle places, and it is easy to cause misrecognition at the edge of the damaged area. The superpixel algorithm can hardly identify the landslide deformation area. To comprehensively analyze and compare the performance of the three algorithms, this paper adds a landslide with a larger damage area to test the algorithm. Figure 3C shows that the U-Net model has obvious misrecognition and missed recognition in the identification of large-scale landslide damage areas, and the improved U-Net model still has a good performance. The traditional landslide deformation area recognition algorithm still does not perform well. The specific data is described below.

FIGURE 3

FIGURE 3. Recognition result graph; (A) Recognition result of small-scale damage; (B) Recognition results in medium-scale damage; (C) Recognition results in large-scale damage.

When small-scale deformation of the landslide occurs, the DSC evaluation indexes of U-Net, Superpixel and Improved U-Net are respectively: 0.953, 0.335, 0.978; VOE evaluation indexes are: 0.088, 0.775, 0.043; RVD evaluation indexes are: 0.066, 0.495, 0.032. When a medium-scale deformation of a landslide occurs, the DSC evaluation indexes of U-Net, Superpixel and Improved U-Net are respectively: 0.871, 0.433, 0.893; VOE evaluation indexes are: 0.084, 0.435, 0.076; RVD evaluation indexes are: 0.089, 0.376, 0.081. When large-scale deformation of the landslide occurs, the DSC evaluation indicators of U-Net, Superpixel and Improved U-Net are respectively: 0.933, 0.335, 0.958; VOE evaluation indicators are: 0.031, 0.775, 0.021; RVD evaluation indicators are: 0.032, 0.495, 0.020.

Discussion

Highlights of Improved U-Net Model

The improved U-Net model is better than the original U-Net and superpixel algorithm. The reason why the superpixel algorithm performs poorly in the large, medium and small-scale destruction is that the algorithm cannot adapt to images with complex backgrounds, and this is the advantage of the deep learning algorithm. In the DSC evaluation index, both U-Net and the improved U-Net model have high indicators. This method uses a large sample training set to automatically and continuously identify and extract deformation features in time series from different deformation scales, the recognition accuracy is high and the robustness is strong under complex environmental conditions.

Limitations and Outlook

In our study, we automatically identify the landslide physical model with relatively simple material particle size in the time series. For some landslide scenarios with complicated background and disordered particle size (such as mine excavation, tailings dam etc.) identification has not been studied yet. In the future, automatic identification of landslide deformation areas in complex environments will be a major trend.

Conclusion

The improved U-Net fully convolutional neural network is used to automatically identify and extract the deformation characteristics of the landslide time series. Some conclusions have been drawn:

1) The increase in depth of the neural network and the introduction of the spatial convolution kernel can effectively extract deep deformation features and retain more local deformation details.

2) In a large-capacity training set, the improved U-Net method has a good batch recognition processing effect.

3) Under the situation of multi-scale landslide damage, three different evaluation indicators verify that the improved U-Net method has higher recognition accuracy and stronger robustness than the U-Net method and the superpixel method.

Data Availability Statement

The raw data supporting the conclusion of this article will be made available by the authors, without undue reservation.

Author Contributions

Drafting of article: BD and SL; planning and supervision of the research: YW; analysis and interpretation of data: CY and QL; acquisition of data: CY; Model construction: YL.

Funding

The work was supported by the National Natural Science Foundation of China (No.51874268), Major science and technology projects of Anhui Province (No.202003a0702002), Fujian Transportation Science and Technology Project (No. 201911) and South Taihu Elite Plan Innovation Team Project of Huzhou city.

Conflict of Interest

Author SL was employed by the company Zhejiang Zhipu Engineering Technology Corporation Limited.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors, and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

We gratefully acknowledge the funding provided by the National Natural Science Foundation of China (No.51874268), Major science and technology projects of Anhui Province (No.202003a0702002), Fujian Transportation Science and Technology Project (No. 201911) and South Taihu Elite Plan Innovation Team Project of Huzhou city. We also thank CY and QL for their assistance in the test. Finally, we would like to thank the handling editor and reviewers whose valuable and constructive comments greatly improved this article.

References

Ardizzone, F., Cardinali, M., Galli, M., Guzzetti, F., and Reichenbach, P. (2007). Identification and Mapping of Recent Rainfall-Induced Landslides Using Elevation Data Collected by Airborne Lidar. Nat. Hazards Earth Syst. Sci. 7 (6), 637–650. doi:10.5194/nhess-7-637-2007