An Intelligent Analysis Method for 3D Wheat Grain and Ventral Sulcus Traits Based on Structured Light Imaging

The wheat grain three-dimensional (3D) phenotypic characters are of great significance for final yield and variety breeding, and the ventral sulcus traits are the important factors to the wheat flour yield. The wheat grain trait measurements are necessary; however, the traditional measurement method is still manual, which is inefficient, subjective, and labor intensive; moreover, the ventral sulcus traits can only be obtained by destructive measurement. In this paper, an intelligent analysis method based on the structured light imaging has been proposed to extract the 3D wheat grain phenotypes and ventral sulcus traits. First, the 3D point cloud data of wheat grain were obtained by the structured light scanner, and then, the specified point cloud processing algorithms including single grain segmentation and ventral sulcus location have been designed; finally, 28 wheat grain 3D phenotypic characters and 4 ventral sulcus traits have been extracted. To evaluate the best experimental conditions, three-level orthogonal experiments, which include rotation angle, scanning angle, and stage color factors, were carried out on 125 grains of 5 wheat varieties, and the results demonstrated that optimum conditions of rotation angle, scanning angle, and stage color were 30°, 37°, black color individually. Additionally, the results also proved that the mean absolute percentage errors (MAPEs) of wheat grain length, width, thickness, and ventral sulcus depth were 1.83, 1.86, 2.19, and 4.81%. Moreover, the 500 wheat grains of five varieties were used to construct and validate the wheat grain weight model by 32 phenotypic traits, and the cross-validation results showed that the R² of the models ranged from 0.77 to 0.83. Finally, the wheat grain phenotype extraction and grain weight prediction were integrated into the specialized software. Therefore, this method was demonstrated to be an efficient and effective way for wheat breeding research.

Introduction

Wheat is one of the most important food crops in the world, which is the main food of approximately 40% global population (Hawkesford et al., 2013). With the rapid growth of the world population, wheat breeding and yield improvement are urgent to meet the growing demand for wheat (Reynolds et al., 2011; Fan et al., 2012; Ray et al., 2012; Lobos et al., 2017). It is well known that the weight of wheat grain would determine the final yield, and the grain shape characters that include grain length, width, thickness, volume are closely related to grain weight, which directly affects the yield and quality of wheat (Furbank and Tester, 2011; Reynolds et al., 2011). Increasing grain size and weight are the important research fields in wheat breeding (Tian, 2002). Meanwhile, the ventral sulcus, the unique morphological feature of wheat, is an important factor to the flour yield, which determines the wheat quality (Dziki and Laskowski, 2005; Le et al., 2019). Therefore, the measurements of wheat grain shape and ventral sulcus traits are significant and necessary to the wheat yield and quality-related research.

The traditional methods of grain measurement mainly rely on manual operation, which is inefficient, subjective, and labor intensive. With the increasing application of machine vision technology in agriculture, the method of phenotypic characters measurement based on image analysis has many advantages, such as high efficiency, high consistency, and non-destructive (Das Choudhury et al., 2019; Yang et al., 2020; Liang et al., 2021). At present, some open source software for image-based grain information measurement has been reported, such as SmartGrain (Tanabata et al., 2012), ImageJ (Schindelin et al., 2015), and WinSEEDLE (Ayele et al., 2017). In addition, Duan et al. (2015) developed a high-throughput automatic analysis system for rice yield-related traits, which includes total grain number, real grain number, 1,000 grain weight, grain length, and grain width. Komyshev et al. (2016) had realized the counting of wheat grains and the calculation of morphological traits on the mobile terminal. Huang and Chien (2017) developed image processing software based on the rice seeds RGB images to extract seven morphological features and classified three rice varieties. However, the above grain phenotype platforms all used three-dimensional (3D) imaging methods, which could only obtain the projected image information such as length, width, and area, while the grain thickness and volume were not able to be extracted. Moreover, the wheat ventral sulcus was complicated and difficult to measure, while the existing measuring method was based on the sliced measurement, which was destructive and unrepeatable, and it would be detrimental to the subsequent research. Therefore, it is urgent to develop a new method for wheat grain 3D and ventral sulcus traits measurement with non-destructive and high precision.

In the recent years, 3D imaging technology was developing rapidly, in which structured light imaging and X-ray computed tomography (CT) imaging technology were able to reconstruct the 3D structure with high precision. Structured light imaging was an active 3D vision technology and could obtain high-precision surface point clouds, which was widely used in industrial inspection, reverse engineering, and cultural relics protection (Paulus, 2019). CT imaging technology was originally used in the medical field, while it was also adopted in the non-destructive observation of small 3D structures such as roots and stems in crop phenotypes. Based on the 3D scanner, Yan and Long (2016) could obtain the corn seed boundary information with an automatic filling algorithm. Le et al. (2019) proposed a method to observe the wheat grains filling based on X-ray μCT imaging technique, while the ventral sulcus was also measured to describe the wheat grain development. Since the wheat grain size was distributed in 6–10 mm and the ventral sulcus was in sub-millimeter (Sun et al., 2007), the spatial resolution of 3D imaging should reach the micron level, and both the techniques could meet the demand (Strange et al., 2015; Liu and Wang, 2021). However, compared with CT 3D reconstruction, structured light imaging had the advantages of lower cost and high acquisition efficiency.

In this paper, an intelligent analysis method based on the structured light imaging had been proposed to extract the wheat grain 3D phenotypes and ventral sulcus traits. First, the 3D point cloud data of wheat grain were obtained by structured light scanner, and then, the specified point cloud processing algorithms that include single grain segmentation and ventral sulcus location have been designed. Finally, an integrated user software was developed for 32 wheat grain traits, which would contribute to the wheat grain-related research.

Materials and Methods

Materials

In this study, five commercial wheat varieties that have large differences in 1,000 grain weight were selected, which are generally distributed in 32–50 g. As shown in Figure 1, the varieties of wheat experimental materials include the following: Jimai 22, Ji 5265, Luyuan 118, Jinchun 6, and Jinmai 163. To evaluate the method accuracy, 25 seeds were randomly selected from each variety, and a total of 125 seeds were used for orthogonal experiment, while the comparison between structured light imaging and X-ray CT imaging was also conducted. In the prediction experiment of grain weight, 100 grains were selected from each wheat variety, and a total of 500 grains were used as the experimental materials.

FIGURE 1

Figure 1. Experimental materials, including Jimai 22, Ji 5265, Luyuan 118, Jinchun 6, and Jingmai 163.

Experiment System

The 3D structured light scanner (Reeyee Pro, China) was adopted in the study, which was based on the white light LED raster scanning technology. Combing the advantages of structured light and binocular stereo vision, the scanner was able to achieve a single-sided accuracy of 0.05 mm within 2 s, which was suitable for high-precision scanning of small-sized work pieces. The equipment was mainly composed of a projector, two cameras, and an internal modulated light source. Based on the principle of triangulation and sinusoidal grating image, the dense point cloud data were able to be obtained. According to the different feature points of the scanned point clouds, the point clouds between various perspectives were merged to obtain the complete wheat grain. The scanner performance is shown in Table 1.

TABLE 1

Table 1. Detailed parameters of Reeyee Pro scanner.

Experiment Design

Orthogonal Experiment

Because Reeyee Pro scanner was based on the white light grating and triangulation principle, the background color of stage, scanning angle, and rotation angle of turntable would affect the reconstruction. To determine the best experimental conditions, 125 grains from 5 wheat varieties were used as the experimental materials. A total of three levels of orthogonal experiments (Huynh, 2011) were carried out on the three influencing factors that include background color, rotation angle, and scanning angle. The influence degree of each factor on the experiment was evaluated by four parameters: length, width, thickness, and ventral sulcus depth. As shown in Figure 2, the different levels of each factor are included (1) background color: black, red, and white; (2) rotation angle: 30°, 45°, and 60°; (3) scanning angle: 30°, 37°, and 45°.

FIGURE 2

Figure 2. Orthogonal experiment design. (A) Wheat stages in 3 different colors. (B) Rotation method in different angle. (C) Scanning method in three different angles. (D) Wheat grain point clouds by structure light imaging.

X-Ray Computed Tomography Contrast Experiment

To compare the result of structured light and X-ray CT imaging, the YXLON FF35 CT system was used in the contrast test. The system has adopted a water-cooled 220 kv output ray tube. Based on the spiral CT scanning mode, the pixel resolution of X-ray transmission image could reach 150 μm. First, the wheat grain was fixed in a foam stick. With the tube voltage of 60 kV and the tube current of 300 μA, 1,440 projection images were taken for X-ray CT rotation scanning of grains. Then, the projection image was used to reconstruct the wheat grain point cloud, which was shown in Figure 3. The projection image acquired by the X-ray detector had image size of 1,430 × 1,430 pixels and pixel size of 150 μm. According to the distance ratio of the X-ray tube to the detector and rotating table, the image magnification was about 5.94, whereas the spatial resolution was 25.23 μm per pixel. More performance parameters are shown in Supplementary Appendix 1. After the same point cloud processing scheme, the length, width, and thickness were calculated, whereas the X-ray CT measurement would be compared with the structured light measurement.

FIGURE 3

Figure 3. X-ray CT 3D reconstruction of wheat grains. (A) Projected image. (B) 3D reconstruction result. (C) Single grain point clouds.

Wheat Grain Point Cloud Processing Pipeline

After obtaining the original point clouds of wheat grain, the processing pipeline is depicted in Figure 4. It could be divided into three main steps: point cloud segmentation, phenotypic parameter calculation, and parameter evaluation and grain weight prediction, in which the point cloud processing algorithms that include downsampling, coordinate transformation, plane fitting, region growing, greedy projection, parameter extraction, and grain weight prediction were applied. In addition, the detailed processing procedures were described in the following.

FIGURE 4

Figure 4. Wheat grain analysis pipeline.

Since the dense point cloud was obtained by the structured light scanner (Figure 5A), to improve the computational efficiency of the subsequent algorithm, the voxel grid sample method was adopted (Chiang and Kuo, 2012). Meanwhile, to facilitate the subsequent object plane segmentation, the coordinate transformation of the initial point cloud was carried out, the process of which was shown in formula 1–3. First, the translation matrix A was computed by the centroid coordinate of the original point clouds, and then, the original wheat point cloud coordinate (T₀) was moved to the centroid point with matrix A. After that, the covariance matrix (M_T) was constructed based on the principal component analysis (PCA) (Wold et al., 1987), and the eigenvector was solved as the main direction of the wheat point cloud. Rotation and translation of the initial point cloud were carried out according to the centroid coordinates and the main direction of the original wheat point cloud to generate new coordinates. The converted result is shown in Figure 5B.

$T_A=M_T\times (T_0-A)(1)$

$A=\frac{{\sum }_{i=1}^N{p}_i}{N}(2)$

$M_T={\{e_1,e_2,e_3\}}^T(3)$

where, P_i is the i-th point coordinate of the original point clouds, and N is the number of points; A is the translation matrix; T₀ is the original point cloud coordinates; T_A is the new coordinates after coordinate transformation; Then, e₁, e₂, e₃ are the three-unit eigenvectors of the covariance matrixM_T.

FIGURE 5

Figure 5. Wheat grain point cloud processing and trait extraction scheme. (A–K) Indicated the process of each step.

After downsampling and coordinate conversion, the point clouds contained both grains and stage, which needed to be separated. The object stage point cloud could be regarded as a plane, which could be estimated by the plane fitting based on random sampling consistency (RANSAC) algorithm (Schnabel et al., 2007). Then, the original point clouds were segmented to obtain wheat grains point cloud, as shown in Figure 5C. With segmented grain point clouds, the discrete point removal was applied, and then, the multi grain point cloud was divided into single grain by region growth algorithm (Date et al., 2014). After that, the pseudo-colors were used to distinguish different wheat grains as shown in Figure 5D.

Phenotypic Character Extraction

Length, Width, and Thickness Extraction

The extraction of length, width, and thickness was achieved by constructing OBB oriented bounding box from AABB Axis-aligned bounding box (Dimitrov et al., 2009). Based on the PCA algorithm, the segmented single grain point cloud (Figure 5E) was rotated and translated into consistent major axis direction (Figure 5F), and the new coordinate origin was at the geometric center of the grains. The maximum and minimum values of the transformed single grain point cloud in the new coordinate system were calculated as x_max’x_min’y_max’y_min’z_max’z_min, respectively. Finally, the grain length, width, and thickness were computed as following formula.

$l=x_{max}-x_{min}(4)$

$w=y_{max}-y_{min}(5)$

$h=z_{max}-z_{min}(6)$

where, l, w, and h are the length, width, and thickness of a grain, respectively.

Surface Area and Volume

With triangulation process, the grain surface was a triangular mesh composed of multiple triangular patches (Figure 5G). After the coordinates of the three vertices of the triangular patch were obtained, the area of each small triangle was calculated using Helen’s formula (Connelly, 2009) as Formula 7. The grain surface area was computed by the sum of all triangular patches areas as Formula 8. The 3D model of grain was a closed space surrounded by triangular mesh, and its volume V was calculated by the sum of the volumes of convex pentahedron projected by all triangular surfaces on the central plane, while the convex pentahedron volume was computed by the sum of three tetrahedral volumes (Zhang and Chen, 2001).

$s_i=\sqrt{p_i\left(p_i-a_i\right)\left(p_i-b_i\right)\left(p_i-c_i\right)}(7)$

$S_a=\sum _{i=1}^k{s}_i(8)$

where, S_a is surface area of a grain, k is total number of triangles, s_i is area of the i-th triangle, p_i is half the perimeter of the triangle, and a_i, b_i, and c_i represent the length of each side of the triangle.

Projected Area and Perimeter Extraction

In this study, the grain point clouds were projected along three axis directions (Figure 5H), and the projected area and perimeter were obtained as the shape description of grain. The single grain point cloud after coordinate transformation was projected along X-, Y-, and Z-axes direction, respectively. Then, the projection point cloud was reconstructed and the area was calculated based on the greedy projection algorithm (Marton et al., 2009). The projected point clouds consisted of several triangular patches, while the boundary edges were identified by the edge sharing in the triangular patches. Finally, the perimeter was calculated by the sum of boundary edges.

Ventral Sulcus Traits

The measurement of grain ventral groove depth was performed according to the manual method. First, the wheat grain point cloud was intercepted based on through filtering of X-axis direction, the interception interval was set as 0.4 mm, and total of nine slices were extracted, while the central slice was at grain center. The slice point cloud was projected along the X-axis (Figure 5I), and the slice point cloud with the largest convex hull area was selected. The two furthest points in the convex hull were taken as the ventral sulcus edge points shown in Figure 5J. Based on RANSAC circle fitting, the nearest point to the circle center was regarded as the deepest point in ventral sulcus (Figure 5K). Finally, the depth of the ventral sulcus was calculated based on the distance from ventral sulcus deepest point to the line of ventral sulcus edge point. Meanwhile, the ventral sulcus slice point should be reordered based on the polar coordinate angle, and the Euclidean distance of each adjacent point was calculated and summed as the perimeter of the slice. Based on the Helen’s theorem (Connelly, 2009), the area of the triangle formed by the center point of the slice and each adjacent two points was calculated and summed as the area of the ventral sulcus slice. Additionally, the convex hull area subtracted the slice area as the ventral sulcus area.

Parameter Evaluation and Grain Weight Prediction

A total of 32 parameters were extracted to quantitatively describe the wheat grains shape, which were consisted of 16 basic parameters and 16 derived parameters (Table 2). Roundness and sphericity (Shouche et al., 2001) were the two important shape parameters, which were used to characterize the grain shape in 2D and 3D. They were calculated by the formulas 9 and 10. mean absolute percentage error (MAPE), root mean square error (RMSE), and R² were taken as the evaluation criteria in the system accuracy. They were computed as the formulas 11–13. After 500 wheat grains phenotypic datasets were obtained, five different machine learning algorithms, which include multivariable linear regression model, Bayesian ridge model, KNN regression model, random forest regression model, and gradient boosting regression model, were adopted to evaluate the wheat grain weight prediction model.

$c=\frac{4\mathrm{\pi }{S}_0}{C^2}(9)$

$E=\frac{S_e}{S_a}(10)$

$MAPE=\frac{1}{n}\sum _i\frac{\left|x_i-y_i\right|}{x_i}(11)$

$RMSE=\sqrt{\frac{{\sum }_i{(x_i-y_i)}^2}{n}}(12)$

$R^2=1-\frac{{\sum }_i{(x_i-y_i)}^2}{{\sum }_i{(x_i-\overline{y})}^2}(13)$

where, c is the roundness index; C is perimeter of cross-section; S₀ is area of cross-section; E is sphericity; S_e is a sphere surface area of equal the wheat volume; S_a is the wheat surface area, n is the total number of measurements; x_i is the manual measurement results; y_i is the system measurement results; and $\overline{y}$ is the mean of the system measurements.

TABLE 2

Table 2. The extracted 32 phenotypic traits of wheat grain.

Software Design

The core configuration of the workstation used in this study was inter core i5 8300h, gtx1060, and the operating system was Windows 10. The development environment of point cloud processing algorithm was Microsoft Visual Studio 2015, open source C++ point cloud library (PCL), and visualization tool library visualization toolkit (VTK). The commercial software was used including Reeyee-Pro_v2.6.1.0, which was attached to the scanner and displayed the wheat grains point clouds, and VG studio 3.4 for X-ray CT 3D reconstruction.

The wheat 3D traits extraction software is shown in Figure 6, and the interface was developed based on QT 5.9.8 and python 3.7, while the wheat grain point cloud processing algorithms that include point cloud segmentation, the wheat ventral sulcus extraction, grain traits calculation, and grain weight prediction were integrated. After setting the range of clustering points, the segmentation results of single grain would be extracted from the original point clouds, and then, wheat phenotypic characters would be calculated and displayed in the table. Meanwhile, the corresponding grain point cloud and ventral sulcus slice could be exhibited, when the data cell in the table was clicked. Finally, the wheat grain weight was predicted and displayed in the table. The detailed operation procedure of wheat 3D traits extraction software is shown in Supplementary Video 1. Additionally, the download address of the wheat 3D traits extraction software was provided freely for non-commercial research purposes¹.

FIGURE 6

Figure 6. Wheat 3D traits extraction software. (A) Original point clouds. (B) Grain traits and weight prediction. (C) Grain segmentation results. (D) Single grain point clouds. (E) Ventral sulcus slicing results.

Results

Orthogonal Experiment Results

To evaluate each condition in the orthogonal experiment, the length, width, and thickness of 125 wheat grains were measured by three experimenters with vernier caliper, and the, average value was taken as the ground truth. After the measurement, the grains were sliced along X-axis to obtain the maximum cross-section, the top view image of which was acquired by backlight imaging. Then, the ventral sulcus point and two slice edge points were selected manually on the image, and the, ventral sulcus depth was calculated as the ground truth. The MAPE of length, width, thickness, and ventral sulcus was taken as the evaluation of orthogonal experiment. The results of the orthogonal experiment are shown in Table 3, and the original data of orthogonal experiment are shown in Supplementary Table 1, which indicated that the system had the best performance in the condition of the black object stage, 30° rotation angle, and 37° scanning angle.

TABLE 3

Table 3. Orthogonal experimental results based on L9 (3^4) orthogonal table.

Accuracy Analysis for Length, Width, Thickness, and Ventral Sulcus Traits

To evaluate the structured light imaging performance, 125 wheat samples were compared with X-ray CT imaging results, in which the grain traits that include length, width, and thickness were both extracted. The measurement results are shown in Figure 7. The scatter plots of structured light measurement and ground truth are shown in Figures 7A–C, which showed that the R², RMSE, and MAPE for length measurement was 0.8705 and 0.149 mm, and 1.83%, for the width measurement was 0.8046 and 0.076 mm, and 1.86%, and for the thickness measurement was 0.8836 and 0.080 mm, and 2.19%, respectively. Meanwhile, the scatter plots of X-ray CT measurement and ground truth are shown in Figures 7D–F, which showed that the R², RMSE, and MAPE for length measurement was 0.9351 and 0.147 mm, and 1.88%, for the width measurement was 0.9293 and 0.086 mm, and 2.16%, and for the thickness measurement was 0.9565 and 0.082 mm, and 2.41%, respectively. The results demonstrated that the structure light measurement had high consistence with the manual measurement and was able to achieve the similar accuracy compared with X-ray CT measurements. Moreover, without X-ray radiation, the structured light imaging would be more effective in wheat ventral sulcus measurement.

FIGURE 7

Figure 7. The automatic measurements vs. manual measurements of length, width, and thickness using structured light imaging (A–C) and X-ray CT imaging (D–F).

The method of manual cutting was used to obtain wheat grain slices, and then, the wheat slice image was taken by the visible light camera, while the depth, slice area, and slice perimeter of the ventral sulcus slice were extracted as ground truth. The scatter plots between structured light measurement and ground truth are depicted in Figure 8. The results showed that the R², RMSE, and MAPE for depth measurement was 0.9409 and 0.021 mm, and 4.81%, for the slice area measurement was 0.8649 and 0.5199 mm, and 4.02%, and for the slice perimeter measurement was 0.7744 and 0.5265 mm, and 3.66%, respectively. The results demonstrated that this study could provide a non-destructive method for wheat ventral sulcus traits extraction with high accuracy. The original data of structured light imaging, X-ray CT, and manual measurements of 125 wheat grains are shown in Supplementary Table 2.

FIGURE 8

Figure 8. The automatic measurements versus manual measurements of wheat ventral sulcus traits. (A) Ventral sulcus depth. (B) Slice area. (C) Slice perimeter.

Statistical Analysis and Grain Weight Prediction

A total of 32 wheat grain traits were extracted in the study, while the correlation matrix of Pearson’s coefficient among parameters was calculated, and the results are shown in Figure 9. The results showed that there was a high correlation between ventral sulcus depth and slice area, and a weak positive correlation with grain width and thickness. Grain weight had a strong positive correlation with length, width, thickness, volume, and surface area. The wheat grain weight was measured by high precision electronic balance, and the weight of single wheat grain was approximately between 25 and 50 mg. Since the wheat grain weight was of great significance to the final yield, five different regression algorithms were applied in the study to predict grain weight with 500 grains in five wheat varieties, which included multivariable linear regression, Bayesian Ridge (BR), KNN regression, random forest (RF) regression, and gradient boosting regression (GBR). All regression models were performed on Sklearn Tool Kit, and the super parameters were determined by grid search and learning curve method. Then, 10-fold cross-validation method was used to evaluate each model, in which the dataset was randomly divided into 10 parts, while 9 of them were taken as the training set in turn, and the rest as the test set, and the average of the 10 results was computed as the model accuracy. The performance of grain weight prediction models is shown in Table 4, and the GBR model had the best performance, the R², MAPE, and RMSE of which was 0.83 and 3.37%, and 2.45 mg, respectively. The original data of five grain weight prediction models using 500 wheat grains are shown in Supplementary Table 3.

FIGURE 9

Figure 9. The result of wheat grain traits correlation analysis.

TABLE 4

Table 4. The performance of grain weight prediction models.

Efficiency Evaluation

Since the optimal rotation angle of the turntable was 30° determined by the orthogonal experiment, each measurement that includes 25 wheat grains should acquire point clouds 12 times, while each acquisition would take about 14 s, and the entire acquisition would take about 3 min for each measurement. Then, it took about 2 min to apply the point cloud algorithms. Therefore, the measurement of 25 wheat grains took about 5 min in total, and the average efficiency was 12 s per grain. In the X-ray CT contrast experiment of wheat, it takes about 6 min to acquire 1,440 projection images of 25 grains and about 2 min to reconstruct the 3D point clouds. Similarly, the processing of point cloud algorithms costs about 2 min, so the total measurement takes about 10 min, and the average efficiency is 24 s per grain. The efficiency of manual measurement in the experiment was about 120 s per grain, but not including the complex grain traits. Therefore, the efficiency of structured light imaging system was two times of X-ray CT imaging system and 10 times of manual measurement, which demonstrated a non-destructive method with high efficiency.

Discussion

With the rapid growth of the world population, wheat breeding and yield improvement are urgent to meet the growing demand. The wheat grain weight would determine the final yield, and the grain phenotypic traits that include grain length, width, thickness, volume, are closely related to grain weight. Meanwhile, the ventral sulcus, wheat unique morphological feature, is of great significance to the flour yield, which is hard to be measured by conventional methods. Since most wheat breeding research is generally based on the single grain, single grain phenotypic trait measurement that includes 3D shape and ventral sulcus is vital and necessary. At present, most grain phenotype detection methods were based on the two-dimensional images and provided phenotypic information about population seeds (Dubey et al., 2006; Anami et al., 2009). In this paper, an intelligent analysis method based on the structured light imaging has been proposed to extract the 3D wheat grain phenotypes and ventral sulcus traits.

To determine the best experimental conditions, three levels of orthogonal experiments were carried out on the three influencing factors that include background color, rotation angle, and scanning angle. As could be seen from Table 3, the R value of stage color factor was the largest. It showed that the background has the greatest influence on structured light scanning. For the factor of rotation angle, it can be seen that the difference between K₁ and K₂ was significantly greater than that between K₂ and K₃. It could be convinced that blindly increasing the rotation angle was limited to improve the accuracy; on the contrary, it would reduce the measurement efficiency. Similarly, for the scanning angle, the effect was basically the same within a certain range. Finally, it indicated that the structure light system in this study had the best performance in the condition of the black object stage, 30° rotation angle, and 37° scanning angle.

The contrast experiment between structured light imaging and X-ray CT imaging was carried out. Figures 10A–F indicates three main directions of the side view in the wheat grain point cloud, which was generated by the PCA algorithm and coordinate transformation. Although the point cloud density reconstructed by X-ray CT system was significantly higher than that of structured light system, structured light imaging was able to achieve similar accuracy in the length, width, and thickness measurement, which could be demonstrated in Figures 7A–F. Moreover, compared with CT imaging, the structured light imaging had the advantages of higher efficiency, lower cost, and non-radiation. In this study, the structured light imagining system collected 25 wheat grain point clouds at a time and the data acquisition would takes about 3 min for each measurement. Then, it took about 2 min for data analysis. Therefore, the average efficiency of structured light imaging system was 12 s per grain which was about two times of X-ray CT imaging system. In fact, data acquisition and data analysis could be processed in parallel. If only the time spent on data acquisition was calculated, the efficiency was 4,000 grains per day based on the standard of working 8 h a day. It cost about $20,000 to build the structured light imagining system, which was about one-tenth of the CT system. Therefore, this method was demonstrated to be an efficient and cost-effective way for wheat breeding research.

FIGURE 10

Figure 10. Comparison of wheat grain point clouds obtained by X-ray CT and structured light imaging. (A–C) X-ray CT system. (D–F) Structured light system.

Since the wheat grain weight was of great significance to the final yield, five different regression algorithms were applied in the study to predict grain weight with 500 grains in five wheat varieties, which include multivariable linear regression, BR, KNN regression, RF regression, and GBR. As shown in Table 4, R² ranged from 0.77 to 0.83, and the results showed that the integrated regression algorithms that include RF and GBR would significantly promote the R², compared with the other independent regression algorithms, and the R² were greater than 0.8. The GBR had the best performance with R² = 0.83, which had adopted the residual optimization iteration method to improve the wheat grain weight model.

Conclusion

In this study, a novel method for wheat ventral sulcus measurement was proposed based on the structured light imaging. A total of 32 phenotypic characters of wheat grain were extracted by the specialized point cloud processing algorithms, and the prediction model of wheat grain weight was developed. The results showed that the structured light measurement of wheat grain traits, which include the 3D wheat grain phenotypes and ventral sulcus traits, had a high consistence with the manual measurement. Compared with X-ray CT and manual measurement, the efficiency of the method was significantly improved. Based on 32 wheat grain traits, the GBR model was selected to achieve the wheat grain weight prediction. Meanwhile, a specialized software was developed, in which wheat grain phenotype extraction algorithm and optimal grain weight prediction model were integrated. The software could automatically segment single wheat grain, obtain the ventral sulcus slice, and display the results visually. In conclusion, this study has provided a new software that could automatically achieve wheat grain 3D phenotypic traits extraction and grain weight prediction, which was convenient and useful for wheat researcher.

Data Availability Statement

The datasets presented in this study can be found in online repositories. The names of the repository/repositories and accession number(s) can be found in the article/Supplementary Material.

Author Contributions

CH and ZQ designed the research, performed the experiments, analyzed the data, and wrote the manuscript. ZZ and XH performed the experiments. WX, XL, and PS revised the manuscript. WY supervised the project and wrote the manuscript. All authors contributed to the article and approved the submitted version.

Funding

This work was supported by grants from the National Natural Science Foundation of China (U21A20205 and 31600287), the Key Projects of Natural Science Foundation of Hubei Province (2021CFA059), the Fundamental Research Funds for the Central Universities (2662018JC004 and 2021ZKPY006), and the Cooperative funding between Huazhong Agricultural University and Shenzhen Institute of Agricultural Genomics (SZYJY2021005 and SZYJY2021007).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpls.2022.840908/full#supplementary-material

Supplementary Table 1 | The original data of the orthogonal experiment data of 125 wheat grains.

Supplementary Table 2 | The original data of structured light imaging, X-ray CT, and manual measurements of 125 wheat grains.

Supplementary Table 3 | The original data of five grain weight prediction models using 500 wheat grains.

Supplementary Video 1 | Introduce the use of the wheat 3D traits extraction software.

Supplementary Appendix 1 | Parameters of X-ray source and detector.

Supplementary Appendix 2 | Comparison of downsampling accuracy (Luyuan 118).

Footnotes

1. ^ https://pan.baidu.com/s/1ujRy92FgSPfqsZSsFyApxQ

References

Anami, B. S., Savakar, D. G., and Biradar, V. S. (2009). Identification of multiple grain image samples from tray. Int. J. Food Sci. Technol. 44, 2452–2458. doi: 10.1111/j.1365-2621.2009.02034.x

CrossRef Full Text | Google Scholar

Ayele, A., Hequet, E., and Kelly, B. (2017). The impact of fiber maturity on estimating the number of cotton (Gossypium hirsutum L.) fibers per seed surface area. Industr. Crops Prod. 102, 16–22. doi: 10.1016/j.indcrop.2017.03.004

CrossRef Full Text | Google Scholar

Chiang, P.-Y., and Kuo, C. C. J. (2012). Voxel-based shape decomposition for feature-preserving 3D thumbnail creation. J. Visual Commun. Image Represent. 23, 1–11. doi: 10.1016/j.jvcir.2011.07.008

CrossRef Full Text | Google Scholar

Connelly, R. (2009). Comments on generalized heron polynomials and robbins’ conjectures. Dis. Mathe. 309, 4192–4196. doi: 10.1016/j.disc.2008.10.031

CrossRef Full Text | Google Scholar

Das Choudhury, S., Samal, A., and Awada, T. (2019). Leveraging image analysis for high-throughput plant phenotyping. Front. Plant Sci. 10:508. doi: 10.3389/fpls.2019.00508

PubMed Abstract | CrossRef Full Text | Google Scholar

Date, H., Kanai, S., and Kawashima, K. (2014). As-built modeling of piping system from terrestrial laser-scanned point clouds using normal-based region growing. J. Comput. Design Eng. 1, 13–26. doi: 10.7315/jcde.2014.002

CrossRef Full Text | Google Scholar

Dimitrov, D., Knauer, C., Kriegel, K., and Rote, G. (2009). Bounds on the quality of the PCA bounding boxes. Comput. Geometry 42, 772–789. doi: 10.1016/j.comgeo.2008.02.007

CrossRef Full Text | Google Scholar

Duan, L., Huang, C., Chen, G., Xiong, L., Liu, Q., and Yang, W. (2015). Determination of rice panicle numbers during heading by multi-angle imaging. Crop J. 3, 211–219. doi: 10.1016/j.cj.2015.03.002

CrossRef Full Text | Google Scholar

Dubey, B. P., Bhagwat, S. G., Shouche, S. P., and Sainis, J. K. (2006). Potential of artificial neural networks in varietal identification using morphometry of wheat grains. Bio. Eng. 95, 61–67. doi: 10.1016/j.biosystemseng.2006.06.001

CrossRef Full Text | Google Scholar

Dziki, D., and Laskowski, J. A. (2005). Wheat kernel physical properties and milling process. Acta Agrophys. 6, 59–71.

Google Scholar

Fan, M., Shen, J., Yuan, L., Jiang, R., Chen, X., Davies, W. J., et al. (2012). Improving crop productivity and resource use efficiency to ensure food security and environmental quality in China. J. Exp. Bot. 63, 13–24. doi: 10.1093/jxb/err248

PubMed Abstract | CrossRef Full Text | Google Scholar

Furbank, R. T., and Tester, M. (2011). Phenomics–technologies to relieve the phenotyping bottleneck. Trends Plant Sci. 16, 635–644. doi: 10.1016/j.tplants.2011.09.005

PubMed Abstract | CrossRef Full Text | Google Scholar

Hawkesford, M. J., Araus, J. L., Park, R., Calderini, D., Miralles, D., Shen, T., et al. (2013). Prospects of doubling global wheat yields. Food Energy Security 2, 34–48. doi: 10.1002/fes3.15

CrossRef Full Text | Google Scholar

Huang, K. Y., and Chien, M. C. (2017). A novel method of identifying paddy seed varieties. Sensors 17:809. doi: 10.3390/s17040809

PubMed Abstract | CrossRef Full Text | Google Scholar

Huynh, T. V. (2011). Orthogonal array experiment in systems engineering and architecting. Syst. Eng. 14, 208–222. doi: 10.1002/sys.20172

CrossRef Full Text | Google Scholar

Komyshev, E., Genaev, M., and Afonnikov, D. (2016). Evaluation of the seed counter, a mobile application for grain phenotyping. Front. Plant Sci. 7:1990. doi: 10.3389/fpls.2016.01990

PubMed Abstract | CrossRef Full Text | Google Scholar

Le, T. D. Q., Alvarado, C., Girousse, C., Legland, D., and Chateigner-Boutin, A. L. (2019). Use of X-ray micro computed tomography imaging to analyze the morphology of wheat grain through its development. Plant Methods 15:84. doi: 10.1186/s13007-019-0468-y

PubMed Abstract | CrossRef Full Text | Google Scholar

Liang, X., Xu, X., Wang, Z., He, L., Zhang, K., Liang, B., et al. (2021). Stomata scorer: a portable and high-throughput leaf stomata trait scorer combined with deep learning and an improved CV model. Plant Biotechnol. J. 2021:13741. doi: 10.1111/pbi.13741

PubMed Abstract | CrossRef Full Text | Google Scholar

Liu, J., and Wang, Y. (2021). 3D surface reconstruction of small height object based on thin structured light scanning. Micron 143:103022. doi: 10.1016/j.micron.2021.103022

PubMed Abstract | CrossRef Full Text | Google Scholar

Lobos, G. A., Camargo, A. V., Del Pozo, A., Araus, J. L., Ortiz, R., and Doonan, J. H. (2017). Editorial: plant phenotyping and phenomics for plant breeding. Front. Plant Sci. 8:2181. doi: 10.3389/fpls.2017.02181

PubMed Abstract | CrossRef Full Text | Google Scholar

Marton, Z. C., Rusu, R. B., and Beetz, M. (2009). “On fast surface reconstruction methods for large and noisy point clouds,” in Proceeding of the IEEE International Conference on Robotics & Automation. IEEE. doi: 10.1016/j.mex.2021.101398

PubMed Abstract | CrossRef Full Text | Google Scholar

Paulus, S. (2019). Measuring crops in 3D: using geometry for plant phenotyping. Plant Methods 15:103. doi: 10.1186/s13007-019-0490-0

PubMed Abstract | CrossRef Full Text | Google Scholar

Ray, D. K., Ramankutty, N., Mueller, N. D., West, P. C., and Foley, J. A. (2012). Recent patterns of crop yield growth and stagnation. Nat. Commun. 3:1293. doi: 10.1038/ncomms2296

PubMed Abstract | CrossRef Full Text | Google Scholar

Reynolds, M., Bonnett, D., Chapman, S. C., Furbank, R. T., Manes, Y., Mather, D. E., et al. (2011). Raising yield potential of wheat. I. overview of a consortium approach and breeding strategies. J. Exp. Bot. 62, 439–452. doi: 10.1093/jxb/erq311

PubMed Abstract | CrossRef Full Text | Google Scholar

Schindelin, J., Rueden, C. T., Hiner, M. C., and Eliceiri, K. W. (2015). The imagej ecosystem: an open platform for biomedical image analysis. Mol. Rep. Dev. 82, 518–529. doi: 10.1002/mrd.22489

PubMed Abstract | CrossRef Full Text | Google Scholar

Schnabel, R., Wahl, R., and Klein, R. (2007). Efficient RANSAC for point-cloud shape detection. Comput. Graph. Forum 26, 214–226. doi: 10.1111/j.1467-8659.2007.01016.x

CrossRef Full Text | Google Scholar

Shouche, S. P., Rastogi, R., Bhagwat, S. G., and Sainis, J. K. (2001). Shape analysis of grains of indian wheat varieties. Comput. Electr. Agric. 33, 55–76. doi: 10.1016/S0168-1699(01)00174-0

CrossRef Full Text | Google Scholar

Strange, H., Zwiggelaar, R., Sturrock, C., Mooney, S. J., and Doonan, J. H. (2015). Automatic estimation of wheat grain morphometry from computed tomography data. Funct. Plant Biol. 42, 452–459. doi: 10.1071/FP14068

PubMed Abstract | CrossRef Full Text | Google Scholar

Sun, C., Berman, M., Coward, D., and Osborne, B. (2007). Thickness measurement and crease detection of wheat grains using stereo vision. Pattern Recogn. Lett. 28, 1501–1508. doi: 10.1016/j.patrec.2007.03.008

CrossRef Full Text | Google Scholar

Tanabata, T., Shibaya, T., Hori, K., Ebana, K., and Yano, M. (2012). Smart grain: high-throughput phenotyping software for measuring seed shape through image analysis. Plant Physiol. 160, 1871–1880. doi: 10.1104/pp.112.205120

PubMed Abstract | CrossRef Full Text | Google Scholar

Tian, J. C. (2002). Super-wheat and breeding methods. J. Triticeae Crops 22.

Google Scholar

Wold, S., Esbensen, K., and Geladi, P. (1987). Principal component analysis. Chemometr. Intelli. Labor. Syst. 2, 37–52. doi: 10.1016/0169-7439(87)80084-9

CrossRef Full Text | Google Scholar

Yan, H., and Long, D. (2016). 3D scanner-based corn seed modeling. Appl. Eng. Agric. 32, 181–188. doi: 10.13031/aea.32.11187

CrossRef Full Text | Google Scholar

Yang, W., Feng, H., Zhang, X., Zhang, J., Doonan, J. H., Batchelor, W. D., et al. (2020). Crop phenomics and high-throughput phenotyping: past decades. Curr. Chall. Future Perspect. Mol. Plant 13, 187–214. doi: 10.1016/j.molp.2020.01.008

PubMed Abstract | CrossRef Full Text | Google Scholar

Zhang, C., and Chen, T. (2001). “Efficient feature extraction for 2D/3D objects in mesh representation,” in Proceeding of the International Conference.

Google Scholar