Vigour testing for the rice seed with computer vision-based techniques

Rice is the staple food for approximately half of the world’s population. Seed vigour has a crucial impact on the yield, which can be evaluated by germination rate, vigor index and etc. Existing seed vigour testing methods heavily rely on manual inspections that are destructive, time-consuming, and labor-intensive. To address the drawbacks of existing rice seed vigour testing, we proposed a multispectral image-based non-destructive seed germination testing approach. Specifically, we collected multispectral data in 19 wavebands for six rice varieties. Furthermore, we designed an end-to-end pipeline, denoted as MsiFormer (MisFormer cod3e will be available at https://github.com/LiaoYun0x0/MisFormer) by integrating a Yolo-based object detector (trained Yolo v5) and a vision transformer-based vigour testing model, which effectively improved the automation and efficiency of existing techniques. In order to objectively evaluate the performance of the proposed method in this paper, we conduct a comparison between MisFormer and other 3 deep learning methods. The results showed that, MisFormer performed much better with the accuracy of 94.17%, which was 2.5%-18.34% higher than the other 3 deep learning methods. Besides MsiFormer, possibilities of CIELab mediated image analysis of TTC (tetrazolium chloride) staining in rice seed viability and nCDA (normalized canonical discriminant analysis) in rice seed vigour were also discussed, where CIELab L^* of TTC staining were negatively correlated with vigor index and germination rate, with Pearson’s correlation coefficient of -0.9874, -0.9802 respectively, and CIELab A^* of TTC staining were and positively correlated with vigor index and germination rate, with Pearson’s correlation coefficient of 0.9624, 0.9544 respectively, and CIELab A^* of nCDA had Pearson’s correlation coefficient of -0.8866 and -0.9340 with vigor index and germination rate, respectively. Besides testing methods, vigour results within and among variety(ies) showed that, there were great variations among the 6 rice varieties, and mean coefficient of variation (CV) of vigor index of individual seed within a variety reached 64.87%, revealing the high risk of conventional methods in random sampling. Vigour variations had close relationship with wavelengths of 780 nm-970 nm, indicating their value in future research.

1 Introduction

Rice (Oryza sativa L.), the staple food consumed by about half of the world’s population, is of great importance due to its various nutrients and caloric contribution (Chen et al., 2022), as well as its model plant role in monocot genetics (Flavell, 2009). In China, rice yield account for 40% of total grain output, where planting area of hybrid rice has exceeded 50% of rice planting (Luo, 2021). Seed vigour is the sum of those properties that determine the activity and performance of seed lots of acceptable germination in a wide range of environments. Growing concerns about improving rice seed vigour has been observed over the past few decades since rice production becomes mechanized, with direct sowing replacing manual transplanting (Liang et al., 2022). Furthermore, seed vigour should receive the utmost attention in order to address issues with climate change and land depletion because it can ensure rapid and uniform germination and subsequent seedling development to survive under the harsh environments, and increase productivity of arable lands (Havé et al., 2017; Kishor et al., 2021). Seed vigour deserves more priority than investments on fertilizers, pesticides and other inputs since it is the first step in crop surviving and has impact on yield and field management. However, seed vigour is less mentioned compared with disease control, cultivation management, etc.

Seed vigour is a comprehensive concept, and it has tight connection with the potential of seed germination and field performance (Sun et al., 2007; Baalbaki, 2009), and is generally explained via germination rate, vigor index and etc. Germination test and seedling growth test are still used as the official method for evaluating seed vigour in most countries which are based on random sampling (small sampling representing batches or bulks of seeds) (Agelet et al., 2012). Seed viability (germination capacity) was defined as the percentage of germinated seeds (Men et al., 2017), and generally tested by tetrazolium chloride (TTC) staining. Generally, the seed vigour high, its viability is also high. TTC staining has been proven its efficiency in assessment of seed viability of castor bean seeds (Gaspar-Oliveira et al., 2010; Diantina et al., 2022). However, the quantitative correlation between intensity of TTC staining and seed viability is unknown.

Phenomics is increasingly applied in seed industry (Fan et al., 2023). Multispectral imaging (MSI) is an emerging technology that integrates imaging and spectroscopy to obtain both spatial and spectral information of the target objects simultaneously (Mastrangelo et al., 2019; França-Silva et al., 2020). MSI can provide information of chemical composition, and phenotypic features (texture, color, shape, size) based on reflected and absorbed light information of interior and surface materials of seeds (Xia et al., 2019; ElMasry et al., 2019a; Mortensen et al., 2021). Since sample pretreatments are not a necessity, MSIs provide the great potential for nondestructive and straightforward measurement in a rapid and robust manner. MSI was initially used to screen transgenic rice seeds (Liu et al., 2014) and discriminate rice varieties (Liu et al., 2016). Currently, this technology has been increasingly promising, e.g. detection of physiological and physical properties of seeds of peanut (Oliveira et al., 2022), soybean (Silva et al., 2021), tomato and carrot (Galletti et al., 2020), identifying of hard and soft seeds of six legume species (Hu et al., 2020), discrimination of different aged and germinated seeds of cowpea (ElMasry et al., 2019b), assessment of seed viability of castor beans (Olesen et al., 2015). Compared with other cash crops, seed vigour is less discussed in rice.

MSI technology in seed testing usually involves multiple steps: sample preparation, device calibration, image and data acquisition, ROI (regions of interest) segmentation, feature extraction, analysis and modelling (Mortensen et al., 2021). For practice purpose, seeds are typically photographed in bulk instead of single seed. Individual seed testing is in great favor of accuracy of the whole sample and removing of unqualified seeds (Li et al., 2018; ElMasry et al., 2019a). Traditional ROI segmentation is done by hand or with the aid of cluster algorithms, which is time-consuming, or dependent on threshold setting. Effective ROI segmentations are few reported (Li et al., 2018; ElMasry et al., 2019a). In the stage of modelling, linear discriminant analysis (LDA), artificial neural network (ANN), support vector machine (SVM) and least squares discriminant analysis (PLS-DA) are usually employed (Xia et al., 2019; ElMasry et al., 2019a; Mortensen et al., 2021). Along with the development of deep learning, more sophisticated algorithms are expected to enhance prediction accuracy of the models.

In our study, research was carried out to reveal the possibility MSI technology in rice seed vigour testing. Six rice varieties were used and treated with 0-, 5-, 10- and 20-day accelerated ageing to generate vigour gradient artificially. Multispectral images and corresponding data of rice seeds were acquired taking advantage of the 19 wavelengths of Videometer Lab4™ instrument. Dataset was constructed based on ROI segmentation, data augmentation and data merging. Taking the data of germination rate of the 6 rice varieties as training and testing samples, a new deep learning network, i.e. prediction model was developed, which was combining residual convolutional structure and Transformer. Performance of our prediction model was evaluated compared with some classical deep learning methods. Finally, MsiFormer was proposed in our study for the purpose of germination detecting of rice seeds. Besides MsiFormer, this paper also revealed the possibility of CIELab-mediated analysis of TTC staining images in viability testing and nCDA (normalized canonical discriminant analysis) images in vigour testing of rice seeds. Vigour variations and their correlations with reflectance features and morphological features were investigated as well.

2 Materials and methods

2.1 Materials

Totally 6 varieties were used in our study, which were from ‘Rice Diversity Panel’ (RDP) established by Genetic stocks Oryza (GSOR) of Ministry of Agriculture of the U.S.A (McCouch et al., 2016) and were propagated in Yunnan in 2020. Among the 6 varieties, 3 varieties belong to japonica subgroup, 1 variety belongs to indica and 1 aus rice (Table 1).

TABLE 1

Table 1 Information of the 6 rice varieties.

2.2 Accelerated ageing

Accelerated ageing was adopted to generate vigour gradients artificially. Seeds of each variety were placed smoothly in one layer in sealing bags and incubated in 42°C water bath for 5 days, 10 days and 20 days. After accelerated ageing, seeds were incubated at 25°C for 1 week before image collection. (Baek et al., 2019)

2.3 Multispectral image analysis on seed vigour

2.3.1 Pipeline for fully automatic seed germination prediction

In order to efficiently perform end-to-end non-destructive seed germination detection, a pipeline as shown in Figure 1 was proposed. Firstly, the seed images were collected through Videometer Lab4™ for data acquisition, which can obtained a large number of multispectral images. Secondly, the object detection of image data was performed by Yolo v5, and the ROI segmentation of individual seed was implemented. Then, the extracted multispectral image of each seed was input into the designed MsiFormer for predicting germination. Finally, the final germination prediction results of individual seeds were obtained by MsiFormer.

FIGURE 1

Figure 1 The pipeline for fully automatic seed germination prediction.

2.3.2 Image and data acquisition

Raw multispectral images were captured by a Videometer Lab4™ instrument (Videometer A/S, Herlev, Denmark), which consists of 19 wavelengths (365, 405, 430, 450, 470, 490, 515, 540, 570, 590, 630, 645, 660, 690, 780, 850, 880, 890 and 970 nm). Videometer Lab4™ consists of a 5 mega pixel CCD camera, mounted inside the top of the integrating sphere, coated with highly white and diffusing paint and illumination by narrowband high-power LED placed at the rim, and thereby ensures uniform diffuse lighting, and minimizes shadows or specular reflection. Before capturing images, calibration was conducted using 3 discs, i.e. the bright disc for reflection calibration, the dark disc for background calibration and the dotted disc for aligning calibration of geometric pixel position. Light configuration was also calibrated to optimize the intensity at each wavelength. In ‘light setup’, 100% reflectance was selected in our study.

For each acquisition, 50 seeds were placed evenly in a 9-cm glass petri dish, which were then placed at the bottom of the integrating sphere. Then 20 high resolution multispectral images of 2192 × 2192 pixels were captured (19 spectral images with different wavelengths and 1 image in RGB form) within 5-10 seconds. Six varieties were applied for multispectral image analysis, and for each variety, 200 seeds were used.

Besides images, corresponding data were also obtained, including reflectance features and morphological features, e.g. area, length, width, ratio width/length, compactness circle, compactness ellipse, betashape a, betashape b, vertical skewness, CIELab L*, CIELab A*, CIELab B*, saturation, hue, vertical orientation.

2.3.3 ROI segmentation

In the proposed pipeline, Yolo v5 was used to segment the pictures of each individual seed. Yolo v5 is a very effective deep learning-based object detection algorithm. It can automatically identify the categories and locations of different objects in the image, and label the bounding box of each object accurately.

In order to train Yolo v5, 240 rice seeds were collected from each of the six varieties, namely CY-7, CY-80, CY-150, CY-203, CY-229, and CY-256. After accelerated aging processing, multispectral images were taken respectively.

Then the above images were manually annotated with the bounding boxes of the seeds. We labeled 6 RGB images without aging. Each image contained 50 seeds, so we labeled 300 samples in total. Finally, transfer learning was performed based on Yolo v5. Afterwards, the IOU loss coefficient was set to be 0.05 and the CLS loss coefficient to be 0.5. After 5 cycles of training, the model could accurately detect the bounding box of the seed in the petri dish.

2.3.4 MsiFormer

2.3.4.1 Dataset for germination prediction

Based on the trained Yolo v5 model, 24,000 seed images (6 varieties, 200 seeds for each variety, and 20 images for each seed) were obtained. Finally, the uniform sampling strategy was used to obtain 100 germinated seed samples and 100 non-germinated seed samples from the samples uniformly. 20000 images (1000 seeds) were selected for training and 4000 images (200 seeds) for testing.

Data augmentation was applied for dataset construction (Figure 2). After ROI segmentation, images of individual seed were rotated, mirrored and flipped, for the purpose of enriching the training data and improving the learning ability of the algorithm for complex data. Finally, images were resized to 256*256, and channels of RGB image and 19 spectral images were merged to generate 22 channels of multi-channel data.

FIGURE 2

Figure 2 Illustration of the dataset construction.

2.3.4.2 Modelling for germination prediction

A new deep learning network (MsiFormer) combining residual convolutional structure and Transformer for germination prediction was proposed in our study.

According to Table 2, the network accepted images of size (256, 256, 22) and outputs features of size (8, 8, 2048) from layers 0 to 4 of the designed residual network structure. The global features of images were further learned by the self-attention module of the Transformer, and features of size (1, 1, 2048) were output. The residual block of our designed deep residual network used skip connections, which relieved the problem of gradient disappearance brought by increasing depth in deep neural networks, and improved the accuracy by increasing a comparable depth. Self-attention was used in our vision Transformer module to make each pixel related to other pixels, which greatly enhanced their correlation with seed germination.

TABLE 2

Table 2 Description on deep convolutional networks stage.

Figure 3 briefly showed the architecture of the neural network used for germination prediction purposes. The input to the deep learning model was an image with height of 256, width of 256, and channels of 22, while the output was a feature map with height of 1, width of 1, and channels of 2048 (Figure 3). Our deep learning backbone network adopted the idea of combining residual structure and Transformer. The residual structure was initially used to design five large residual convolution layers to perform residual convolution operation on the feature map and learn the feature information of the seed. The skip connection of residual structure was used in each convolutional layer, which greatly reduced the loss of details of features and solved the problem of gradient disappearance and gradient explosion of convolutional networks.

FIGURE 3

Figure 3 Architecture of MisFromer used in germination prediction. "*" means multiplication.

Feature maps with length 8, width 8 and number of channels 2048 were output from the final convolutional layer. A “self-attention mechanism” was used to design a Transformer architecture, which greatly improved the relevance of each feature information in the seed feature map. The designed architecture used five self-attention layers. Unlike convolutional neural networks, Transformers did not change the size and number of channels of the data, but directly extracted features through global attention. Different from the traditional transformer, linear attention (Katharopoulos et al., 2020) was used to replace dot-product attention, where the dot-product attention used an exponential kernel and the linear attention used an alternative kernel function $sim(Q,K)=\phi (Q)·\phi {(K)}^T$, where $\phi (·)=elu(·)+1$. Linear attention can reduce the computational complexity from $O(N^2)$ to $O(N)$, which greatly improved the operation speed of the model. Afterwards, the size of the output data and the number of channels were still 8*8*2048. With the Transformer structure, the size of the feature map was further reduced and the number of channels was unchanged. The final output was a feature map of (1, 1, 2048).

The feature maps generated by the deep learning model were output to the fully connected layer. All the previously extracted features were integrated and classified by the cross-entropy function. The softmax function was first used to output the probabilities of germinant and non-germinant seeds with values of ‘0’ and ‘1’, respectively. When calculating the final loss of model sprouting, cross-entropy loss function was designed to evaluate the accuracy of softmax to further calculate the logarithmic loss. N was the number of samples; $y_i$ presented the label of the ground truth of the sample $i$, i.e. 1 for positive class and 0 for negative class; $p$ denoted the probability that sample $i$ was predicted to be positive.

$\begin{array}{ll}L=\frac{1}{N}{\displaystyle \sum {}_i{L}_i=\frac{1}{N}}{\displaystyle \sum {}_i[y_i·\log (p_i)+(1-y_i)·\text{log}(1-p_i)]}& (1)\end{array}$

2.4 Biological analysis on seed vigour

2.4.1 Conventional seed vigour testing

Seed vigour test was conducted according to the GB/T 3543.4-1995 (National Standards of P. R. China, 1995) to discriminate the viable seeds from the non-viable seeds. After multispectral imaging, seeds were transferred in sequence to another dish lined with 2 pieces of wet sterilized filter paper, and incubated for 10 days in growth chamber (28°C at day and 25°C at night with a 14-h/10-h light/dark photoperiod). Germinated seeds and corresponding germination time were individually recorded everyday based on the standard of root length ≥ 2 mm. On the 5^th day, shoot lengths of each seed were measured, and vigor index and germination rate were calculated according to Song method (Song et al., 2015).

2.4.2 Tetrazolium chloride staining

After accelerate aging, rice seeds (CY-7, CY-80, CY-150, CY-203, CY-229, CY-256) were hulled and soaked in sterile water, and then incubated at 25°C dark condition for 16 h. After water on the seed surface was removed with filter paper, seeds were stained with 1% TTC for 3 h, and washed with PBS buffer (pH 7.0) for 3 times (Olesen et al., 2015; National Standards of P. R. China, 2017; Chen et al., 2021). Seeds were cut in half lengthways, and the embryo staining was observed, where red embryos indicated viable seeds. Staining percentages were calculated based on the stained seeds. There were 3 replicates for each treatment, and 30 seeds for each replicate.

To determine the quantitative correlation between staining intensity of each seed and corresponding seed viability, CY-150 was used. After staining and cutting, the stained embryos were observed and photographed using stereoscope (Leica, M205FC). To determine the staining intensity and staining area, images were analyzed by our software with independent intellectual property rights (Liu, 2022), which can do the quantitative analysis on color via color space conversion from RGB to CIELab.

2.5 Statistic analysis

To analyze the correlation between seed vigour and multispectral Imaging, software packages of python and R language (version 4.2.1) was applied, including nCDA, tidyverse, ggplot2, WGCNA, ggsci, etc.

To evaluate the performance of our modelling, four types of metrics were applied, including True Positives (TP, prediction is positive and correct), False Positives (FP, prediction is positive and wrong), True Negative (TN, prediction is negative and correct) and False Negative (FN, prediction is negative and wrong). Based on the 4 types of metrics, Accuracy (ACC), Precision, True Positive Rate (TPR), False Positive Rate (FPR), True Negative Rate (TNR), False Negative Rate (FNR) were also used for further evaluation.

3 Results

3.1 Seed vigour testing based on conventional methods

Figure 4 showed that, seed vigour of the 6 rice varieties were generally dropping, including germination rate, vigor index and TTC staining. There were significantly positive correlations between germination rate and vigor index (Pearson’s correlation coefficient: 0.9101), between germination rate and TTC staining (Pearson’s correlation coefficient: 0.8929) and between vigor index and TTC staining (Pearson’s correlation coefficient: 0.8170). Besides the general dropping trend, there were also variations within and among the 6 varieties. The mean coefficient of variation (CV) of germination rate and vigor index of individual seed of the 6 varieties were 11.18% and 64.87%, respectively, indicating the high risk in seed vigour testing via small sampling representing batches or bulks of seeds.

FIGURE 4

Figure 4 Germination rate (A, D), vigor index (B, E) and TTC staining (C, F) of 6 rice varieties after treatments of 0-, 5- 10- and 20-day accelerated ageing).

As for variety, CY-80 seeds kept the highest vigour under 0-, 5- or 10-day accelerated ageing (Figure 4); CY-80, CY-7, CY-150, CY-256 and CY-229 dropped their vigour much less after 10-day accelerated ageing than those after 20-day accelerate ageing; CY-203 had obviously dropping vigour after 10-day accelerated ageing compared with those after 20-day accelerated ageing (Figure 4). Therefore, the 4 types of accelerated ageing had different impact on the 6 varieties, as well as on individual seed within the variety.

Comparison among the results of germination rate, vigor index and TTC staining showed that, germination rate of the 6 varieties were high (after 0- or 5-day accelerated ageing) and low (after 20-day accelerated ageing) as a whole, with little difference (Figures 4A, D), while vigor index of the 6 varieties showed great difference after 0-, 5- and 10-day accelerated ageing, with the most obvious difference after 10-day accelerated ageing (Figures 4B, E), indicating vigor index had more potential in distinguishing different viable seeds in early stages, which was in correspondence with the fact that seeds with germination power may lose the capability of growing (Corbineau et al., 2010; Olesen et al., 2015).

It was interesting to note that, CY-203 had low germination rate (10%) and vigor index (0.0164) after 10-day accelerated ageing, but moderately high TTC staining (61.11%), and similar inconformity was also found in vigor index (0) and TTC staining (58.73%) of CY-7 after 20-day accelerated ageing. The reasons may be that, the stained cells had no responsibility to or had lost the capability to germinate or grow despite their ability to respire.

3.2 Seed vigour testing based on RGB imaging analysis

To further explore the correlation between vigor index (or germination rate) and TTC staining, RGB image analysis of staining of seed embryo was conducted. Figure 5 illustrated that, the intensity of TTC staining became weak gradually (Figure 5). CIELab values of TTC staining had significant correlation with vigor index, and germination rate as well, where CIELab L^* was negatively correlated with vigor index and germination rate, with Pearson’s correlation coefficient of -0.9874, -0.9802 respectively, and CIELab A^* was positively correlated with vigor index and germination rate, with Pearson’s correlation coefficient of 0.9624, 0.9544 respectively, (Figure 6A), indicating the potential of CIELab analysis of TTC staining images in the testing of seed viability. Furthermore, according to our results, TTC staining was always weakening from inside to outside, rather than weakening as a whole, indicating cell ageing of scutellum may contribute little to viability loss, while ageing of hypocotyl and radicle may play a major role. Since staining time in our study was ensured, above weakening features in intensity and distribution revealed the cell ageing of individual seed was unsynchronous.

FIGURE 5

Figure 5 TTC staining of rice variety CY-150 after accelerated ageing. (A): structure diagram of embryo of rice seed after TTC staining; (B–F): indication of intensity and distribution of TTC staining of the whole embryo.

FIGURE 6

Figure 6 Correlation between vigor index and TTC staining (A) as well as morphological features (B). (A): after accelerated ageing and TTC staining, correlation between vigor index of CY-150 and CIELab L^*, CIELab A^*, CIELab B^* of staining images was concluded; (B): after accelerated ageing, correlation between vigor index of the 6 rice varieties and the 15 morphological features was analyzed. *, **, ***: significant correlation with p<0.05, p<0.01, p<0.001, respectively.

Besides TTC staining, correlation between vigor index of individual seed and morphological features was also analyzed. Figure 6B indicated that, CIELab L^* and CIELab A^* had negative correlation with vigor index, with Pearson’s correlation coefficient of -0.1513 and -0.2756, respectively, at the level of P<0.001. Since the correlation coefficient were not very high and besides, instead of natural aging, heating (42°C) was used for accelerated ageing purpose in our study, whether heating made the hull color changed was unknown. Therefore, above correlation between external color change and vigour difference inside need to be further explored.

3.3 Seed vigour testing based on multispectral imaging analysis

According to Figure 7, nCDA was able to distinguish the 4 groups of seeds with different ageing duration into blue, green, yellow and red as a whole, which was in general correspondence with their vigour results. The ability of nCDA in global assessment of seed vigour was also reported in researches of peanut, castor, et al. (Olesen et al., 2015; Oliveira et al., 2022).

FIGURE 7

Figure 7 Overview of seeds of CY-256 after 0-, 5- 10- and 20-day accelerated ageing. (A): RGB image; (B): transformed image by nCDA (normalized canonical discriminant analysis); (C): correlation between CIELab L^*, CIELab A^*, CIELab B^* of nCDA images and vigor index, germination rate.

Besides global assessment, CIELab values of nCDA were all negatively correlated with vigor index and germination rate, where CIELab A^* had Pearson’s correlation coefficient of -0.8866 and -0.9340 with vigor index and germination rate, respectively (Figure 7C), also revealing the potential of CIELab analysis of nCDA image in the testing of seed vigour.

The spectral reflection results showed that, there was little variation among different types of accelerated ageing in lower wavelength (Figure 8A), while the variation became obvious when it came to higher wavelength (from 780 nm to 970 nm, Figure 8B). Compared with non-aged seeds, seeds with accelerated ageing had higher reflection intensity (Figures 8A, B), which were in accordance with previously reported researches (ElMasry et al., 2019b). Meanwhile, correlation between vigor index and spectra also illustrated their significantly negative correlation, where the Pearson’s correlation coefficient were from -0.2523 to -0.2994 in wavelength from 780 nm to 970 nm, and much higher than those of lower wavelength (Pearson’s correlation coefficient: from -0.1411 to -0.2267) (Figure 8C). Therefore, 780 nm - 970 nm may be the feature spectrum for seed vigour. Since 780 nm - 970 nm has close relationship with fat (Song et al., 2015; Sendin et al., 2018), further researches on material composition and gene expression are required.

FIGURE 8

Figure 8 Mean reflectance spectra (from 365 nm to 970 nm) of the 6 rice varieties after 0-, 5-, 10- and 20-day accelerated ageing (A), mean reflectance spectra (from 780 nm to 970 nm) (B) and correlation between vigor index of individual seed and spectra (C) ***: significant correlation with p<0.001..

Four types of metrics were applied in our study for the purpose of evaluating the capability of our modelling in germination prediction, including TP, FP, TN, FN, and based on which, ACC, Precision, TPR (also knwon as recall), FPR, TNR and FNR were also used for further evaluation.

$\begin{array}{ll} ACC= \frac{TP+TN}{TP+TN+FP+FN}& (2)\end{array}$

$\begin{array}{ll} Precision=\frac{TP}{TP+FP}& (3)\end{array}$

$\begin{array}{ll}TPR=\frac{TP}{TP+FN}& (4)\end{array}$

$\begin{array}{ll}FPR=\frac{FP}{FP+TN}& (5)\end{array}$

$\begin{array}{ll}TNR=\frac{TN}{TN+FP}& (6)\end{array}$

$\begin{array}{ll}FNR=\frac{FN}{FN+TP}& (7)\end{array}$

Based on the metrics mentioned above, the MsiFormer proposed in our study was evaluated in comparison with several excellent deep learning methods, e.g. ResNet (He et al., 2016), DenseNet (Huang et al., 2017) and EfficientNet (Tan and Le, 2019). ResNet (He et al., 2016) is a convolutional neural network and has good performance in image classification and object recognition. ResNet was designed with many residual blocks that used skip connections to improve accuracy by adding considerable depth, alleviating the problems of gradient disappearance and gradient explosion caused by increasing depth in deep neural networks. DenseNet (Huang et al., 2017) is a densely connected convolutional network which connects each layer to other layers in a feed-forward fashion, where the input of each layer comes from the output of all previous layers. DenseNet reduces the number of parameters and enhances the transfer and reuse of features, and performs well in image classification, object detection, and image generation. EfficientNet (Tan and Le, 2019) is a deep learning method and uses the web search technique NAS to increase the resolution, depth and width of the network simultaneously for better results. EfficientNet is also a general architecture for computer vision that could handle multiple vision tasks including image classification. Based on special design (Mentioned in ‘2. Materials and Methods’), MsiFormer can classify multispectral images containing more channels (22 channels), while ResNet, DenseNet and EfficientNet can only classify RGB images with 3 channels.

According to Table 3, MsiFormer surpassed other methods in all evaluation metrics, revealing the great efficiency of our deep learning architecture.

TABLE 3

Table 3 Evaluation of MsiFormer in predictions of seed germination prediction in comparison with other methods.

As shown in Figure 9, the ROC curve comparison of different methods was plotted, where TPR and FPR were taken as ordinate and abscissa, respectively. Generally, the closer the ROC curve is to the top left, the higher the accuracy; the higher the AUC value (AUC: the area under curve, or the area under the ROC curve), the better the prediction ability of the method. Compared with other methods, MsiFormer had a better ROC curve, and higher AUC value as well. Therefore, MsiFormer is robust in predicting the germination of seeds individually, and germination rate of a given variety can be calculated accordingly.

FIGURE 9

Figure 9 The ROC plots of different methods in seed germination prediction.

4 Discussion

Rice is a very important food crop. In recent years, rice seed vigour raises increasing concerns due to the growing application of hybrid rice and direct sowing, along with environmental deteriorating and arable land decreasing. However, rice seed vigour is mainly tested with the aid of conventional methods, including visual detecting via germination rate and vigor index.

Conventional testing methods require intricate procedures and skilled staff, which are sample-destructive, labor-intensive and time-consuming (Baek et al., 2019; Xia et al., 2019). The importance of developing new technologies for seed vigour determination is recognized by both the International Seed Testing Association (ISTA) and Association of Official Seed Analysts (AOSA) (Boelt et al., 2018). In our study, new testing methods based on computer vision were proposed, including CIELab-mediated image analysis of TTC staining and nCDA, and particularly, MsiFormer-based MSI techniques. Compared with conventional methods, our new methods can reduce human interference and subjectivity, and thus, enhancing the automation and accuracy.

Our MSI technique, with the integration of Yolo v5 and MsiFormer, was developed in our study. Compared with SVM, ANN, etc., our MSI technique can improve automation with little pretreatment, including manual threshold setting, extraction of feature spectrum, etc. Based on our MSI technique, ROI segmentation can be implemented in high efficiency, and then seeds with no germinating ability can be screened out from the sound ones. Since conventional testing of rice seed vigour is based on single seed and adopt the strategy of small sampling representing the overall batches or bulks of seeds, it is not only tedious and time-consuming, but also neglects variation among individual seeds. According to our results, mean coefficient of variation (CV) of vigor index of individual seed reached 64.87%. Therefore, development of on-site seed screening to detect the variations among seed lots and among individual seeds within a seed lot will be in great favor of improving the accuracy of rice seed vigour testing, and providing convenience for seed production and processing. Besides ROI segmentation, our MsiFormer also offers a new prediction model for germination prediction with good accuracy of 94.17%, which is better than other deep learning methods. Since our model is to predict whether rice seeds will germinate or not and seed ageing is a continuous process, further research on quantitative discrimination in rice seed vigour is expected.

Most MSI technologies were dependent on prediction modelling and multispectral images. In our study, CIELab-mediated image analysis of TTC staining and nCDA were proposed as other options in case that prediction modelling and multispectral images are unavailable.

5 Conclusion

Our study further revealed the great demand of automated and accurate methods in rice seed vigour testing, and offered new computer vision-based methods accordingly, which can reduce human interference and subjectivity.

Data availability statement

The datasets presented in this study can be found in online repositories. The names of the repository/repositories and accession number(s) can be found below: https://github.com/LiaoYun0x0/MisFormer.

Author contributions

YFL and WW conceived and designed the experiments; JQ, CY, XY, HT performed the experiments and conducted the data analysis; YL established the models; YFL and YL wrote the article; WW revised the article. All authors contributed to the article and approved the submitted version.

Funding

This work was supported by Project of Kunming International Scientific & Technological Cooperation Base (grant No. GHJD-2022012); Yunnan Province international Science and technology Special Commissioner (individual) identification in 2022 (grant No. 202303AK140026); "Zhihui Yunnan" Program in 2022 - Young scientist and entrepreneur Hoàng Minh Tú (grant No. 202303am140011).

Acknowledgments

We thank all lab members and the reviewers for their help in this study.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Agelet, L. E., Ellis, D. D., Duvick, S., Goggi, A. S., Hurburgh, C. R., Gardner, C. A. (2012). Feasibility of near-infrared spectroscopy for analyzing corn kernel damage and viability of soybean and corn kernels. J. Cereal Science. 55 (2), 160–165. doi: 10.1016/j.jcs.2011.11.002

CrossRef Full Text | Google Scholar

Liu, J. (2022) International union for the protection of new varieties of plants. Available at: https://www.upov.int/meetings/en/details.jsp?meeting_id=67781,TWM/1/5.

Google Scholar

Baalbaki, R. (2009). Association of official seed analysts (Ithaca, NY, USA: Seed vigor testing handbook).

Google Scholar

Baek, I., Kusumaningrum, D., Kandpal, L. M., Lohumi, S., Mo, C., Kim, M. S., et al. (2019). Rapid measurement of soybean seed viability using kernel-based multispectral image analysis. Sensors. 19 (2), 271. doi: 10.3390/s19020271

CrossRef Full Text | Google Scholar

Boelt, B., Shrestha, S., Salimi, Z., Jørgensen, J. R., Nicolaisen, M., Carstensen, J. M. (2018). Multispectral imaging - A new tool in seed quality assessment? Seed Sci. Res. 28, 222–228. doi: 10.1017/S0960258518000235

CrossRef Full Text | Google Scholar

Chen, R. Z., Deng, Y. W., Ding, Y. L., Guo, J. X., Qiu, J., Wang, B., et al. (2022). Rice functional genomics: decades' efforts and roads ahead. Sci. China Life Sci. 65 (1), 33–92. doi: 10.1007/s11427-021-2024-0

PubMed Abstract | CrossRef Full Text | Google Scholar

Chen, B. X., Zhang, Q., Dai, Z. Y., Gao, J. D., Zhang, W. H., Wu, R. X., et al. (2021). Study on TTC staining and germination characteristics of rice seeds. Guangdong Agric. Science. 48 (7), 1–8.

Google Scholar

Corbineau, F., Gay-Mathieu, C., Vinel, D., Côme, D. (2010). Decrease in sunflower (Helianthus annuus) seed viability caused by high temperature as related to energy metabolism, membrane damage and lipid composition. Physiologia Plantarum. 116 (4), 489–496.

Google Scholar

Diantina, S., McGill, C., Millner, J., Nadarajan, J., Pritchard, H. W., Colville, L., et al. (2022). Seed viability and fatty acid profiles of five orchid species before and after ageing. Plant Biol. (Stuttg). 24 (1), 168–175. doi: 10.1111/plb.13345

PubMed Abstract | CrossRef Full Text | Google Scholar

ElMasry, G., Mandour, N., Al-Rejaie, S., Belin, E., Rousseau, D. (2019a). Recent applications of multispectral imaging in seed phenotyping and quality monitoring-an overview. Sensors. 19, 1090. doi: 10.3390/s19051090

CrossRef Full Text | Google Scholar

ElMasry, G., Mandour, N., Wagner, M. H., Demilly, D., Verdier, J., Belin, E., et al. (2019b). Utilization of computer vision and multispectral imaging techniques for classification of cowpea (Vigna unguiculata) seeds. Plant Methods 15 (1), 24. doi: 10.1186/s13007-019-0411-2

PubMed Abstract | CrossRef Full Text | Google Scholar

Fan, J. C., Li, Y. L., Yu, S., Gou, W. B., Guo, X. Y., Zhao, C. J. (2023). Application of internet of things to agriculture-the LQ-fieldPheno platform: A high-throughput platform for obtaining crop phenotypes in field. Res. (Wash D C). 6, 59. doi: 10.34133/research.0059

CrossRef Full Text | Google Scholar

Flavell, R. (2009). Role of model plant species. Methods Mol. Biol. 513, 1–18. doi: 10.1007/978-1-59745-427-8_1

PubMed Abstract | CrossRef Full Text | Google Scholar

França-Silva, F., Rego, C. H. Q., Gomes-Junior, F. G., Moraes, M. H. D., Medeiros, A. D., Silva, C. B. D. (2020). Detection of Drechslera avenae (Eidam) Sharif [Helminthosporium avenae (Eidam)] in Black Oat Seeds (Avena strigosa Schreb) Using Multispectral Imaging. Sensors (Basel). 20 (12), 3343. doi: 10.3390/s20123343

PubMed Abstract | CrossRef Full Text | Google Scholar

Galletti, P. A., Carvalho, M. E. A., Hirai, W. Y., Brancaglioni, V. A., Arthur, V., Silva, C. B. (2020). Integrating Optical Imaging Tools for Rapid and Non-invasive Characterization of Seed Quality: Tomato (Solanum lycopersicum L.) and Carrot (Daucus carota L.) as Study Cases. Front. Plant Science. 11.

Google Scholar

Gaspar-Oliveira, C. M., Martins, C. C., Nakagawa, J. (2010). Avaliação da qualidade fisiológica de sementes de mamoneira (Ricinus communis L.) pelo teste de tetrazólio. Rev. Bras. Sementes. 32, 186–196. doi: 10.3389/fpls.2020.577851

CrossRef Full Text | Google Scholar

Havé, M., Marmagne, A., Chardon, F., Masclaux-Daubresse, C. (2017). Nitrogen remobilization during leaf senescence: lessons from Arabidopsis to crops. J. Exp. Botany. 8 (10), 2513–2529. doi: 10.1093/jxb/erw365

CrossRef Full Text | Google Scholar

He, K. M., Zhang, X. Y., Ren, S. Q., Sun, J. (2016). “Deep residual learning for image recognition,” in Conference on Computer Vision and Pattern Recognition. 770–778. doi: 10.1109/cvpr.2016.90

CrossRef Full Text | Google Scholar

Hu, X. W., Yang, L. J., Zhang, Z. X. (2020). Non−destructive identification of single hard seed via multispectral imaging analysis in six legume species. Plant Methods 16 (1), 116. doi: 10.1186/s13007-020-00659-5

PubMed Abstract | CrossRef Full Text | Google Scholar

Huang, G., Liu, Z., Maaten, L. V., Weinberger, K. Q. (2017). “Densely connected convolutional networks,” in Conference on Computer Vision and Pattern Recognition. 2261–2269. doi: 10.1109/cvpr.2017.243

CrossRef Full Text | Google Scholar

Katharopoulos, A., Vyas, A., Pappas, N., Fleuret, F. (2020). Transformers are RNNs: Fast autoregressive transformers with linear attention. Int. Conf. Mach. Learning., 5156–5165. doi: 10.48550/arXiv.2006.16236

CrossRef Full Text | Google Scholar

Kishor, P. B. K., Kumar, S. A., Naravula, J., Kumari, P. H., Kummari, D., Rajasheker Guddimalli, R., et al. (2021). Improvement of small seed for big nutritional feed. Physiol. Mol. Biol. Plants. 27 (10), 2433–2446. doi: 10.1007/s12298-021-01071-6

PubMed Abstract | CrossRef Full Text | Google Scholar

Li, Z., Li, C., Gao, Y., Ma, W. G., Zheng, Y. Y., Niu, Y. Z., et al. (2018). Identification of oil, sugar and crude fiber during tobacco (Nicotiana tabacum L.) seed development based on near infrared spectroscopy. Biomass Bioenergy. 111, 39–45. doi: 10.1016/j.biombioe.2018.01.017

CrossRef Full Text | Google Scholar

Liang, Y. G., Li, J. Y., Zhou, J., Wu, T., Fang, B. H., Yu, Z. J. (2022). Evolution and prospect of rice cultivation technology in China. Crop Res. 36 (2), 180–188.

Google Scholar

Liu, C., Liu, W., Lu, X., Chen, W., Chen, F., Yang, J. B., et al. (2016). Non-destructive discrimination of conventional and glyphosate-resistant soybean seeds and their hybrid descendants using multispectral imaging and chemometric method. J. Agric. Science. 154, 1–12. doi: 10.1017/S0021859614001142

CrossRef Full Text | Google Scholar

Liu, C. H., Liu, W., Lu, X. Z., Chen, W., Yang, J. B., Zheng, L. (2014). Nondestructive determination of transgenic Bacillus thuringiensis rice seeds (Oryza sativa L.) using multispectral imaging and chemometric methods. Food Chem. 153, 87–93. doi: 10.1016/j.foodchem.2013.11.166

PubMed Abstract | CrossRef Full Text | Google Scholar

Luo, Q. J. (2021). Study on seed quality detection based on hyperspectral technique (Beijing: Beijjing Forestry University). [doctoral thesis]. doi: 10.26949/d.cnki.gblyu.2021.000909

CrossRef Full Text | Google Scholar

Mastrangelo, T., Silva, F., Mascarin, G. M., Silva, C. B. (2019). Multispectral imaging for quality control of laboratory-reared Anastrepha fraterculus (Diptera: Tephritidae) pupae. J. Appl. Entomology. 143 (10), 1–8. doi: 10.1111/jen.12716

CrossRef Full Text | Google Scholar

McCouch, S. R., Wright, M. H., Tung, C. W., Maron, L. G., McNally, K. L., Fitzgerald, M., et al. (2016). Open access resources for genome-wide association mapping in rice. Nat. Commun. 7, 10532. doi: 10.1038/ncomms10532

PubMed Abstract | CrossRef Full Text | Google Scholar

Men, S., Yan, L., Liu, J., Qian, H., Luo, Q. (2017). A classification method for seed viability assessment with infrared thermography. Sensors (Basel). 17 (4), 845. doi: 10.3390/s17040845

PubMed Abstract | CrossRef Full Text | Google Scholar

Mortensen, A. K., Gislum, R., Jørgensen, J. R., Boelt, B. (2021). The use of multispectral imaging and single seed and bulk near-infrared spectroscopy to characterize seed covering structures: Methods and applications in seed testing and research. Agriculture. 11, 01. doi: 10.3390/agriculture11040301

CrossRef Full Text | Google Scholar

National Standards of P. R. China (1995). Rules for agricultural seed testing -germination test (China: National Standardization Administration of the People's Republic of China). GB/T 3543.4-1995.

Google Scholar

National Standards of P. R. China (2017). Rules of seed testing for forage, turfgrass and other herbaceous plant - Biochemical test for viability (China: National Standardization Administration of the People's Republic of China). GB/T 2930.5-2017.

Google Scholar

Olesen, M. H., Nikneshan, P., Shrestha, S., Tadayyon, A., Deleuran, L. C., Boelt, B., et al. (2015). Viability prediction of ricinus cummunis L. Seeds using multispectral imaging. Sensors. 15, 4592–4604. doi: 10.3390/s150204592

CrossRef Full Text | Google Scholar

Oliveira, G. R. F., Mastrangelo, C. B., Hirai, W. Y., Batista, T. B., Sudki, J. M., Petronilio, A. C. P., et al. (2022). An approach using emerging optical technologies and artificial intelligence brings new markers to evaluate peanut seed quality. Front. Plant Science. 14. doi: 10.3389/fpls.2022.849986

CrossRef Full Text | Google Scholar

Sendin, K., Manley, M., Williams, P. J. (2018). Classification of white maize defects with multispectral imaging. Food Chem. 243, 311–318. doi: 10.1016/j.foodchem.2017.09.133

PubMed Abstract | CrossRef Full Text | Google Scholar

Silva, C. B., Oliveira, N. M., Carvalho, M. E. A., Medeiros, A. D., Nogueira, M. L., Reis, A. R. (2021). Autofluorescence−spectral imaging as an innovative method for rapid, non−destructive and reliable assessing of soybean seed quality. Sci. Rep. 11, 17834. doi: 10.1038/s41598-021-97223-5

PubMed Abstract | CrossRef Full Text | Google Scholar

Song, L., Wang, Q., Wang, C. Y., Lin, Y. Q., Yu, D., Xu, Z. P., et al. (2015). Effect of γ-irradiation on rice seed vigor assessed by near-infrared spectroscopy. J. Stored Products Res. 62, 46–51. doi: 10.1016/j.jspr.2015.03.009

CrossRef Full Text | Google Scholar

Sun, Q., Wang, J. H., Sun, B. Q. (2007). Advances on seed vigor physiological and genetic mechanisms. Agric. Sci. China. 6 (9), 1060–1066. doi: 10.1016/S1671-2927(07)60147-3

CrossRef Full Text | Google Scholar

Tan, M. X., Le, Q. V. (2019). EfficientNet: Rethinking model scaling for convolutional neural networks. Int. Conf. Mach. Learning., 6105–6114. doi: 10.48550/arXiv.1905.11946

CrossRef Full Text | Google Scholar

Xia, Y., Xu, Y. F., Li, J. B., Zhang, C., Fan, S. X. (2019). Recent advances in emerging techniques for non-destructive detection of seed viability: A review. Artif. Intell. Agric. 1, 35–47. doi: 10.1016/j.aiia.2019.05.001

CrossRef Full Text | Google Scholar