Hyperspectral detection of fresh corn peeling damage using germinating sparse classification method

Peeling damage reduces the quality of fresh corn ear and affects the purchasing decisions of consumers. Hyperspectral imaging technique has great potential to be used for detection of peeling-damaged fresh corn. However, conventional non-machine-learning methods are limited by unsatisfactory detection accuracy, and machine-learning methods rely heavily on training samples. To address this problem, the germinating sparse classification (GSC) method is proposed to detect the peeling-damaged fresh corn. The germinating strategy is developed to refine training samples, and to dynamically adjust the number of atoms to improve the performance of dictionary, furthermore, the threshold sparse recovery algorithm is proposed to realize pixel level classification. The results demonstrated that the GSC method had the best classification effect with the overall classification accuracy of the training set was 98.33%, and that of the test set was 95.00%. The GSC method also had the highest average pixel prediction accuracy of 84.51% for the entire HSI regions and 91.94% for the damaged regions. This work represents a new method for mechanical damage detection of fresh corn using hyperspectral image (HSI).

1. Introduction

Fresh corn is favored by consumers for its excellent nutritional value and edible quality (Saka et al., 2018). Mechanical peeling is the main link in the processing of fresh corn, and the collision, extrusion, and friction between the ear and the high-speed rotating peeling roller can easily cause the mechanical damage of varying degrees to the crisp and tender kernels (Zhao et al., 2011). At present, most fresh corn processing plants still use the manual method to select and grade damaged ears. However, manual grading is a tedious and inefficient work, and it is more difficult to observe with naked eyes in the case of slight abrasion of seed coats (Wang, 2010). Therefore, it is urgent to find a fast automatic detection technology to detect the damage of fresh corn ear.

For detection of damaged corn ears, researchers identified them based on RGB images (Fu et al., 2020), and constructed a classification model using the color characteristics and texture characteristics (Zhang F et al., 2015). The detection of damaged corn based on traditional RGB images mainly utilizes the color difference and spatial feature difference of the target, and the classification results are seriously affected by the image acquisition environment (Gao and Liu, 2016). HSI data is a type of three-dimensional cube containing spatial pixel information (two-dimensional) and spectral information (one-dimensional) of an object (Torres et al., 2019). The wavelength of each pixel covers the entire spectral range, and meanwhile, all the data in each wavelength can form an image (Yin et al., 2017). Hence, HSI can extract more feature information for more complex spectral analysis and image processing, which cannot be achieved by the traditional machine vision. With the development of optical sensors and imaging technology, more and more studies have reported the application of HSI in the quality assessment and safety detection of various agricultural products (Kandpal et al., 2015; Su et al., 2021; Nazir et al., 2022). Among them, in the aspect of damage detection, HSI has been successfully used for the detection of crack in fresh jujube (Yu et al., 2014), scratch, scar, and spot in peach (Zhang B et al., 2015), insect damage in soybeans (Huang et al., 2013), wheat kernels damaged by Fusarium head blight (Lv et al., 2022), and freeze-damage of corn seed (Zhang et al., 2019). As far as we know, there is no report on the damage detection of fresh corn ear based on HSI. Therefore, an investigation into the potential of detecting the peeling-damaged regions of fresh corn with the spectral range of 400 nm–1000 nm deserves special attention.

At present, methods of partial least squares discriminant analysis (PLS-DA) (Ambrose et al., 2016), support vector machine (SVM) (Wakholi et al., 2018), K-singular value decomposition (K-SVD) (Fang et al., 2015), random forest (RF) (Che et al., 2018), and artificial neural networks (ANNs) (Zhang et al., 2021) are often used for HSI classification and have achieved relatively good classification results. However, a study of existing classification methods found that there are still some deficiencies in the current researches, which can be summarized as follows: (1) Non-machine-learning methods are limited by unsatisfactory detection accuracy. (2) Machine-learning methods, especially deep learning methods, rely heavily on training samples. However, there usually exist some interference samples while building training datasets, which significantly influence the accuracy of machine learning. (3) The background contents are not fixed in different detecting situations, containing various features, and are difficult to be accurately classified. Moreover, it is more difficult to classify the sound and damaged regions of fresh corn than other agricultural products due to the large number of kernels on the ear, the vertical and horizontal gaps between kernels, and the interspersed corn whisker. Therefore, it is necessary to develop a suitable algorithm for identifying the damage regions of fresh corn.

Aiming at the shortcomings of current methods, the germinating sparse classification (GSC) method was proposed to detect the sound and peeling-damaged regions of fresh corn. The innovations of this method could be summarized as follows: (1) The ‘seed germinating’ strategy is developed to refine training samples, which reduces the influence of unqualified training samples on dictionary learning. (2) For dictionary learning process, the atoms that are incompetent to accurately represent any training samples are removed during iterations, and the number of atoms can be dynamically adjusted. The strategy improves the performance of dictionary and reduces computational complexity. (3) For sparse classification process, the threshold algorithm with respect to the energy of sparse representation residual is proposed to determine the pixels of background. In addition, three commonly used classification methods K-SVD, SVM, and back propagation (BP) neural network were used to classify the sound, peeling-damaged, and background pixels in fresh corn hyperspectral images (HSIs). The classification results of the three methods were compared with that of the proposed GSC method to verify the superiority of the GSC method in the detection of fresh corn peeling damage.

2. Experimental materials and data

In this section, the fresh corn materials and equipment used in this study are described. Next, the hyperspectral data acquisition and preprocessing are introduced.

2.1. Fresh corn materials

The fresh corn variety used in this study was Jinxiangnuo, which was widely cultivated in Northeast China. One hundred and eighty fresh corn ears were hand-selected from the fresh corn production line at Fumin Food Processing Plant in Songyuan city, Jilin Province, China. The fresh corn ears were inspected by five trained workers and divided into sound and peeling-damaged. The 180 fresh corn ears were composed of 90 sound ears and 90 peeling-damaged ears. The hyperspectral data of randomly selected 60 sound ears and 60 peeling-damaged ears formed the training set, and the hyperspectral data of the remaining ears constituted the test set. The selected fresh corns were sealed in airtight bags, then stored at the optimum condition of 4°C to retain moisture. All fresh corn HSIs were taken within two hours after peeling.

2.2. Hyperspectral image acquisition

Hyperspectral data of fresh corn ears were collected by using a hand-held visible-NIR hyperspectral imaging system (Specim IQ, Specim Ltd., Finland). The system integrated a hyperspectral camera, scanning platform, image acquisition card, data acquisition software, and data processing software. As presented in Figure 1, the hyperspectral imaging system kept the internal environment consistent during all the acquisition processes to reduce the interference from the outside. The imaging spectrograph covered a spectral range of 400 nm–1000 nm. The resulting hyperspectral data cube had dimensions of 512×512 pixels and 204 wavebands. The lighting system consisted of two 150 W halogen tungsten lamps (QVF133, Philips Lighting (Shanghai) Co., LTD., Shanghai, China) which were fixed on both sides of the test platform at an angle of 45°. The hyperspectral imaging system operated at an exposure time of 22 ms during data acquisition. The distance between the lens and the fresh corns was 38 cm.

FIGURE 1

Figure 1 Schematic diagram of the hyperspectral imaging system for acquiring reflectance images of fresh corns.

2.3. Hyperspectral data preprocessing

HSIs need to be calibrated by using white and dark reference images. The raw HSI was calibrated using equation (1):

$\begin{array}{ll}{R}_c=\frac{R_o-R_b}{R_w-R_b}& (1)\end{array}$

where, R_c is the calibrated HSI; R_o is the raw HSI; R_b is a dark reference image obtained by completely blocking the lens with an opaque cover; R_w is a white reference image of a pure Teflon whiteboard (Spectralon, Labsphere Inc, North Sutton, NH, USA) with 99% reflectivity, obtained under the same environment as the raw HSI.

The RGB images corresponding to fresh corn HSIs were collected, and the region of interest (ROI) of fresh corn were labeled at pixel level by human judgment. The pixels corresponding to the sound regions and damage regions on the corn were denoted as the sound class and peeling-damaged class, respectively. However, it was inevitable to generate some impure training samples because of human mistakes or limited labeling conditions, which would directly affect the discriminant performance of the subsequent models and lead to a reduction of the classification accuracy. In order to improve the purity of training samples, all samples in training sets of the two classes were refined. The detailed method is described in subsection 3.2.2.

Modeling with all-waveband data not only takes a long time to compute and occupies a large amount of memory, but also degrades the performance due to the curse of dimensionality. Therefore, the dimension of hyperspectral data was reduced before using for training in this study. To determine the required spectral wavebands, the dimensionality reduction test including two steps was executed. By given the starting waveband and the ending waveband, first, the average spectra of the refined training samples of the sound class and peeling-damage class were calculated, respectively. Second, the spectral angle was obtained from the average spectra of the two classes of training samples. The spectral angle is utilized to reflect the difference between two classes of spectra. A larger spectral angle indicates greater difference between the two classes of training samples, and implying better performance for classification. Through the results of test presented in Table 1, it can be noted that the spectral angle of the two classes of training samples obtained by using the first 70 wavebands (corresponding wavelength range from 400 nm to 607 nm) was the largest, implying that the first 70 bands may have better potential to provide satisfactory classification performance.

TABLE 1

Table 1 Spectral angles of average spectra of refining samples of two classes in terms of different used bands.

The spectra after 70 wavebands (corresponding wavelength range from 608 nm to1000 nm) were mainly infrared and near-infrared, which was not helpful to the damage identification of fresh corn and even interfered with the classification results. Moreover, the complexity of our algorithm was low, and processing 70 wavebands would not significantly increase the running speed. The selected 70 wavebands were universal to both the training sample and the test target. Therefore, the first 70 wavebands were selected and applied to the four classification methods.

3. Experimental methods

In recent years, there has been increasing interest in sparse representation of signals. Sparse representation is widely used in the computer vision and pattern recognition in various fields, including image denoising (Sun et al., 2014), image classification (Zheng et al., 2020), face recognition (Liu et al., 2019), disease recognition (Feng and Zhou, 2016), and target tracking (Ma and Xu, 2021), etc. In these applications, the sparse representation method often leads to the most advanced performance. Therefore, this study aimed to develop a classification method based on sparse representation for fresh corn peeling damage. This section briefly reviews the theoretical background of the HSI classification based on sparse representation, and then introduces the detailed processes of the GSC method proposed in this paper.

3.1. Related work

In the sparse representation classification (SRC), pixels in the same class are assumed to approximately lie in the same low-dimensional subspace. Suppose there are C distinct classes in HSI, and the c-th class has M_c training samples. $M={\displaystyle {\sum }_{c=1}^C{M}_c}$ is the total number of training samples. A spatial pixel in HSI can be approximately represented as x=[x₁,x₂,⋯,x_N]∈ℝ^N×1 , where N is the number of wavebands. The subdictionary D_c∈ℝ^N×M_c is constructed by directly extracting the pixels of the c-th class in original HSI. All subdictionaries for C classes can be obtained, and all these subdictionaries constitute the final dictionary D=[d₁,d₂,⋯,d_c,⋯,d_C]∈ℝ^N×M . The SRC assumes that the pixel x of a particular class can be represented as a sparse linear combination of a dictionary D. Then x can be sparsely represented as x = Dα or approximate x≈Dα , satisfying

$\begin{array}{ll}{\Vert \mathit{x}-\mathit{D}\mathit{\alpha }\Vert }_2\le e& (2)\end{array}$

where α∈ℝ^M×1 contains the representation coefficients for the pixel x and e is the residual.

The sparse coefficient vector $\widehat{\mathit{\alpha }}$ can be obtained by solving the following optimization problem:

$\begin{array}{ll}\hat{\mathit{\alpha }}=\text{arg}\underset{\alpha }{\min }{\Vert \mathit{x}\mathbf{-}\mathit{D}\mathit{\alpha }\Vert }_2 \text{s}.\text{t}.{\Vert \mathit{\alpha }\Vert }_0\le K_0& (3)\end{array}$

where ∥·∥₀ represents the l₀-norm of the vector α which counts the number of nonzero entries in the vector and K₀ is the upper bound of the sparse level which is equal to the number of nonzero rows in $\widehat{\mathit{\alpha }}$. The solution with the fewest number of nonzero coefficients is certainly an appealing representation. However, equation (3) is NP-hard (Li et al., 2017), which can be approximately solved by matching pursuit algorithms. Once $\widehat{\mathit{\alpha }}$ is obtained, the class of the pixel x is determined as the one with the minimal reconstruction residual (Fang et al., 2017),

$\begin{array}{ll}\text{class}(\mathit{\text{x}})=\arg \underset{c=1,\cdots ,C}{\min }{\mathit{\text{r}}}^c(\mathit{\text{x}})=\arg \underset{c=1,\cdots ,C}{\min }\Vert \mathit{\text{x}}-\mathit{\text{D}}{\widehat{\mathit{\alpha }}}_c{\Vert }_2& (4)\end{array}$

where ${\widehat{\mathit{\alpha }}}_c$ is the sparse coefficient subset of $\widehat{\mathbf{\text{α}}}$ belonging to c-th class.

3.2. Proposed method

3.2.1. Algorithm framework

The proposed method contains three processes: sample refining, dictionary learning, and sparse classification. The graphical representation of the overall process of the proposed GSC algorithm is shown in in Figure 2.

FIGURE 2

Figure 2 The graphical representation of the overall process of the proposed GSC algorithm.

3.2.2. Sample refining

The sample refining process is used to remove the unqualified training samples (Gong et al., 2019; Lv et al., 2019). For each class, the strategy is manually selecting a set of qualified training samples first, which are referred to as sample seeds in this study. These seeds are then employed as the baseline to be compared with each training sample. If a training sample has insufficient correlation with all seeds, the sample is considered not to belong to this class, and therefore, it will be removed from the training samples. The detailed process is summarized in Algorithm 1. In this algorithm, the correlation between a seed and a training sample is evaluated based on the 2-norm, expressed as $\parallel {\mathit{\text{z}}}_j-{\mathit{\text{v}}}_{c,i}{\parallel }_2^2/\parallel {\mathit{\text{z}}}_j{\parallel }_2^2,$ where z_j–v_c,i is the residual between the seed v_ci and the training sample z_j. The ratio $\parallel {\mathit{\text{z}}}_j-{\mathit{\text{v}}}_{c,i}{\parallel }_2^2/\parallel {\mathit{\text{z}}}_j{\parallel }_2^2$ is larger than the threshold γ means z_j has insufficient correlation to v_c,i. The above evaluation of z_j is repeated for all seeds. If θ_min ≥ γ, i.e., z_j has insufficient correlation to all seeds, the training sample z_j will be removed. The algorithm is executed for the peeling-damaged class and sound class to obtain training samples Z₁∈ℝ^n_p×η₁ and Z₂∈ℝ^n_p×η₂ , respectively, where η₁ and η₂ denote the number of training samples of two classes, respectively.

ALGORITHM 1SAMPLING REFINING PROCESS OF THE GSC.

Algorithm 1Sampling refining process of the GSC..

3.2.3. Dictionary learning

Next, the dictionary learning process is executed to obtain the sparse dictionaries of peeling-damaged class and sound class, respectively. The process is summarized as Algorithm 2. Given the training samples Z, The first step is to initialize the dictionary D∈ℝ^n_p×q , where q represents the number of atoms(column) of D. In this study the dictionary is initialized to be the Gaussian random matrix. The training process is an iterative process with the maximum number of iterative cycles of t_max. For each iterative cycle, the sparse coefficients s_j is computed based on the l₀-optimization problem, ${\mathit{s}}_j=\arg \underset{s}{\min }\parallel z_j-\mathit{\text{D}}\mathit{\text{s}}{\parallel }_2 \text{s}.\text{t}.\parallel \mathit{\text{s}}{\parallel }_0\le k$, which can be solved by algorithms such as the orthogonal matching pursuit (OMP) (Wang et al., 2012). The step of sparse coefficients computation is called sparse coding. By completing the sparse coding step, the matrix S = [s₁, s₂, ⋯, s_ζ] is obtained. Then, for each atom d_i, we inspect whether the atom has been used in the sparse coding step. If an atom was not used, it will be removed from the dictionary, shown as lines 7–8 in Algorithm 2. The other atoms are then updated by using the strategy of the K-SVD. The Algorithm 2 is executed for both two classes, respectively, and two dictionaries D₁∈ℝ^n_p×q₁ and D₂∈ℝ^n_p×q₂ , where q₁ and q₂ denote the number of atoms of two dictionaries, respectively. It should be pointed out that the background, i.e., the pixels that do not belong to peeling-damaged class or sound class, usually contains various features. Thus, it is difficult to obtain a dictionary that can represent all features accurately. Based on this consideration, the background class is not involved in the dictionary learning process, but the classification among the background class and other two classes can be still realized, which is described as follows.

ALGORITHM 2 DICTIONARY LEARNING PROCESS OF THE GSC.

Algorithm 2 Dictionary learning process of the GSC..

3.2.4. Sparse classification

The obtained dictionaries are used for pixel level classification of HSIs. The algorithm is summarized in Algorithm 3. The matrix Y denotes the classification result of the HSI $\underset{\_}{\mathit{\text{X}}}$, and it is initialized by the zero matrix (all entries of Y are zero). The classification process is realized by classifying each spatial pixel. Given an arbitrary spatial pixel, the spectral data is extracted from $\underset{\_}{\mathit{\text{X}}}$, expressed as $\mathit{\text{x}}=\underset{\_}{\mathit{\text{X}}}\left(i,j,:\right)$. Then, the sparse recovery problems with respect to x and two dictionaries D₁ and D₂ are solved to obtain the sparse coefficients s₁ and s₂, respectively, shown as steps 3 and 5 in Algorithm 3. Similar to the sparse coding process in Algorithm 2, the sparse recovery problem can be solved by using the OMP algorithm or other l₀-optimization algorithms. The residuals of sparse recovery results are computed, denoted as r_c = x – D_cs_c, c = 1, 2. The 2-norm of residuals are computed, denoted as e₁ and e₂, respectively. Then, the classification of the pixel is determined by the judgement given by steps 7 and 8 of Algorithm 3. If the smaller one of e₁ and e₂ is larger than threshold ε, it means the spectral data x cannot be accurately represented by neither D₁ nor D₂. Hence, the pixel is considered to belong to neither peeling-damaged class nor sound class, and it is determined to belong to the background class (denoted as value-0 in Y). If the smaller one of e₁ and e₂ is not larger than threshold ε, the pixel is determined to belong to the class that satisfies $\arg \underset{\text{c}}{\min }\left\{e_{\text{c}}\right\}$. The steps 2–8 are repeated for each spatial pixel, and finally the classification result Y∈ℝ^n_s1×n_s2 is obtained.

ALGORITHM 3 SPARSE CLASSIFICATION PROCESS OF THE GSC.

Algorithm 3 Sparse classification process of the GSC..

3.2.5. Evaluation index

The classification result of the HSI was obtained by pixel level classification. Considering that some outliers might reduce the accuracy of the algorithm, the block level calculation method was adopted to process the original classification result and eliminate the influence of outliers in this paper. In the block level method, first of all, every 2×2 pixels in the prediction classification results were divided into small blocks; and then the class with the largest number of pixels in the small block was statistically obtained; finally, all pixels in this small block were divided into this class. The evaluation indexes were the overall classification accuracy and the pixel prediction accuracy. The overall classification accuracy is the percentage of the correctly classified sound fresh corn and damaged fresh corn in the training set and test set. The pixel prediction accuracy of the entire HSI region can be calculated by dividing the number of the correctly predicted pixels in the classification result by the total number of the pixels in the test image. The pixel prediction accuracies of the damaged region can be calculated by dividing the number of correctly predicted pixels in the damaged region by the total number of such pixels in the ground-truth.

4. Experimental results and analysis

The experiment results and discussion are introduced in this section. Firstly, the refined training samples are presented, and the characteristics of reflectance spectra of fresh corn are analyzed. Then, the key parameters of the proposed GSC method are determined through experiments. Finally, the classification results of fresh corn HSIs are described. All experiments were carried out using the Matlab 2021a software.

4.1. Refined samples and spectral characteristic

There were 146893 and 190242 training samples for sound class and peeling-damaged class of fresh corn, respectively. After the sample refining process, 39085 and 116427 impure training samples were removed, and then 107808 and 73815 refined training samples were left in sound class and peeling-damaged class, respectively. The same refined training samples were used in the GSC, K-SVD, SVM, and BP classification methods. Figure 3 shows the original training samples and the refined training samples of the sound class and peeling-damaged class of fresh corn. As shown in Figure 3, some obviously unqualified training samples in the sound and peeling-damaged classes had been removed after sample refining. In addition, it could be seen that the spectral reflection intensity decreased first and then increased. The relative intensity of the peeling-damaged fresh corn was lower than that of the sound one, and it was because the damaged pigment and collapsed tissue of fresh corn could cause a reduced light reflection (Gao et al., 2019).

FIGURE 3

Figure 3 Original training samples and refined training samples of the sound class and peeling-damaged class of fresh corn under the first 70 wavebands: (A) original training samples of sound class; (B) refined training samples of sound class; (C) original training samples of peeling-damaged class; (D) refined training samples of peeling-damaged class.

4.2. Experimental parameter selection

In order to obtain better classification performance, some key parameters of the proposed GSC methods needed to be determined, such as the sparse level k, maximum number of iterative cycles t_max, and threshold ε. The optimal parameters were determined by experiments, k ranged in {1, 2, 3, 4, 5}, t_max ranged in {3, 6, 10, 15, 20}, and ε ranged in {0.02, 0.04, 0.06, 0.08, 0.10}. If the parameter values were selected appropriately, the pixel prediction accuracies were high. Therefore, different parameter values were tested repeatedly on 12 HSIs of peeling-damaged fresh corn selected from the test set to determine the appropriate values of k, t_max, and ε. The following parameter combinations were selected during the test: the sparsity level was 2, maximum number of iterative cycles was 15, and threshold was 0.08. Two parameters were fixed in each group of tests to explore the appropriate parameter values for maximizing the pixel prediction accuracies of the GSC method. The average pixel prediction accuracies under different parameters are displayed in Figure 4. Aiming for the highest average pixel prediction accuracies, the optimal parameter values of the proposed GSC method were finally set as k=2, t_max=10, and ε =0.06 based on the experimental results shown in Figure 4. Taking one HSI as an example, the classification maps under different parameters are shown in Figure 5. The yellow pixels in classification maps represented the sound corn, the white pixels represented the peeling-damaged corn, and the black pixels represented the background.

FIGURE 4

Figure 4 Average pixel prediction accuracies under different parameter levels. (A) sparsity level k; (B) iterative cycles t_max; (C) threshold ϵ.

FIGURE 5

Figure 5 Classification maps under different parameter levels.

4.3. Classification results analysis

In this subsection, the GSC method and other three commonly used K-SVD, SVM, and BP methods were used to conduct the overall classification of fresh corn HSIs in the training set and test set, and finally determined that the fresh corn belonged to sound or damaged ear. Besides, all test fresh corn HSIs in the test set were precisely classified, and each pixel in the image was classified at the pixel level. Finally, the pixel was ascertained to belong to sound kernel, damaged kernel, or background classes. The overall classification accuracy and average pixel prediction accuracy of the four methods were obtained and compared.

4.3.1. Results on overall classification accuracy

The overall classification accuracy results of the fresh corn HSIs in the training set and test set using four classification methods are listed in Table 2. It can be seen that the proposed GSC method performed best in distinguishing sound and damaged fresh corn ears. For the training set, the identification accuracy of sound and damaged fresh corn by GSC method was 98.33%, with 2 damaged fresh corns misjudged as sound fresh corns. For the test set of the GSC method, the identification accuracy of sound and damaged fresh corn by was 95.00%, with 1 sound fresh corn misjudged as damaged fresh corn and 2 damaged fresh corns misjudged as sound fresh corns. Dysplastic corn kernels on fresh corn were often identified as damaged class, resulting in the sound fresh corn might be misjudged as the damaged fresh corn. In addition, the damaged fresh corn was misjudged as the sound fresh corn because the chemical and physical information on the surface tissue did not change significantly when the seed coat of fresh corn was slightly damaged. At this time, the spectral curves of the damaged fresh corn were similar to that of the normal fresh corn. Qiao et al., (2021) drew the same conclusion in the nondestructive detection of decayed blueberry.

TABLE 2

Table 2 Overall classification accuracies of HSIs in the training/test set using four methods under the selected optimal first 70 wavebands.

4.3.2. Results on average pixel prediction accuracy

Four classification methods were applied to classify the test fresh corn HSIs in the test set for precise pixel classification, and the average pixel prediction accuracies of 60 test images are shown in Figure 6. For the pixel classification results of the entire HSI region, the proposed GSC method had the highest average pixel prediction accuracy of 84.51%, followed by the BP neural network method which reached 76.23%. The average pixel prediction accuracy of the GSC method was 41.39%, 21.04%, and 10.86% higher than that of K-SVD, SVM, and BP methods, respectively. For the pixel classification results of the damaged region, the proposed GSC method had the highest average pixel prediction accuracy of 91.94%, followed by the BP, K-SVD and SVM methods with 77.31%, 61.50%, and 44.39%, respectively. The average pixel prediction accuracy of the GSC method was at least 18.92% higher than that of other methods. The average pixel prediction accuracy of the entire HSI region of four methods including the GSC method did not reach above 85%, but this did not affect the practical application of the GSC method. This is because the accurate identification of the damaged region of fresh corn in the practical application process is the key to realize the automatic detection and grading of peeled fresh corns. However, the average pixel prediction accuracy of the GSC method for the damaged region was higher than 90%, which satisfied the practical application.

FIGURE 6

Figure 6 Average pixel prediction accuracies of the HSIs in the test set using four methods based on the given ground-truths.

4.3.3. Analysis of classification results on typical scenes

Three typical fresh corn HSIs were selected from the test set for detailed study, and denoted as Scene 1, Scene 2, and Scene 3, respectively. The classification results by using four methods of the selected three scenes are presented in Figures 7–9. Each method had two classification result images. One was the original classification result image based on pixel-level classification, the other was the classification result image processed by the 2×2 block-level method. In the following analysis, the pixel prediction accuracy of each method was considered to be the larger value of the two classification results. The RGB images corresponding to fresh corn HSIs (‘Objective’) and the corresponding ground-truths with manual labeling (‘label’) were given as the reference images of classification results in Figures 7–9.

FIGURE 7

Figure 7 Classification results on a single seriously damaged fresh corn in Scene 1. (A) Pixel-level; (B) 2×2 block-level.

(1) Classification results on Scene 1

There was a seriously damaged fresh corn ear in Scene 1, in which the seed coats of most kernels had disappeared and the liquid endosperm was exposed. Table 3 shows the pixel prediction accuracy of Scene 1. Among the four methods, the GSC method had the highest pixel prediction accuracy of 78.77%, while the SVM method had the lowest pixel prediction accuracy of 54.41%. The prediction accuracy values of Scene 1 using K-SVD, SVM, and BP methods were all lower than the average values given in Figure 6. The possible reason is that the pure Teflon whiteboard as background class was wrongly classified as damaged kernel class by the three methods. As shown in Figure 7, the GSC and K-SVD methods could identify the complete contour of the objective fresh corn, while the SVM and BP methods incorrectly classified most of the background pixels into sound kernel class. The pixel prediction accuracy of the GSC method for the damaged regions of fresh corn in Scene 1 was 92.89%. The GSC method could recognize almost all damaged kernels, while the SVM and BP methods could only recognize a part of damaged kernels. However, the K-SVD method misclassified most of damaged kernels into background class.

TABLE 3

Table 3 Pixel prediction accuracies of the four methods based on the given ground-truth of Scene 1.

(2) Classification results on Scene 2

There were two fresh corn ears in Scene 2. One was a sound ear and the other was a seriously damaged ear. Table 4 shows the pixel prediction accuracy of Scene 2. Among the four methods, the GSC method had the highest pixel prediction accuracy of 85.15%, while the K-SVD method had the lowest pixel prediction accuracy of 45.66%. As shown in Figure 8, first, the GSC method could detect the complete contours of the two fresh corns and background, this indicated that the GSC method had the potential of detecting multiple fresh corns; second, it could identify almost all the damaged kernels in the damaged fresh corn; and third, it could completely recognize all the sound kernels in the sound fresh corn, in addition to identifying the gaps between kernels as background. The BP method could also identify the regions of fresh corns and background, but it could only identify a part of damaged kernels. The SVM method could not detect the edges of fresh corns, and they wrongly classified the background between two fresh corns into sound kernel class. The K-SVD method could identify the complete region of background, but it misclassified almost all damaged kernels into background class.

TABLE 4

Table 4 Pixel prediction accuracies of the four methods based on the given ground-truth of Scene 2.

FIGURE 8

Figure 8 Classification results on one sound and one seriously damaged fresh corns in Scene 2. (A) Pixel-level; (B) 2×2 block-level.

(3) Classification results on Scene 3

There were two fresh corn ears in Scene 3. One was a sound ear and the other was a damaged ear with slight abrasion of seed coats. Table 5 shows the pixel prediction accuracy of Scene 3. Among the four methods, the GSC method had the highest pixel prediction accuracy of 87.68%, while the K-SVD method had the lowest pixel prediction accuracy of 65.85%. As shown in Figure 9, the GSC methods could accurately and completely detect fresh corns and background with clear edges, but it wrongly identified dysplastic corn kernels on the sound ear as damaged class. The K-SVD and BP methods could also effectively distinguish the fresh corn region and background region, but there were more pixels misclassified at the edge of fresh corn. Similar to the situation in Scene 2, the K-SVD method still misclassified the damaged kernels into background class, and the SVM method still misclassified the background between two fresh corns into sound kernel class. The SVM and BP methods could only identify a few damaged kernels.

TABLE 5

Table 5 Pixel prediction accuracies of the four methods based on the given ground-truth of Scene 3.

FIGURE 9

Figure 9 Classification results on one sound fresh corn and one damaged fresh corn with slight abrasion of seed coats in Scene 3. (A) Pixel-level; (B) 2×2 block-level.

4.4. Discussion

4.4.1. Discussion on using HSI detection instead of traditional RGB image

It should be noted that the HSI was chose for damage identification of fresh corn instead of the traditional RGB images in this paper, mainly for the following reasons. Traditional RGB image classification method mainly relies on two points. (1) Differences in colors. However, there are many varieties of fresh corn, and the colors of seed coats are diverse, such as yellow, white, purple, and even multiple colors on an ear. Moreover, the color of white seed coat is the same as that of inner endosperm, so the classification accuracy based on the color differences is not high. Therefore, it is not feasible to accurately identify fresh corn peeling-damage by color differences. (2) Differences in spatial characteristics. However, the shape of the damage regions caused by mechanical peeling is random, which may be triangular, circular, or irregular. Therefore, there are no fixed spatial geometric characteristics, and it is not feasible to accurately identify fresh corn peeling-damage using spatial geometric characteristics.

4.4.2. Discussion on dimensionality reduction method

The dimensionality reduction method is to reduce the computation amount and improve the classification accuracy. For example, principal component analysis (PCA), a commonly used feature reduction method, takes training data and test data as a whole to extract the principal components. The results obtained in this case are good for both the training data and the test data. Therefore, the dimensionality reduction data are used for training to obtain the principal components, which are used to classify the test data and get better classification effect. For the problem addressed in this study, the test data needed to deal with cannot be obtained in advance while executing the training process, and therefore, the PCA results obtained by using training data may be not suitable for test data.

4.4.3. Discussion on comparison with existing methods

It could be seen from the experimental results that the developed GSC method had an absolute advantage in hyperspectral detection of fresh corn damage when background contents were containing multiple features. This can be explained at a theoretical level. As shown in Figure 7, the GSC method could correctly classify the pure Teflon whiteboard, large gaps between kernels, and marks on fresh corns made by a marking pen into the background class, while the other three methods had poor extraction effect for the background. This is because the GSC method determines background pixels by the threshold algorithm with respect to the energy of sparse representation residual during the sparse classification process. Even if the background contents are not fixed in different detecting situations, the classification effect will not be affected. It is worth mentioning that it is always a challenge to detect the slight abrasion of seed coats. By comparing the classification results in Figures 8, 9, it can be seen that the recognition effect of the fresh corn with slight abrasion of seed coats is obviously inferior to that of the seriously damaged fresh corn, and there was still a gap between the detection result of the abrasion region and the ground-truth. However, compared with other classification methods, the GSC method had the best recognition effect on the abrasion region with the pixel prediction accuracy of 81.13%, while the other three methods almost fail in the abrasion detection. This proves the absolute superiority of our proposed method. One possibility is that the reflectance spectra of slightly bruised seed coats are closer to sound seed coats, thus increasing the difficulty of classification. Another possibility is that it is difficult to recognize slightly bruised kernels with naked eyes, resulting that the ground-truths (‘label’) of the RGB images by manually labeling may be not completely accurate.

4.4.4. Discussion on the practical applications

During fresh corn processing, online detection of mechanical damage of fresh corn based on the GSC algorithm can be realized by using the imaging spectrograph. The peeled fresh corns are transferred to the sorting equipment by a conveyor covered with corn trays, and then vacuum packed. In order to improve the quality of fresh corn, damage detection is carried out before sorting. The lighting system and imaging spectrograph are mounted in a custom-made box above the conveyor. The speed of the conveyor is adjusted according to the spatial resolution and the integration time of the imaging spectrograph. The hyperspectral data on the front of the fresh corn are collected when the fresh corn is transported to the bottom of the imaging spectrograph. Then, the corn trays rotate and drive the fresh corns to rotate 180° to collect the hyperspectral data on the opposite side of the fresh corn. The damage detection of the whole fresh corn ear can be realized. Schematic diagram of the practical applications is shown in Figure 10.

FIGURE 10

Figure 10 Schematic diagram of the practical applications.

5. Conclusions

In this paper, the feasibility of using hyperspectral imaging technique to detect the fresh corn peeling damage was studied. The GSC method was proposed to classify the pixels of sound kernel, peeling-damaged kernel, and background. For this purpose, each process of the GSC method was introduced, including sample refining, dictionary learning, and sparse classification. The classification results of fresh corn HSIs with serious damage and slight abrasion of seed coats were also presented. Although complete extraction of damage regions in fresh corn ear with slight abrasion of seed coats was still a challenge, the experimental results demonstrated that the GSC method had the highest accuracy regardless of the damage degrees of test images. Experimental results verified the feasibility of the GSC method. The overall classification accuracy of the training set was 98.33%, and that of the test set was 95.00%. It also had the highest average pixel prediction accuracy of 84.51% for the entire HSI regions and 91.94% for the damaged regions, which were significantly higher than compared methods, including the K-SVD, SVM, and BP methods. The peeling-damaged regions of fresh corn could be directly observed by the classification results based on pixel-level classification. This study made up for the gap in the detection method of fresh corn peeling damage. The datasets used for hyperspectral fresh corn damage detection study was built. In general, the results confirmed the feasibility of hyperspectral imaging technique in detecting the fresh corn peeling-damage in laboratory environment.

Data availability statement

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author/s.

Author contributions

ZL: Conceptualization, Methodology, Software, Writing-original draft preparation. JF: Resources, Writing-review and editing, Project administration, Funding acquisition. QF: Validation, Investigation. ZC: Supervision. XL: Supervision. All authors contributed to the article and approved the submitted version.

Funding

This work was supported by the National Natural Science Foundation of China (No. 52105257).

Acknowledgments

The authors are grateful for the fresh corn provided by the Fumin Food Processing Plant.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Ambrose, A., Kandpal, L. M., Kim, M. S., Lee, W. H., Cho, B. K. (2016). High speed measurement of corn seed viability using hyperspectral imaging. Infrared Phys. Technol. 75, 173–179. doi: 10.1016/j.infrared.2015.12.008

CrossRef Full Text | Google Scholar

Che, W., Sun, L., Zhang, Q., Tan, W., Ye, D., Zhang, D., et al. (2018). Pixel based bruise region extraction of apple using vis-NIR hyperspectral imaging. Comput. Electron. Agric. 146, 12–21. doi: 10.1016/j.compag.2018.01.013

CrossRef Full Text | Google Scholar

Fang, L. Y., Li, S. T., Kang, X. D., Benediktsson, J. A. (2015). Spectral-spatial classification of hyperspectral images with a superpixel-based discriminative sparse model. IEEE Trans. Geosci. Remote Sens. 53, 4186–4201. doi: 10.1109/tgrs.2015.2392755

CrossRef Full Text | Google Scholar

Fang, L. Y., Wang, C., Li, S. T., Benediktsson, J. A. (2017). Hyperspectral image classification via multiple-feature-based adaptive sparse representation. IEEE Trans. Instrumentation Measurement 66, 1646–1657. doi: 10.1109/tim.2017.2664480

CrossRef Full Text | Google Scholar

Feng, Q. X., Zhou, Y. C. (2016). Kernel combined sparse representation for disease recognition. IEEE Trans. Multimedia 18, 1956–1968. doi: 10.1109/tmm.2016.2602062

CrossRef Full Text | Google Scholar

Fu, J., Yuan, H. K., Zhao, R. Q., Chen, Z., Ren, L. Q. (2020). Peeling damage recognition method for corn ear harvest using RGB image. Appl. Sciences-Basel 10 (10), 3371. doi: 10.3390/app10103371

CrossRef Full Text | Google Scholar

Gao, X. H., Liu, B. (2016). Design and experiment of fresh corn quality detection classifier based on machine vision. Trans. Chin. Soc. Agric. Eng. (Transactions CSAE) 32, 298–303. doi: 10.11975/j.issn.1002-6819.2016.01.041

CrossRef Full Text | Google Scholar

Gao, Z., Zhao, Y., Khot, L. R., Hoheisel, G. A., Zhang, Q. (2019). Optical sensing for early spring freeze related blueberry bud damage detection: Hyperspectral imaging for salient spectral wavelengths identification. Comput. Electron. Agric. 167, 105025. doi: 10.1016/j.compag.2019.105025

CrossRef Full Text | Google Scholar

Gong, X., Shen, L., Lu, T. (2019). Refining training samples using median absolute deviation for supervised classification of remote sensing images. J. Indian Soc. Remote Sens. 47 (4), 647–659. doi: 10.1007/s12524-018-0887-7

CrossRef Full Text | Google Scholar

Huang, M., Wan, X., Zhang, M., Zhu, Q. (2013). Detection of insect-damaged vegetable soybeans using hyperspectral transmittance image. J. Food Eng. 116 (1), 45–49. doi: 10.1016/j.jfoodeng.2012.11.014

CrossRef Full Text | Google Scholar

Kandpal, L. M., Lee, S., Kim, M. S., Bae, H., Cho, B. K. (2015). Short wave infrared (SWIR) hyperspectral imaging technique for examination of aflatoxin b-1 (AFB(1)) on corn kernels. Food Control 51, 171–176. doi: 10.1016/j.foodcont.2014.11.020

CrossRef Full Text | Google Scholar

Li, R. Z., Hu, S. L., Gao, J., Zhou, Y. P., Wang, Y. Y., Yin, M. H. (2017). GRASP for connected dominating set problems. Neural Computing Appl. 28, S1059–S1067. doi: 10.1007/s00521-016-2429-y

CrossRef Full Text | Google Scholar

Liu, Z., Wu, X. J., Shu, Z. Q. (2019). Sparsity augmented discriminative sparse representation for face recognition. Pattern Anal. Appl. 22, 1527–1535. doi: 10.1007/s10044-019-00792-5

CrossRef Full Text | Google Scholar

Lv, Z., Li, G., Benediktsson, J. A., Zhang, Z., Yan, J. (2019). Training sample refining method using an adaptive neighbor to improve the classification performance of very high-spatial resolution remote sensing images. J. Appl. Remote Sens. 13 (3), 36501. doi: 10.1117/1.JRS.13.036501

CrossRef Full Text | Google Scholar

Lv, Y. P., Lv, W. B., Han, K. X., Tao, W. T., Zheng, L., Weng, S. Z., et al. (2022). Determination of wheat kernels damaged by fusarium head blight using monochromatic images of effective wavelengths from hyperspectral imaging coupled with an architecture self-search deep network. Food Control 135. 108819. doi: 10.1016/j.foodcont.2022.108819

CrossRef Full Text | Google Scholar

Ma, W. J., Xu, F. (2021). Study on computer vision target tracking algorithm based on sparse representation. J. Real-Time Image Process. 18, 407–418. doi: 10.1007/s11554-020-00999-4

CrossRef Full Text | Google Scholar

Nazir, A., AlDhaheri, M., Mudgil, P., Marpu, P., Kamal-Eldin, A. (2022). Hyperspectral imaging based kinetic approach to assess quality deterioration in fresh mushrooms (Agaricus bisporus) during postharvest storage. Food Control 131, 108298. doi: 10.1016/j.foodcont.2021.108298

CrossRef Full Text | Google Scholar

Qiao, S., Tian, Y., Wang, Q., Song, S., Song, P. (2021). Nondestructive detection of decayed blueberry based on information fusion of hyperspectral imaging (HSI) and low-field nuclear magnetic resonance (LF-NMR). Comput. Electron. Agric. 184, 106100. doi: 10.1016/j.compag.2021.106100

CrossRef Full Text | Google Scholar

Saka, L., Kasapila, W., Manani, T. A. N., Mlotha, V. (2018). Influence of maturity, smoking, and drying of fresh maize on sensory acceptability and nutritional content of the developed porridges. Food Sci. Nutr. 6, 2402–2413. doi: 10.1002/fsn3.838

PubMed Abstract | CrossRef Full Text | Google Scholar

Sun, D., Gao, Q. W., Lu, Y. X., Huang, Z. X., Li, T. (2014). A novel image denoising algorithm using linear Bayesian MAP estimation based on sparse representation. Signal Process. 100, 132–145. doi: 10.1016/j.sigpro.2014.01.022

CrossRef Full Text | Google Scholar

Su, Z., Zhang, C., Yan, T., Zhu, J., Zeng, Y., Lu, X. (2021). Application of hyperspectral imaging for maturity and soluble solids content determination of strawberry with deep learning approaches. Front. Plant Sci. 12, 1897–1907. doi: 10.3389/fpls.2021.736334

CrossRef Full Text | Google Scholar

Torres, I., Sancheza, M. T., Cho, B. K., Garrido-Varo, A., Perez-Marin, D. (2019). Setting up a methodology to distinguish between green oranges and leaves using hyperspectral imaging. Comput. Electron. Agric. 167, 105070. doi: 10.1016/j.compag.2019.105070

CrossRef Full Text | Google Scholar

Wakholi, C., Kandpal, L. M., Lee, H., Bae, H., Park, E., Kim, M. S., et al. (2018). Rapid assessment of corn seed viability using short wave infrared line-scan hyperspectral imaging and chemometrics. Sensors Actuators B-Chemical 255, 498–507. doi: 10.1016/j.snb.2017.08.036

CrossRef Full Text | Google Scholar

Wang, J., Kwon, S., Shim, B. (2012). Generalized orthogonal matching pursuit. IEEE Trans. Signal Process. 60, 6202–6216. doi: 10.1109/tsp.2012.2218810

CrossRef Full Text | Google Scholar

Wang, H. H., Sun, Y. H., Zhang, T. T., Zhang, G. L., Li, Y., Liu, T. P. (2010). Appearance quality grading for fresh corn ear using computer vision. Trans. Chin. Soc. Agric. Machinery 41, 157–159. doi: 10.3969/j.issn.1000-1298.2010.08.032

CrossRef Full Text | Google Scholar

Yin, W. X., Zhang, C., Zhu, H. Y., Zhao, Y. R., He, Y. (2017). Application of near-infrared hyperspectral imaging to discriminate different geographical origins of Chinese wolfberries. PloS One 12 (7), e0180534. doi: 10.1371/journal.pone.0180534

PubMed Abstract | CrossRef Full Text | Google Scholar

Yu, K., Zhao, Y., Li, X., Shao, Y., Zhu, F., He, Y. (2014). Identification of crack features in fresh jujube using Vis/NIR hyperspectral imaging combined with image processing. Comput. Electron. Agric. 103, 1–10. doi: 10.1016/j.compag.2014.01.016

CrossRef Full Text | Google Scholar

Zhang, J., Dai, L. M., Cheng, F. (2019). Classification of frozen corn seeds using hyperspectral VIS/NIR reflectance imaging. Molecules 24 (1), 1–25. doi: 10.3390/molecules24010149

CrossRef Full Text | Google Scholar

Zhang, J., Dai, L. M., Cheng, F. (2021). Identification of corn seeds with different freezing damage degree based on hyperspectral reflectance imaging and deep learning method. Food Analytical Methods 14, 389–400. doi: 10.1007/s12161-020-01871-8

CrossRef Full Text | Google Scholar

Zhang, B., Li, J., Fan, S., Huang, W., Zhao, C., Liu, C., et al. (2015). Hyperspectral imaging combined with multivariate analysis and band math for detection of common defects on peaches (Prunus persica). Comput. Electron. Agric. 114, 14–24. doi: 10.1016/j.compag.2015.03.015

CrossRef Full Text | Google Scholar

Zhang, F., Li, S. M., Liu, Z., Zhu, D. H., Wang, Y., Ma, Q. (2015). Screening method of abnormal corn ears based on machine vision. Trans. Chin. Soc. Agric. Machinery 46, 45–49. doi: 10.6041/j.issn.1000-1298.2015.S0.008

CrossRef Full Text | Google Scholar

Zhao, Y. Q., He, X. P., Shi, J. F., Liu, Q., Shao, G., Xie, Q. Z. (2011). Design and development of sweet and waxy corn husker. Trans. Chin. Soc. Agric. Eng. (Transactions CSAE) 27, 114–118. doi: 10.3969/j.issn.1002-6819.2011.02.018

CrossRef Full Text | Google Scholar

Zheng, S. J., Zhang, Y. J., Liu, W. J., Zou, Y. J. (2020). Improved image representation and sparse representation for image classification. Appl. Intell. 50, 1687–1698. doi: 10.1007/s10489-019-01612-3

CrossRef Full Text | Google Scholar