Research on target recognition technology of GISC spectral imaging based on active laser lighting

Aiming at the application requirements of spectral imaging technology in satellite remote sensing, biomedical diagnosis, marine detection and rescue, agricultural and forestry monitoring and classification, military camouflage identification, etc., this paper uses 532 and 650 nm lasers as light sources, and uses multi-spectral intensity correlation imaging equipment—snapshot spectroscopic cameras based on ghost imaging via sparsity constraints (GISC) enable precise identification of targets. In this paper, the principle of snapshot GISC spectral imaging is expounded, and the experimental research work of GISC spectral imaging target recognition technology based on active laser illumination is carried out. The experimental results show that using a 532 nm laser as the light source to illuminate the target object can accurately identify the green target letter “I”; using a 650 nm laser as the light source to illuminate the target object can accurately identify the red target letter “Q”. And gives spectral imaging results of the color target “QIT” acquired by the GISC spectroscopic camera through a single exposure at the wavelength range from 446 to 698nm, with both pseudo-color map and color fusion map. In order to further illustrate the feasibility of the experiment, the spectral distribution of the reconstructed image is analyzed, which has important practical significance and engineering value.

1 Introduction

Spectral imaging technology based on ghost imaging via sparsity constraints (GISC) is a brand-new imaging system [1], which has the advantages of non-locality and strong anti-interference ability, and has great application potential in the fields of remote sensing imaging, microscopic imaging and medical imaging [2]; [3]. In GISC spectral imaging, the spectral image information can be retrieved from the high-order correlation between detected light fields and pre-measured light fields, thus it is actually a variant of the ghost imaging [4]; [5]; [6] system. In 1988, [7] proposed spontaneous parametric downconversion (SPDC) to generate entangled two-photons, and proposed an entangled two-photon quantum correlation imaging scheme. Later, both ghost image and diffraction were demonstrated to be realized by intensity correlation using classical light sources [8]; [9]; [10]. Also, apart from the correlation algorithm, other reconstruction algorithms have been further developed. In 2017, Situ Guohai [11] proposed a “GIDL” scheme based on computational quantum correlation imaging and deep learning, which is one of the representative works in the domestic quantum correlation imaging field combined with cutting-edge technologies. In 2018, [12] combined computational imaging technology with artificial intelligence, providing a new perspective for imaging technology, which has important application prospects in the fields of sensors and data analysis. In the same year, Xu Zhuo et al. [13] proposed a new type of deep learning quantum correlation imaging combined with artificial intelligence technology, which can obtain faster and more accurate target images at low sampling rates. In addition to those studies on active illuminating ghost imaging schemes, the researchers found that collecting light information of objects illuminated by natural light can also achieve image restoration through correlation computing, which is called passive quantum correlation imaging. Furthermore, the associated image formed by this light source-less imaging mechanism has the function of a single exposure, and can obtain a multispectral, three-dimensional image. In 2014, the research group of Han Shensheng of the Shanghai Institute of Optics and Mechanics [14] proposed a snapshot-type sparsely constrained quantum correlation imaging (GISC) spectral imaging system, and verified the feasibility of the imaging scheme in principle through compression imaging experiments. In 2018, the research group [15] used an 800-m-high tethered balloon to carry a snapshot-type GISC spectroscopic camera to achieve snapshot-type passive optical multispectral quantum correlation imaging of natural scenes. In the same year, the research group [16] proposed adding a flat-field grating to the snapshot GISC spectroscopic camera, which can increase the spectral resolution to 1 nm and realize the imaging function of the GISC hyperspectral camera, but the system structure is complex, which is not conducive to engineering applications. From 2020 to 2021, the research group will improve the anti-noise capability and imaging signal-to-noise ratio of the GISC spectroscopic camera through the optimized design of the hyper-Rayleigh speckle field and polarization characteristics [17]; [18]. In this paper, 532 and 650 nm solid-state lasers are used as the light source [19], and the GISC snapshot spectroscopic camera is used to realize the accurate identification of the target and carry out experimental research work.

2 Theoretical analysis

The optical path of the GISC spectral imaging system based on active laser illumination is shown in Figure 1. The coherent light source required for the test is provided by a solid-state laser, and the reflected light irradiated on the target object is 50% reflected and 50% transmitted through the beam splitter. The transmitted light images the target scene on the area array photodetector CCD1, which is an RGB camera and is used to determine the target shooting area. The reflected light passes through the filter, so that the light beam in the range of 440–700 nm is transmitted. The diffraction optical element (DOE) randomly phase modulates the light field emitted by each point on the broadband image on the pre-imaging surface, and after modulation, forms a speckle pattern in which two-dimensional spatial information and one-dimensional spectral information are aliased. The detection signal is obtained on the area array photodetector CCD2, and then according to the detection signal and the calibration obtained measurement matrix, a high-performance server is used to obtain a three-dimensional multispectral image by means of computational imaging.¹

FIGURE 1

FIGURE 1. Optical path diagram of GISC spectral imaging system based on active laser illumination.

The detection process of the GISC spectral imaging system can be expressed by the following formula [1]; [20]:

$\left[\begin{array}{c}\hfill y_{11}\hfill \\ \hfill ⋮\hfill \\ \hfill y_{m1}\hfill \\ \hfill y_{12}\hfill \\ \hfill ⋮\hfill \\ \hfill y_{mn}\hfill \end{array}\right]=\left[\begin{array}{cccccccccc}\hfill a_{\mathrm{11,11}}^{{\lambda }_1}\hfill & \hfill \cdots \hfill & \hfill a_{11,p1}^{{\lambda }_1}\hfill & \hfill a_{\mathrm{11,12}}^{{\lambda }_1}\hfill & \hfill \cdots \hfill & \hfill a_{11,pq}^{{\lambda }_1}\hfill & \hfill \cdots \hfill & \hfill a_{\mathrm{11,11}}^{{\lambda }_S}\hfill & \hfill \cdots \hfill & \hfill a_{11,pq}^{{\lambda }_S}\hfill \\ \hfill ⋮\hfill & \hfill \hfill & \hfill ⋮\hfill & \hfill ⋮\hfill & \hfill \hfill & \hfill ⋮\hfill & \hfill \hfill & \hfill ⋮\hfill & \hfill \hfill & \hfill ⋮\hfill \\ \hfill a_{m\mathrm{1,11}}^{{\lambda }_1}\hfill & \hfill \cdots \hfill & \hfill a_{m1,p1}^{{\lambda }_1}\hfill & \hfill a_{m\mathrm{1,12}}^{{\lambda }_1}\hfill & \hfill \cdots \hfill & \hfill a_{m1,pq}^{{\lambda }_1}\hfill & \hfill \cdots \hfill & \hfill a_{m\mathrm{1,11}}^{{\lambda }_S}\hfill & \hfill \cdots \hfill & \hfill a_{m1,pq}^{{\lambda }_S}\hfill \\ \hfill a_{\mathrm{12,11}}^{{\lambda }_1}\hfill & \hfill \cdots \hfill & \hfill a_{12,p1}^{{\lambda }_1}\hfill & \hfill a_{\mathrm{12,12}}^{{\lambda }_1}\hfill & \hfill \cdots \hfill & \hfill a_{12,pq}^{{\lambda }_1}\hfill & \hfill \cdots \hfill & \hfill a_{\mathrm{12,11}}^{{\lambda }_S}\hfill & \hfill \cdots \hfill & \hfill a_{12,pq}^{{\lambda }_S}\hfill \\ \hfill ⋮\hfill & \hfill \hfill & \hfill ⋮\hfill & \hfill ⋮\hfill & \hfill \hfill & \hfill ⋮\hfill & \hfill \hfill & \hfill ⋮\hfill & \hfill \hfill & \hfill ⋮\hfill \\ \hfill a_{mn,11}^{{\lambda }_1}\hfill & \hfill \cdots \hfill & \hfill a_{mn,p1}^{{\lambda }_1}\hfill & \hfill a_{mn,12}^{{\lambda }_1}\hfill & \hfill \cdots \hfill & \hfill a_{mn,pq}^{{\lambda }_1}\hfill & \hfill \cdots \hfill & \hfill a_{mn,11}^{{\lambda }_S}\hfill & \hfill \cdots \hfill & \hfill a_{mn,pq}^{{\lambda }_S}\hfill \end{array}\right]\left[\begin{array}{c}\hfill x_{11}^{{\lambda }_1}\hfill \\ \hfill ⋮\hfill \\ \hfill x_{p1}^{{\lambda }_1}\hfill \\ \hfill x_{12}^{{\lambda }_1}\hfill \\ \hfill ⋮\hfill \\ \hfill x_{pq}^{{\lambda }_1}\hfill \\ \hfill ⋮\hfill \\ \hfill x_{11}^{{\lambda }_S}\hfill \\ \hfill ⋮\hfill \\ \hfill x_{pq}^{{\lambda }_S}\hfill \end{array}\right]+\left[\begin{array}{c}\hfill {\epsilon }_{11}\hfill \\ \hfill ⋮\hfill \\ \hfill {\epsilon }_{m1}\hfill \\ \hfill {\epsilon }_{12}\hfill \\ \hfill ⋮\hfill \\ \hfill {\epsilon }_{mn}\hfill \end{array}\right](1)$

In the formula, $X_{\mathrm{p}\mathrm{q}}^{{\lambda }_{\text{s}}}(p=1,\dots ,P,q=1,\dots ,Q)$ represents the multispectral image information reconstructed by $s\left(s=1,\dots ,S\right)$ in the s spectral bands of the pixels in the pth row and the q column on the object surface, and $a_{\text{mn,\hspace{0.17em}pq}}^{{\lambda }_{\text{s}}}(m=1,\dots ,M,n=1,\dots ,N)$ represents the point light source used for the calibration of the imaging system on the pth row and qth columns on the object surface. When moving up, the light intensity value obtained by the pixel in the mth row and nth column of the CCD on the detection surface. y_mn represents the intensity distribution of the detection signal in the mth row and nth column on the detection surface during imaging, ɛ represents probe signal noise. According to Eq. 1, the reconstruction of multispectral images can be achieved by solving the following optimization problem [21]; [1]:

$\underset{x}{\min }{‖Y-AX‖}_2^2+{\mu }_1{‖{\nabla }_{i,j}X‖}_1+{\mu }_2{‖X‖}_{\ast },\mathrm{s}.\mathrm{t}.X\ge 0,(2)$

where ${‖{\nabla }_{i,j}X‖}_1$ is the gradient norm, which is equivalent to extracting the segmented edge of the image, making the transformed image more sparse; ${‖X‖}_{\ast }$ is the matrix nuclear norm, indicating the low rank of the multispectral image matrix; μ₁, μ₂ ≥ 0 is the constraint item weight factor. In this paper, the TV-RANK compressed sensing algorithm is used for image reconstruction.

3 GISC spectral imaging scheme and experiment based on active laser illumination

3.1 Experimental setup

The GISC spectral imaging experimental system based on active laser illumination is shown in Figure 2. Among them, (1) is the imaging system; (2) is the target to be measured; (3) is the 532 nm solid-state laser. The experimental parameters are set as: the spectral range of the GISC camera is 440–700nm, there are 16 spectral channels, the spectral resolution is $<20$ nm, and the pixel resolution is $\ge 0.5$ mrad. The exposure time of the camera is 1 s, and the exposure time of the RGB camera is 0.3 s. The target to be tested is the color letter “QIT”, and the height of the letters is 3.3 cm. A 532 nm solid-state laser is used as the illumination light source, the target object to be measured is 3.5 m away from the imaging system, and the imaging field of view is 61 mm × 619 mm wide.

FIGURE 2

FIGURE 2. The left subfigure shows GISC spectral imaging experimental system based on active laser illumination, lights from the laser (3) pass through a reflector and an expander to illuminate the object (2), and the illuminated object is detected by the GISC camera (1). The subfigure in the right shows appearances of those three experimental parts in the left subfigure.

3.2 Experimental results and data analysis

Figure 3 shows the experimental results of spectral imaging of the target object irradiated by a 532 nm laser. When the average number of electrons recorded by the CCD is 402e-, the CCD detection signal obtained by the GISC camera through a single exposure is shown in Figure 3A. The image is shown in Figure 3B; the pseudo-color corresponding to the wavelength is added to Figure 3B, and its multispectral image is shown in Figure 3C; Figure 3D is 445, 460, 475 nm, 490, 500, 515, 530, 545, 560, 580, 590, 615, 635, 655, 675, 700 nm, the color fusion map of the 16 spectral channels is compared with Figure 2 in Figure 2, the shape and spectrum of the letter I The information restoration degree is high.

FIGURE 3

FIGURE 3. Experimental results of spectral imaging of the target object irradiated by a 532 nm laser. (A) Detection signal; (B) Spectral grayscale image; (C) Multispectral false color image; (D) Multispectral fusion image.

Figure 4 shows the experimental results of spectral imaging of the target object irradiated by a 650 nm laser. When the average number of electrons recorded by the CCD is 693e-, the CCD detection signal obtained by the GISC camera through a single exposure is shown in Figure 4A. The image is shown in Figure 4B; then add the false color corresponding to the wavelength to Figure 4B, which are 446, 461, 4745, 487, 501, 515, 530, 545, 561, 578, 590, 614 nm, 633, 653, 674, 698 nm, the multi-spectral grayscale image of 16 spectral channels is shown in Figure 4C; Figure 4D is the color fusion map compared with Figure 2 in Figure 2, the shape of the letter Q and spectral information reduction degree is high.

FIGURE 4

FIGURE 4. Experimental results of spectral imaging of the target object irradiated by 650 nm laser. (A) Detection signal; (B) Spectral grayscale image; (C) Multispectral false color image; (D) Multispectral fusion image.

In order to further illustrate the feasibility of the experiment, the spectral distribution of different colors in the target object from 440 to 700 nm was tested using the USB4000-VIS-NIR spectrometer, as shown in Figure 5A. The USB4000-VIS-NIR spectrometer was tested at 532 nm, and the intensity value of the target letter “Q” was 1264, the intensity value of the letter “I” was 14919, and the intensity value of the letter “T” was 2530. The intensity value of the letter “I” is the highest at 532 nm, so the target letter “I” can be accurately identified by irradiating the target object with 532 nm laser light. The spectral distribution curve of the letter “I” in the reconstructed image of the target object irradiated by 532 nm laser light is shown in Figure 5B. The USB4000-VIS-NIR spectrometer was tested at 650 nm, and the intensity value of the target letter “Q” was 5118, the intensity value of the letter “I″ was 852, and the intensity value of the letter “T” was 405. The intensity value of the letter “Q” is the highest at 650nm, so the target letter “Q” can be accurately identified by irradiating the target object with 650 nm laser light. The spectral distribution curve of the letter “Q” in the reconstructed image of the target object irradiated by 650 nm laser light is shown in Figure 5C. Those intensity value data representing spectral information of targets are detailed listed in Table 1.

FIGURE 5

FIGURE 5. Spectral distribution of USB4000-VIS-NIR spectrometer and color targets in multispectral images. (A) The spectral distribution curve of letter QIT in the test target of USB4000-VIS-NIR spectrometer (B) The spectral distribution curve of the letter “I″ in the reconstructed image of the target object irradiated by 532 nm laser (C) The spectral distribution curve of the letter “Q” in the reconstructed image of the target object irradiated by 650 nm laser.

TABLE 1

TABLE 1. List of intensity values of targets “Q” and “I” corresponding to Figure 5. Bold numbers show that the reconstructed intensity values of both letters reach the highest at the wavelengths correspond to the illuminated lasers.

4 Conclusion

Aiming at the problems of low detection sensitivity, many sampling times and serious light energy loss in traditional spectral imaging technology, a GISC spectral imaging target recognition scheme based on active laser illumination is proposed. This scheme combines laser technology, spectral imaging technology, correlated imaging technology and compressed sensing. The combination of theory has the advantages of high energy utilization rate and fast information acquisition efficiency. The imaging principle of GISC camera is theoretically analyzed, and the GISC spectral imaging target recognition experiment based on active laser illumination is completed in the laboratory environment. The results show that by selecting the appropriate wavelength of the laser light source, the GISC spectral imaging technology can be used to accurately identify the target spectral information. Next, how to establish the imaging quality evaluation and analysis system of the experimental results is a problem that needs to be solved in the future work.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Author contributions

ML and HW conceived the idea and designed the experiment. YL supervised the experiment and data analysis. HW performed the experiment. ML analyzed the data, conducted the image reconstruction. YL and ML wrote the manuscript.

Funding

This work is supported by the Department of science and technology of Jilin Province (Grant No. 20220204106YY and 20220401098YY).

Acknowledgments

The authors would like to thank Lijun Song in Jilin Engineering Laboratory for Quantum Information Technology and YL in Jilin Engineering Normal University for their meaningful discussion and helpful advice.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnotes

¹For the calibration, a monochormatic point source is used as the object rather than a real target, and a series of responses of the GISC camera system to the point source at different positions and wavelengths can be achieved by performing scanning-type pre-measurements.

References

1. Liu Z, Tan S, Wu J, Li E, Shen X, Han S. Spectral camera based on ghost imaging via sparsity constraints. Sci Rep (2016) 6:25718–0. doi:10.1038/srep25718

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Bai Z, Williams RJ, Kitzler O, Sarang S, Spence DJ, Wang Y, et al. Diamond brillouin laser in the visible. APL Photon (2020) 5:031301. doi:10.1063/1.5134907

CrossRef Full Text | Google Scholar

3. Chen H, Bai Z, Yang X, Ding J, Qi Y, Yan B, et al. Enhanced stimulated brillouin scattering utilizing Raman conversion in diamond. Appl Phys Lett (2022) 120:181103. doi:10.1063/5.0087092

CrossRef Full Text | Google Scholar

4. Shapiro JH, Boyd RW. The physics of ghost imaging. Quan Inf Process (2012) 11:949–93. doi:10.1007/s11128-011-0356-5

CrossRef Full Text | Google Scholar

5. Shih Y. The physics of ghost imaging. In: Classical, semi-classical and quantum noise. Berlin, Germany: Springer (2012). p. 169–222.

CrossRef Full Text | Google Scholar

6. Han S, Yu H, Shen X, Liu H, Gong W, Liu Z. A review of ghost imaging via sparsity constraints. Appl Sci (2018) 8:1379. doi:10.3390/app8081379

CrossRef Full Text | Google Scholar

7. Shih YH, Alley CO. New type of einstein-podolsky-rosen-bohm experiment using pairs of light quanta produced by optical parametric down conversion. Phys Rev Lett (1988) 61:2921–4. doi:10.1103/PhysRevLett.61.2921

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Bennink RS, Bentley SJ, Boyd RW. two-photon” coincidence imaging with a classical source. Phys Rev Lett (2002) 89:113601. doi:10.1103/physrevlett.89.113601

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Gatti A, Brambilla E, Bache M, Lugiato LA. Ghost imaging with thermal light: Comparing entanglement and classicalcorrelation. Phys Rev Lett (2004) 93:093602. doi:10.1103/physrevlett.93.093602

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Cheng J, Han S. Incoherent coincidence imaging and its applicability in x-ray diffraction. Phys Rev Lett (2004) 92:093903. doi:10.1103/physrevlett.92.093903

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Lyu M, Wang W, Wang H, Wang H, Li G, Chen N, et al. Deep-learning-based ghost imaging. Sci Rep (2017) 7:17865–6. doi:10.1038/s41598-017-18171-7

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Altmann Y, McLaughlin S, Padgett MJ, Goyal VK, Hero AO, Faccio D. Quantum-inspired computational imaging. Science (2018) 361:eaat2298. doi:10.1126/science.aat2298

PubMed Abstract | CrossRef Full Text | Google Scholar

13. He Y, Wang G, Dong G, Zhu S, Chen H, Zhang A, et al. Ghost imaging based on deep learning. Sci Rep (2018) 8:6469–7. doi:10.1038/s41598-018-24731-2

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Wu J, Shen X, Yu H, Liu Z, Tan S, Han S, et al. Snapshot compressive imaging by phase modulation. Acta Optica Sinica (2014) 34:1011005. doi:10.3788/aos201434.1011005

CrossRef Full Text | Google Scholar

15. Wu J, Li E, Shen X, Yao S, Tong Z, Hu C, et al. Experimental results of the balloon-borne spectral camera based on ghost imaging via sparsity constraints. IEEE Access (2018) 6:68740–8. doi:10.1109/access.2018.2879849

CrossRef Full Text | Google Scholar

16. Liu S, Liu Z, Wu J, Li E, Hu C, Tong Z, et al. Hyperspectral ghost imaging camera based on a flat-field grating. Opt Express (2018) 26:17705–16. doi:10.1364/oe.26.017705

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Liu S, Liu Z, Hu C, Li E, Shen X, Han S. Spectral ghost imaging camera with super-Rayleigh modulator. Opt Commun (2020) 472:126017. doi:10.1016/j.optcom.2020.126017

CrossRef Full Text | Google Scholar

18. Chu C, Liu S, Liu Z, Hu C, Zhao Y, Han S. Spectral polarization camera based on ghost imaging via sparsity constraints. Appl Opt (2021) 60:4632–8. doi:10.1364/ao.417022

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Cui C, Wang Y, Lu Z, Bai Z, Yuan H, Chen Y, et al. High-visibility pseudothermal light source based on a cr4+: Yag passively q-switched single-longitudinal-mode laser. Int J Opt (2020) 2020:1–7. doi:10.1155/2020/3160837

CrossRef Full Text | Google Scholar

20. Meixuan L, Hong W, Xiaohan L, Ming L, Lijun S. Research on multispectral correlation imaging of moving target based on phase modulation. Infrared Laser Eng (2021) 50:20210184–1.

Google Scholar

21. Tan S. The application research of structured compressed sensing in multispectral ghost imaging. Ph.D. thesis. Shanghai: Shanghai Institute of Optics and Fine Mechanics, University of Chinese Academy of Sciences (2015).