Editorial: Multi-sensor imaging and fusion: methods, evaluations, and applications

Editorial on the Research Topic
Multi-sensor imaging and fusion: methods, evaluations, and applications

Introduction

The technology of multi-sensor imaging and fusion plays an increasingly important role in various fields such as remote sensing [1], medical imaging [2], contraband detection [3], and engineering construction [4]. Multi-sensor image fusion focuses on processing images of the same object or scene captured by multiple sensors, which complement and combine various sensors with multi-level and multi-spatial information, ultimately providing a consistent interpretation of the observed environment [5]. In recent years, multi-sensor image fusion has become a highly active topic, and various fusion methods have been proposed. Moreover, the performance evaluation and downstream applications of multi-sensor imaging and fusion technology are receiving increasing attention. This Research Topic highlights advanced research related to multi-sensor imaging and fusion technology, including image detection and fusion methods, objective evaluation methods, and specific applications in engineering problems. After a thorough peer-review process, all 17 of the articles submitted to this Research Topic were accepted for publication. The following summarizes the main research findings of these works from three aspects.

Imaging detection, feature extraction, and fusion methods in multi-sensors

Object detection is an important application of multi-sensor imaging and fusion technologies. He et al. proposed a deep learning object detection network, MSS-YOLOv5, which integrates multi-scale information to enhance feature robustness, improves pooling methods to capture more details, and introduces an angle cost with new weights to accelerate network convergence and improve accuracy. Yang et al. proposed a multi spectral pedestrian detection algorithm that includes a cascaded information enhancement module and a cross-modal attention feature fusion module to enhance pedestrian features and reduce background interference. Chen et al. proposed a network guided by atomic number Z (ZPGNet), which is used to accurately detect prohibited items in complex X-ray images while reducing the collection cost of atomic number images. Zhou et al. proposed an unsupervised smoke detection algorithm that reduces domain differences and improves the generalization ability of the model through feature alignment and fusion. Meanwhile, multi-level feature fusion of network depth enhances the recognition ability of small targets. Wang et al. proposed an unsupervised method that uses an asymmetric convolution feature extraction network and a pose estimation network with attention mechanisms to solve the problem of monocular depth estimation. They also used a loss function that minimizes the reprojection error to solve the occlusion problem in the projection process. Chen et al. proposed a two-stage domain gap-aware framework to eliminate the bias between the synthetic low-light and real low-light domains, thereby enhancing the generalization capability of low-light image enhancement methods. By utilizing a reverse domain distance guidance strategy, the network can better handle low-light image areas that do not align with the real-world distribution. Liu et al. proposed a new method for the fusion of infrared and visible light images, optimizing edge detail through separate processing of source images and edge detail information. Their two-branch framework extracts features and edge map features directly from source images, and a large number of experiments have verified the effectiveness of their method. Zhou et al. proposed a model to solve the problem of semantic alignment and feature extraction in person text-image matching. The model achieves more efficient feature matching and extraction by adding consistent, clear semantic information and applying an information supplementation network. Li et al. proposed a novel cross-modal hashing method named FSSPDH, which preserves the intrinsic attributes of each modality by learning the hash codes of each modality and constructing a fine-grained similarity matrix. In addition, they used quantization loss to learn hash codes, effectively reducing information loss during the quantization process. Jin et al. proposed a polarization image fusion method that fuses intensity images and linear polarization degrees. It processes the base layer and detail layer through quality evaluation and attention mechanisms. The base layer ensures high contrast of the fused image through a quality evaluation unit, and the detail layer improves the preservation of detail information through an attention enhancement unit.

Objective evaluation methods in multi-sensor imaging

In medical image analysis and evaluation, Xu et al. developed a 16-electrode capacitance imaging (ECT) system for two-dimensional tomography of intracerebral hemorrhage (ICH). The feasibility of ECT in ICH imaging was confirmed through simulation and physical experiments. Qiu et al. retrospectively evaluated patients who underwent stent-assisted coiling (SAC) for intracranial aneurysms, focusing primarily on the rate of embolization and complications. The results showed that all hemodynamic parameters significantly decreased after SAC with four different stents, and laser-cut stents seemed to be more effective than woven stents in reducing aneurysm hemodynamics. Finally, there was no significant difference between the follow-up RROC grades of the four stents. In traffic safety evaluation, Yang et al. proposed a method for immersive tunnel traffic safety evaluation based on the degradation of lighting performance using big data technology. The method utilized numerical simulation, small target recognition tests, and developed a real-time model to illustrate the relationship between the degradation of lighting performance and visual cognition.

Specific applications of multi-sensor technology in engineering problems

Multi-sensor technologies play a significant role in fault detection and signal monitoring. Liu et al. proposed a novel method that combines Improved Energy Fluctuation Index (IEFI) and Modified Variational Mode Decomposition (MVMD) to overcome limitations related to the mode number and balancing parameters. This method can effectively resist interference and accurately extract fault features. Experimental results demonstrate its superior performance in fault signal detection. Meanwhile, Guo et al., by introducing close-range photo grammetry, successfully monitored the differential deformation of immersed tunnel element joints. They not only developed a micro-displacement correction algorithm based on three-dimensional calibration objects, but also a fully automatic system for monitoring the differential deformation of immersed tunnel element joints. In emotion analysis, Yan et al. proposed a Modal Smoothing Fusion Network (MSFNet) that can effectively bridge the semantic gap between text and image at the aspect level of emotional expression. Through feature smoothing and multi-channel attention mechanisms, the model has improved performance in emotion classification. Facing the challenge of defect classification, Liang et al. proposed a multi-level semantic method based on residual adversarial learning for sample enhancement and defect classification. By introducing residual modules and multiple convolutional layers, the network structure is optimized, and the feature extraction capability is enhanced. A multi-level semantic extractor is designed, combined with Wasserstein loss, to solve the instability of network training. This method can generate high-quality defect samples and accurately classify defects.

Conclusion

To conclude, a wide range of related topics have been collected for the special issue. Especially some of the hot Research Topics are from object detection, medical image analysis and evaluation, signal monitoring and fault detection.

Special thanks to Frontier in Physics for the support and efforts provided to this special issue. We would also like to thank all the authors who contributed their original work to this special issue and all the reviewers for sharing their thoughts on the submissions. We hope that this special issue can inspire the researchers in the field and push the research on multi-sensor imaging and fusion to new frontiers.

Author contributions

GQ: Writing–original draft, Writing–review and editing. ZZ: Writing–original draft, Writing–review and editing. YL: Writing–original draft, Writing–review and editing. HL: Writing–original draft, Writing–review and editing. BX: Writing–original draft, Writing–review and editing.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. Zhu Z, Luo Y, Qi G, Meng J, Li Y, Mazur N. Remote sensing image defogging networks based on dual self-attention boost residual octave convolution. Remote Sensing (2021) 13(16):3104. doi:10.3390/rs13163104

CrossRef Full Text | Google Scholar

2. Zhu Z, He X, Qi G, Li Y, Cong B, Liu Y. Brain tumor segmentation based on the fusion of deep semantics and edge information in multimodal MRI. Inf Fusion (2023) 91:376–87. doi:10.1016/j.inffus.2022.10.022

CrossRef Full Text | Google Scholar

3. Zhu Z, Qi G, Lei Y, Jiang D, Mazur N, Liu Y, et al. A long short-term memory neural network based simultaneous quantitative analysis of multiple tobacco chemical components by near-infrared hyperspectroscopy images. Chemosensors (2022) 10(5):164. doi:10.3390/chemosensors10050164

CrossRef Full Text | Google Scholar

4. Tang L, Huang H, Zhang Y, Qi G, Yu Z. Structure-embedded ghosting artifact suppression network for high dynamic range image reconstruction. Knowledge-Based Syst (2023) 263:110278. doi:10.1016/j.knosys.2023.110278

CrossRef Full Text | Google Scholar

5. Zhu Z, Wei H, Hu G, Li Y, Qi G, Mazur N. A novel fast single image dehazing algorithm based on artificial multiexposure image fusion. IEEE Trans Instrumentation Meas (2021) 70:1–23. doi:10.1109/tim.2020.3024335

CrossRef Full Text | Google Scholar