- 1Visual Computing Center, King Abdullah University of Science and Technology, Thuwal, Saudi Arabia
- 2Department of Electrical Engineering & Computer Sciences, University of California, Berkeley, Berkeley, CA, United States
Holographic display is an ultimate three-dimensional (3D) display technique that can produce the wavefront of 3D objects. The dynamic holographic display usually requires a spatial light modulator (SLM) with a following 4f system to eliminate the unnecessary orders produced by the grating structure of the SLM. We present a technique that displays the images without the 4f system. We detect the unnecessary wavefield by phase-shifting holography and suppress it using computational optimization. Experimental results are presented to verify the proposed method.
1 Introduction
Holography could create a light field identical to the one generated by a real three-dimensional (3D) object. Thus, it is regarded as the ultimate display technique for 3D displays (Hong et al., 2011). One of the most challenging issues in holographic displays is modulating the complex wavefield at wavelength-scale, pixel by pixel, yielding ultra-high spatial resolution. The typical light modulation devices are spatial light modulators (SLMs) or digital micromirror devices (DMDs). The commercial SLMs or DMDs are mainly phase-only or amplitude-only, and the practical realization of a complex one is challenging. Various approaches have been reported to conduct complex field modulation in optical or computational manners (Hsueh and Sawchuk, 1978; Juday et al., 1991; Gregory et al., 1992; Neto et al., 1996; de Bougrenet de la Tocnaye and Dupont, 1997; Arrizón et al., 1998; Birch et al., 2000; Hsieh, 2007; Bingxia Wang et al., 2021). The optical approaches usually achieve complex modulation by integrating and controlling phase-only or amplitude-only SLMs in different ways (Juday et al., 1991; Gregory et al., 1992; Neto et al., 1996; de Bougrenet de la Tocnaye and Dupont, 1997; Hsieh, 2007), while the computational techniques convert the complex field into equivalent phase-only (Hsueh and Sawchuk, 1978; Birch et al., 2000) or amplitude-only (Arrizón et al., 1998; Bingxia Wang et al., 2021) ones that adapt to the physical constraints of commercial light modulators. The latter is also referred to as hologram encoding. Analytical solutions and computational optimization are two main approaches for hologram encoding. Detour phase (Arrizón et al., 1998; Bingxia Wang et al., 2021) and double-phase methods (Hsueh and Sawchuk, 1978; Song et al., 2012; Mendoza-Yero et al., 2014) are typical analytical techniques, which are fast but with compromised qualities, and can be easily applied to arbitrary complex holograms. Optimization techniques such as Gerchberg–Saxton (GS) (Gerchberg, 2002; Sun et al., 2018; Chen et al., 2020; Wu et al., 2021) are time-consuming with improved image quality. However, these are difficult to be applied to arbitrary complex fields (Chakravarthula et al., 2019). The rasterization structure of SLM or DMD also introduces multiple orders. The fill factor of the pixels results in the zero-order issue, which interferes with and distorts the modulated light wave. To improve the displayed image quality, it requires suppression of the zero-order. The 4f optical filter system (Arrizón et al., 2007; Liang et al., 2012; Ronzitti et al., 2012; Siemion et al., 2012; Improso et al., 2014) is a common approach that can block the zero-order in the Fourier plane. The 4f system and other variants of zero-order blocking techniques result in a non-accessible region in the final reconstruction because any part of the desired pattern near the zero-order area would also be affected. An ideal solution would be to create a correction beam with the same profile as the zero-order together with the desired target (Palima and Daria, 2007), which could create a destructive interference with the unwanted beam. However, developing complex field optimization may be the barrier to this approach (Improso et al., 2017).
Herein, we propose a computational holographic display technique that integrates phase-shifting holography and automatic differentiable (AD) optimization. The former helps us detect the unwanted complex wavefield in a holographic display system, and the latter achieves complex field optimization to suppress the unwanted wavefields. It shows that optimization through AD can obtain a phase-only hologram that acts equally as a complex field in an efficient way. In the simulation, the peak signal-to-noise (PSNR) and structural similarity index measure (SSIM) can reach 50 dB and 0.9 in around one second. Thanks to the powerful AD optimization, we can achieve comparable image quality in hologram display while deputing the 4f system. We present the methodology in Section 2 and experimental results in Section 3.
2 Methods
Figure 1A represents a typical holographic display system, wherein a laser light beam illuminates an SLM that shows the holograms. The diffracted light goes through a 4f system and reproduces the desired objects. The 4f system, as denoted by the dashed rectangle in Figures 1A, is used to block unwanted terms induced by the rasterized SLM and its limited fill factor. It usually keeps only the first order of the diffraction (Zaperty et al., 2018). The 4f makes the system bulky and may introduce extra aberrations if the focal lengths of the lenses are small. Therefore, it is demanded to be eliminated. The ideal display system is shown in Figure 1B.
However, without the 4f system, the detected images along the optical axis is an interference pattern between the directly reflected beam and the diffracted beam from the SLM, as shown in Figure 2 and Eq. 1:
where
2.1 Unwanted Wavefront Detection
Figure 2 and Eq. 1 show the unwanted complex amplitude of the wavefield we want to suppress is an interference pattern. Obtaining it is a holographic imaging problem. Therefore, we introduce the four-step phase-shifting holography (Yamaguchi and Zhang, 1997; Jeong et al., 2008) to measure it. We treat the wavefield udead(x, y) at the camera sensor plane as the target and the wavefield from the SLM active area as the reference beam and perform phase-shifting hologram recording. The four holograms are taken by shifting the reference beams’ phase with a step of 0.5π; that is, α is of 0π, 0.5π, π and 1.5π, respectively. Thus, the captured holograms under phase-shifting α can be represented by
Thus, the phase and amplitude of the light beam of
It should be mentioned that we suppose the dead area of the SLM acts as a “mirror.” However, the cover glass of the SLMs may be imperfect, which could result in an inaccurate estimation of the zero-order.
2.2 Unwanted Wavefront Suppression Through Automatic Differentiable Optimization
With the detected complex amplitude of the wavefield udead(x, y), we need to calculate a desired phase only hologram that can be loaded into the active area of the SLM to produce the desired image after interfering with udead(x, y). Suppose b is the known target, the problem is to find a phase ϕ(x, y), such that for a known distance z, the interference between the propagated wavefield and udead(x, y) approaches the target b, which can be presented as
We can optimize the above equation using a gradient descent. At iteration n, update ϕn+1 given a step size τ:
with the derivatives of
Computing
where
where H is the transfer function of free-space wave propagation, ◦ is the element-wise product, and
From the aforementioned equations, we get the classic definition of Wirtinger calculus:
For step s and loss L, we have
The complex numbers are represented by the cascading of real and imaginary parts in the last dimension. The derivative implementation of the complex elementary functions follows the gradient rules in Table 1 of Jurling and Fienup (2014). We implement the proposed method with PyTorch 1.8 (Chilamkurthy and Tanamala, 2019), while the derivatives are calculated automatically. All presented results in the following sections were performed on a workstation with an Intel Core i7-6820 CPU and an NVIDIA GTX1080 GPU. Figure 3 shows the image quality of the proposed AD and the traditional Gerchberg–Saxton (GS) concerning the computation time. It takes around one second to optimize one 1,024 × 1,024 hologram with a PSNR ratio more significant than 70 dB and SSIM larger than 0.8. In comparison, the GS method can hardly reach a PSNR larger than 20 dB and SSIM larger than 0.6.
FIGURE 3. Image quality of the proposed AD and the traditional Gerchberg–Saxton (GS) versus computation time.
2.3 Free-Space Wavefield Diffraction
The wave propagation mentioned in the previous sections is free-space diffraction and is implemented with the angular spectrum method (ASM) of the Rayleigh–Sommerfeld diffraction (Goodman, 2005):
where u(x, y, z) is the complex amplitude at a plane located at z,
3 Experiments
The SLM in the experiment is HOLOEYE PLUTO with a full resolution of 1,920 × 1,080 pixels and 8 µm pixel pitch. Moreover, the image sensor is Point Grey Research GS3-U3-50S5C, with a resolution of 2,248 × 2,048 pixels and 3.45 µm pixel pitch. The light source was a laser diode with a center wavelength of 532 nm, and the image sensor distance was 250 mm away from the SLM. The four phases used for detection udead(x, y) are shown in the above row of Figure 4A, and the corresponding captured holograms are shown in the below row. The phase-shifting hologram reconstruction is shown in Figure 4B, and the propagated wavefield at the SLM plane is shown in Figure 4C. We conducted a calibration to match the size difference between the image sensor and the SLM.
FIGURE 4. Four phases and the holograms (A), and the calculated wavefield at the image sensor (B) and SLM (C) planes.
For a target image shown in Figure 5A, we calculated the phase-only holograms without and with the unwanted term suppression, which are shown in the left column of Figures 5B,C. Whether the difference between the conventional and the proposed method is optimizing the zero-order term or not, the computational calculation time is the same. However, we conducted the phase-shifting holography in the proposed method to obtain the zero-order field. The calculation time can be overlooked, but the capture takes time. This can be improved by automatically synchronizing and controlling the SLM and the camera. The corresponding experimentally reconstructed images are shown in the right columns. Comparing the proposed method with the conventional one, we can observe less background noise in Figure 5C, indicating that the proposed technique suppresses some of the unwanted terms. However, some noise still exists, which may be due to the inaccurate modeling of the optical system. This could be improved if we further consider the laser speckle and the SLM’s fill factor that reflects only part of the incident light wave.
FIGURE 5. Experimental results of conventional (B) and proposed (C) methods for displaying a target image (A).
4 Discussion and Conclusion
We presented a computational holographic display technology that can achieve a lightweight holographic display with high quality. The results show that the automatic differentiable complex wavefield optimization can suppress the unwanted wavefield with the assistance of phase-shifting holography. The automatic differentiable complex wavefield optimization can also be applied to other optical systems requiring aberration or system error corrections.
Data Availability Statement
The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author.
Author Contributions
NC proposed the idea and conducted the simulation and experiments. CW and WH worked together on the manuscript writing.
Funding
This work was supported by the KAUST individual baseline funding.
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s Note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations or those of the publisher, the editors, and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
Arrizón, V., Kinne, S., and Sinzinger, S. (1998). Efficient Detour-phase Encoding of One-Dimensional Multilevel Phase Diffractive Elements. Appl. Opt. 37, 5454. doi:10.1364/ao.37.005454
Arrizón, V., Ruiz, U., Carrada, R., and González, L. A. (2007). Pixelated Phase Computer Holograms for the Accurate Encoding of Scalar Complex fields. J. Opt. Soc. Am. A. 24, 3500. doi:10.1364/josaa.24.003500
Birch, P., Young, R., Chatwin, C., Farsari, M., Budgett, D., and Richardson, J. (2000). Fully Complex Optical Modulation with an Analogue Ferroelectric Liquid crystal Spatial Light Modulator. Opt. Commun. 175, 347–352. doi:10.1016/s0030-4018(00)00478-810.1016/s0030-4018(00)00478-8
Blennow, M. (2018). Mathematical Methods for Physics and Engineering. New York, NY: CRC Press. doi:10.1201/b22209
Chakravarthula, P., Peng, Y., Kollin, J., Fuchs, H., and Heide, F. (2019). Wirtinger Holography for Near-Eye Displays. ACM Trans. Graph. 38, 1–13. doi:10.1145/3355089.3356539
Chen, C., Lee, B., Li, N.-N., Chae, M., Wang, D., Wang, Q.-H., et al. (2021). Multi-depth Hologram Generation Using Stochastic Gradient Descent Algorithm with Complex Loss Function. Opt. Express, 29, 15089. doi:10.1364/oe.425077
Chen, L., Zhang, H., Cao, L., and Jin, G. (2020). Non-iterative Phase Hologram Generation with Optimized Phase Modulation. Opt. Express 28, 11380. doi:10.1364/oe.391518
Chilamkurthy, S., and Tanamala, S. (2019). Deep Learning and Medical Diagnosis - Authors' Reply. The Lancet 394, 1711. doi:10.1016/s0140-6736(19)32614-510.1016/s0140-6736(19)32614-5
de Bougrenet de la Tocnaye, J. L., and Dupont, L. (1997). Complex Amplitude Modulation by Use of Liquid-crystal Spatial Light Modulators. Appl. Opt. 36, 1730. doi:10.1364/ao.36.001730
Gerchberg, R. W. (2002). A New Approach to Phase Retrieval of a Wave Front. J. Mod. Opt. 49, 1185–1196. doi:10.1080/09500340110114425
Gregory, D. A., Kirsch, J. C., and Tam, E. C. (1992). Full Complex Modulation Using Liquid-crystal Televisions. Appl. Opt. 31, 163. doi:10.1364/ao.31.000163
Griewank, A., and Walther, A. (2008). Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation. University City, Philadelphia: Society for Industrial and Applied Mathematics. doi:10.1137/1.9780898717761
Hong, J., Kim, Y., Choi, H.-J., Hahn, J., Park, J.-H., Kim, H., et al. (2011). Three-dimensional Display Technologies of Recent Interest: Principles, Status, and Issues [invited]. Appl. Opt. 50, H87. doi:10.1364/ao.50.000h87
Hsieh, M.-L. (2007). Improvement of the Complex Modulated Characteristic of Cascaded Liquid crystal Spatial Light Modulators by Using a Novel Amplitude Compensated Technique. Opt. Eng. 46, 070501. doi:10.1117/1.2750658
Hsueh, C. K., and Sawchuk, A. A. (1978). Computer-generated Double-phase Holograms. Appl. Opt. 17, 3874. doi:10.1364/ao.17.00387410.1364/ao.17.003874
Improso, W. D. G. D., Tapang, G. A., and Saloma, C. A. (2017). “Suppression of Zeroth-Order Diffraction in Phase-Only Spatial Light Modulator via Destructive Interference with a Correction Beam,” in Proceedings of the 5th International Conference on Photonics, Optics and Laser Technology (Porto, Portugal: SCITEPRESS - Science and Technology Publications). doi:10.5220/0006129802080214
Improso, W. D. G., Hilario, P. L. A., and Tapang, G. (2014). “Zero Order Diffraction Suppression in a Phase-Only Spatial Light Modulator via the GS Algorithm,” in Frontiers in Optics 2014 (Tucson, AZ: OSA). doi:10.1364/fio.2014.ftu4c.3
Jeong, M.-O., Kim, N., and Park, J.-H. (2008). Elemental Image Synthesis for Integral Imaging Using Phase-Shifting Digital Holography. J. Opt. Soc. Korea 12, 275–280. doi:10.3807/josk.2008.12.4.275
Juday, R. D., and Florence, J. M. (1991). “Full-complex Modulation with Two One-Parameter Slms,” in Wave Propagation and Scattering in Varied Media II. Editor VK. Varadan (San Diego, CA: SPIE). doi:10.1117/12.49656
Jurling, A. S., and Fienup, J. R. (2014). Applications of Algorithmic Differentiation to Phase Retrieval Algorithms. J. Opt. Soc. Am. A. 31, 1348. doi:10.1364/josaa.31.001348
Liang, J., Wu, S.-Y., Fatemi, F. K., and Becker, M. F. (2012). Suppression of the Zero-Order Diffracted Beam from a Pixelated Spatial Light Modulator by Phase Compression. Appl. Opt. 51, 3294. doi:10.1364/ao.51.003294
Mendoza-Yero, O., Mínguez-Vega, G., and Lancis, J. (2014). Encoding Complex fields by Using a Phase-Only Optical Element. Opt. Lett. 39, 1740. doi:10.1364/ol.39.001740
Neto, L. G., Roberge, D., and Sheng, Y. (1996). Full-range, Continuous, Complex Modulation by the Use of Two Coupled-Mode Liquid-crystal Televisions. Appl. Opt. 35, 4567. doi:10.1364/ao.35.004567
Palima, D., and Daria, V. R. (2007). Holographic Projection of Arbitrary Light Patterns with a Suppressed Zero-Order Beam. Appl. Opt. 46, 4197. doi:10.1364/ao.46.004197
Peng, Y., Choi, S., Padmanaban, N., Kim, J., and Wetzstein, G. (2020). “Neural Holography,” in ACM SIGGRAPH 2020 Emerging Technologies (Virtual Event, United States: ACM). doi:10.1145/3388534.3407295
Ronzitti, E., Guillon, M., de Sars, V., and Emiliani, V. (2012). LCoS Nematic SLM Characterization and Modeling for Diffraction Efficiency Optimization, Zero and Ghost Orders Suppression. Opt. Express 20, 17843. doi:10.1364/oe.20.017843
Siemion, A., Sypek, M., Suszek, J., Makowski, M., Siemion, A., Kolodziejczyk, A., et al. (2012). Diffuserless Holographic Projection Working on Twin Spatial Light Modulators. Opt. Lett. 37, 5064. doi:10.1364/ol.37.005064
Song, H., Sung, G., Choi, S., Won, K., Lee, H.-S., and Kim, H. (2012). Optimal Synthesis of Double-phase Computer Generated Holograms Using a Phase-Only Spatial Light Modulator with Grating Filter. Opt. Express 20, 29844. doi:10.1364/oe.20.029844
Sun, P., Chang, S., Liu, S., Tao, X., Wang, C., and Zheng, Z. (2018). Holographic Near-Eye Display System Based on Double-Convergence Light Gerchberg-saxton Algorithm. Opt. Express 26, 10140. doi:10.1364/oe.26.010140
Wang, B., Hong, X., Wang, K., Chen, X., Liu, S., Krolikowski, W., et al. (2021). Nonlinear Detour Phase Holography. Nanoscale 13, 2693–2702. doi:10.1039/d0nr07069f
Wang, C., Chen, N., and Heidrich, W. (2021). Towards Self-Calibrated Lens Metrology by Differentiable Refractive Deflectometry. Opt. Express 29, 30284–30295. doi:10.1364/oe.433237
Wu, Y., Wang, J., Chen, C., Liu, C.-J., Jin, F.-M., and Chen, N. (2021). Adaptive Weighted Gerchberg-saxton Algorithm for Generation of Phase-Only Hologram with Artifacts Suppression. Opt. Express 29, 1412. doi:10.1364/oe.413723
Yamaguchi, I., and Zhang, T. (1997). Phase-shifting Digital Holography. Opt. Lett. 22, 1268. doi:10.1364/ol.22.001268
Zaperty, W., and Kozacki, T. (2018). “Numerical Model of Diffraction Effects of Pixelated Phase-Only Spatial Light Modulators,” in Speckle 2018: VII International Conference on Speckle Metrology. Editors M Józwik, L. R Jaroszewicz, and M Kujawińska (Janów Podlaski, Poland: SPIE). doi:10.1117/12.2319739
Keywords: holography, computational display, optimization, display, imaging
Citation: Chen N, Wang C and Heidrich W (2022) Compact Computational Holographic Display. Front. Photonics 3:835962. doi: 10.3389/fphot.2022.835962
Received: 15 December 2021; Accepted: 31 January 2022;
Published: 25 February 2022.
Edited by:
Yaping Zhang, Kunming University of Science and Technology, ChinaReviewed by:
Jia Jia, Independent Researcher, Shenzhen, ChinaTomasz Kozacki, Warsaw University of Technology, Poland
Copyright © 2022 Chen, Wang and Heidrich. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Ni Chen, bmkuY2hlbkBrYXVzdC5lZHUuc2E=