Skip to main content

ORIGINAL RESEARCH article

Front. Comput. Sci.
Sec. Computer Vision
Volume 6 - 2024 | doi: 10.3389/fcomp.2024.1388174

Alignment of a 360 • Image with Posed Color Images for Locally Accurate Texturing of 3D Mesh

Provisionally accepted
Sanjay Rijal Sanjay Rijal *Madhav Om Madhav Om Bishwash Khanal Bishwash Khanal Vaghawan Ojha Vaghawan Ojha
  • IKebana Solutions LLC, Tokyo, Japan

The final, formatted version of the article will be published soon.

    With the popularity of 3D content like virtual tours, the challenges of 3D data registration have become increasingly significant. Registration of heterogeneous data obtained from 2D and 3D sensors is required to create photo-realistic 3D models. However, the alignment of 2D images with 3D models introduces a significant challenge due to their inherent differences. This paper introduces a rigorous mathematical approach to align a 360 • image with its corresponding 3D model generated from images with known camera poses followed by texture projection on the model. We use SIFT feature descriptors enhanced with a homography-based metric to establish correspondences between the faces of a cubemap and the posed images. To achieve optimal alignment, we use a nonlinear least squares optimization technique with a custom objective function. Subsequently, the outcomes of the alignment process are evaluated through texturing using a customized raytracing algorithm. The resulting projections are compared against the original textures, with a comprehensive assessment of the alignment's fidelity and precision.

    Keywords: Cubemap projection, least squares optimization, Raytracing, Texturing, 360, image alignment

    Received: 19 Feb 2024; Accepted: 27 Aug 2024.

    Copyright: © 2024 Rijal, Om, Khanal and Ojha. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

    * Correspondence: Sanjay Rijal, IKebana Solutions LLC, Tokyo, Japan

    Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.