Evaluation of photogrammetry for medical application in cardiology

Background: In the field of medicine, photogrammetry has played for long time a marginal role due to the significant amount of work required that made it impractical for an extended medical use. Developments in digital photogrammetry occurred in the recent years, that have steadily increased the interest and application of this technique. The present study aims to compare photogrammetry reconstruction of heart with computed tomography (CT) as a reference.

Methods: The photogrammetric reconstructions of digital images from ECG imaging derived images were performed. In particular, the ventricles of 15 patients with Brugada syndrome were reconstructed by using the free Zephyr Lite software. In order to evaluate the accuracy of the technique, measurements on the reconstructions were compared to patient-specific CT scan imported in ECG imaging software UZBCIT.

Result: The results showed that digital photogrammetry in the context of ventricle reconstruction is feasible. The photogrammetric derived measurements of ventricles were not statistically different from CT scan measurements. Furthermore, the analysis showed high correlation of photogrammetry reconstructions with CT scan and a correlation coefficient close to 1.

Conclusion: It is possible to reproduce digital objects by photogrammetry if the process described in this study is performed. The reconstruction of the ventricles from CT scan was very close to the values of the respective photogrammetric reconstruction.

Introduction

Automatic digital photogrammetry is a surveying methodology that allows to build a three-dimensional model from digital photographs or videos. It offers an attractive alternative since it enables a radiation-free reconstruction for the patient. In the medical field, photogrammetry has played for a long time a marginal role due to the significant amount of work required. The developments of digital photogrammetry in the recent years increased the interest and the possible applications of this technique (Mitchell, 1995). In particular, the use of a photogrammetric approach allows the extraction of information related to the anatomical structure and the cardiac mapping under evaluation.

This can be used in cardiology to reconstruct the heart anatomy with the aim to build patient-specific models to guide invasive procedures, such as catheter ablation. The latter is based on targeting and elimination of an arrhythmogenic area in the heart, while sparing healthy tissue. The culprit region can be outlined by mapping techniques that allow to visualize the abnormal electrical activity of the region of interest (Pannone et al., 2022a; Pannone et al., 2022b; Pannone et al., 2022c).

Photogrammetry combines 3D volume acquisition and mapping in a single step. Indeed, often the information elaborated for mapping cannot be extracted from the mapping software but data can be used only as a marker of the presence of an arrhythmogenic area, as in the case of Brugada syndrome (BrS) (Cheniti et al., 2019). In this case the pathological area is an area of conduction delay in the absence of structural abnormalities.

Therefore, the aim of this study is to describe a novel photogrammetry procedure for the creation of a 3D model to plan and target the pathological area during epicardial ablation of Brugada syndrome.

Materials and methods

Non-invasive ECG imaging mapping

Fifteen consecutive patients with BrS underwent CT scan and non-invasive ECG imaging (ECGi) with CardioInsight™ Non-invasive 3D Mapping System (Medtronic Inc, Minneapolis, MN) technology. Previous studies in Brugada syndrome, showed that the ECGi mapping can delineate the pathological area and this is consistent with invasive mapping (Rudy and Messinger-Rapport, 1988; Zhang et al., 2016; Pappone et al., 2017; Salghetti et al., 2019). CardioInsight™ Non-invasive 3D Mapping System (Medtronic Inc, Minneapolis, MN) technology uses a vest with 252 electrodes applied to the patient; the unipolar electrocardiograms obtained from electrodes are projected on the 3D geometry derived from patient-specific CT scan. Using CardioInsight technology, activation time (AT), repolarization time (RT), and activation-repolarization interval (ARI) maps are obtained automatically.

The activation time for each unipolar electrogram was defined by the elapsed time between the onset of depolarization (QRS) and the maximum negative slope (maximum negative dV/dt) of the unipolar EGM. The activation delay was assessed through a post-analysis process with next-generation software (UZBCIT) developed for our Center by Medtronic Inc. that can provide a quantitative output to determine the pathological area in the right ventricle outflow tract. In particular, the region of interest was identified by the ECGi mapping examination, together with an experienced cardiologist, based on the late activation time.

The mapping software offers the possibility to visualize the volumetric mesh of the patient ventricle and electrical conduction. However, the anatomy and mapping information cannot be directly extracted from the software. Photogrammetry was developed to extract not only the anatomical structure but also the cardiac mapping information in a single 3D object. The studies involving human participants were reviewed and approved by Ethical Committee of UZ Brussel, Brussels, Belgium.

Photographic report

The ventricles reconstructed in the ECGi software (UZBCIT) were photographed from different angles, which corresponded to the frontal, right lateral, left lateral, posterior, and superior projections. The projections of the ventricles were virtually divided for the purpose of making the photographs. Photographs were made concentrically with respect to the z-axis at angles of 30°,45°, 60°, 90°, 120°, 135°,150°, 180°, 210°, 225°, 240°, 270°, 300°, and 330° with respect to three elevation angles on the x-axis of −30°, 0°, 30°. Thus, reproduction of each ventricle was performed with 45 photographs (Figure 1). Figure 1 shows how the photographs overlap. This is necessary so that the software, upon processing, easily recognizes similarities between photographs depicting adjacent areas (Figure 1).

FIGURE 1

FIGURE 1. Image collection from the UZCBIT software. Starting images for photogrammetric reconstruction; (A) Image 30° angle with elevation angle on the x-axis of 0°, (B) Image 60° angle with elevation angle on the x-axis of 0°, (C) Image 120° angle with elevation angle on the x-axis of 0°, (D) Image 150° angle with elevation angle on the x-axis of 0°, (E) Image 210° angle with elevation angles on the x-axis of 0°, (F) Image 240° angle with elevation angle on the x-axis of 0°, (G) Image 300° angle with elevation angle on the x-axis of 0°, (H) Image 330° angle with elevation angle on the x-axis of 0°, (I) Image 30° angle with elevation angle on the x-axis of 30°, (J) Image 45° angle with elevation angle on the x-axis of 30°.

In order to avoid errors and increase the precision of the results, the images were processed (pre-processed) using Masquerade software. The latter is a support application for the Zephir software, which allows to highlight the object to be reconstructed. In other words, Zephir will use for the reconstruction of the object, only the areas previously selected on Masquerade, ignoring the rest.

Image processing by photogrammetry

The images were processed using a free version of the ZephirLite program (3DFlow^©). The final 3D volume can be created by selecting a video or a series of images from which the software will extrapolate the dense point cloud and then form the 3D mesh (Figure 2).

FIGURE 2

FIGURE 2. Model produced with photogrammetry. Photogrammetric reconstruction view in Blender. (A) Antero-posterior view; (B) Latero-lateral view, (C) Postero-anterior view.

Furthermore, the software enables the addition of a new calibration, and the modification of the automatic calibration performed previously. Afterwards, interactions are created with an interface that provides selection regarding camera orientation, category, and presets. The software works semi-automatically. Once the photos are loaded, the masks obtained with Masquerade are loaded and the parameters for reconstruction are set. The categories presets are divided into generic object, near objects, urban environment and human body; thus, based on the type of object to be processed, an optimal corresponding category can be chosen. Presets are divided into fast, default and deep analysis. They represent the search for a progressively larger number of common points between each image. For the purpose of our study, the reconstruction was obtained through the high-definition option for a generic object.

Once the scattered point cloud has been processed, if the results show that the object has been entirely reproduced as a point cloud, then the processing of the dense point cloud, the creation of the mesh, and ultimately the textured mesh can be performed. In the first place the software creates a cloud of scattered points, which are obtained automatically by locating the point in three or more images and subsequently a dense cloud is generated, to which a densification filter is applied to obtain a better visualization of the cloud itself. From this cloud, the software creates the mesh, and a filter is applied to the mesh for topology optimization. This processing can take about 2 h. At the end, the software allows the mesh to be exposed in the.obj format. This format is compatible with the 3D graphics software Blender.

Image processing from CT scan and comparison with photogrammetry

In this study, a CT scan of the ventricles, employed for cardiac mapping visualization was used to obtain a comparison for accuracy of photogrammetric reconstruction. The CT scan was performed with 64 slices image acquisition and 360° rotations. After CT scan images were imported in the ECGi proprietary software and the selection of the region of interest for 3D reconstruction (ventricles), was performed manually, by proceeding to the segmentation of each component: ventricles, arteries and veins. The file processed was exported in .xml format and introduced into the UZBCIT software.

The UZBCIT software allows the simultaneous visualization of the ventricular volume and the mapping of the electrical activity of the heart (Figure 1). The software also enables the calculation of the area of specific sections in cm^2.

Once the photogrammetric model was created with Zephyr, it was imported into Blender 3.0 (Blender Foundation). In order to assess the level of accuracy of the Zephyr software’s photogrammetric reconstruction, the latter was compared to the 3D virtual model based on the CT scan of the UZBCIT software through the function Analyze > statistics > Area, present in Blender. Reference measurements are taken from the CT mesh, as it is considered the most accurate mesh (gold standard) (Covino et al., 1996). Comparisons were performed by calculating the area of the aorta and vena cava sections for both photogrammetric and CT-based models. The following measurements were defined: 1) U-aorta: area section of the aorta calculated on UZBCIT; 2) B-aorta: area section of the aorta calculated on Blender; 3) U-vena cava: area section of the vena cava calculated on UZBCIT; 4) B-vena cava: area section of the vena cava calculated on Blender.

These measurements have to be the same in the photogrammetry mesh. If the measurements of this mesh do not match those of the CT mesh, the mesh size can be increased or decreased through the “scale” tool by selecting the points of interest and increasing the size by dragging the mouse. Blender software can zoom in or out on the 3D mesh evenly throughout its volume. The 3D object is then enlarged according to each projection of its volume equally.

Statistical analysis

Statistical analysis was performed using JASP software (GNU Affero GPL). Mean differences between meshes created by photogrammetric method and CT were used to assess either overestimation or underestimation of each method. If this is significantly different from zero, it is indicative of an overestimation or underestimation of one technique over the other. The mean difference was evaluated through the use of the paired t-test or Wilcoxon’s test on ranks, in the case of non-normal distributions. To evaluate the trend of the distributions, a descriptive analysis and a normality test with Shapiro Wilk’s test were performed; in addition, the Q-Q plot was performed as an auxiliary test of normality confirmation. Once the normality of the distributions was investigated, the Wilcoxon signed-rank test was performed, and to conclude, a correlation test base on a Bayesian information criterion was also performed.

Results

Photogrammetric reconstruction of all 15 test subjects was successfully completed. The procedure took a total time of 2 h for each reconstruction. In detail, about 45 min were required for image acquisition, 50 min were required for preprocessing with Masquerade, and about 25 min were required for photogrammetric reconstruction from image loading to mesh export.

Descriptive analysis was performed. The values of the mean for the aorta data were 4.880 for U-aorta and 4.882 for B-aorta, while the mean values were 5.092 for U-vena cava and 5.070 for B-vena cava; with a Standard error of mean (SEM) of .557 for both aortas (U/B), .501 for U-vena cava and .514 for B-vena cava (Table 1). The analysis of the data showed a good correlation between the measurements acquired with photogrammetric reconstruction and CT images. Although the QQ plot (Figure 3) showed a distribution of data close to the diagonal, the test for normality showed a non-normal distribution (Table 2). Therefore, the Wilcoxon signed-rank test was performed, whose p-value was >.05 (Table 3), with a result of p = .348 for the aorta measurements and p = .638 for the vena cava measurements, respectively. This demonstrated a non-significant difference between photogrammetry and CT scan derived images. For further confirmation, a Pearson’s correlation test was also performed, with coefficient value proximal to one for both analysis; the graphs supported a strong correlation between the mesh reconstruction derived from CT and that derived from photogrammetry (Figure 4). The test showed for the section of the aorta a homogeneity of results. From the graph in Figure 5, it can be deduced that photogrammetry slightly overestimated the measurements compared with CT for the aorta; on the other hand, for the vena cava section, photogrammetry slightly underestimated the measurements compared with CT.

TABLE 1

TABLE 1. Descriptive statistics.

FIGURE 3

FIGURE 3. Q-Q Plots. Q-Q plot showing a distribution of data close to diagonal.

TABLE 2

TABLE 2. Test of normality (Shapiro-Wilk).

TABLE 3

TABLE 3. Wilcoxon Signed-rank test.

FIGURE 4

FIGURE 4. Pearsons’s correlation plot. Pearson’s correlation plot between the aorta and vena cava measurements from the two software programs. The data show high correlation.

FIGURE 5

FIGURE 5. Descriptive Plots. U aorta-B aorta U vena cava-B vena cava. Graphical results obtained from descriptive analysis. The graphs show a not significant difference between the measurements obtained between CT and photogrammetry.

Discussion

Automatic photogrammetry allows the processing of 3D imaging. In the medical field, its application is still limited (Mitchell, 1995). This study performed virtual reproduction of cardiac mapping for digital dimensional assessment. Zephyr Lite (3DFlow^©) software was used for 3D volume creation. Because of the use of this software, the realization of a three-dimensional mesh was possible, starting from jpg images. The accuracy of the reconstructions has been evaluated using 45 photographs collected concentrically with respect to the z-axis from different angles. The mesh generation process involves structure-from-motion (SfM) methods, which identifies common features of the images using the scale-invariant feature transform (SIFT) algorithm (Bruschweiler et al., 2003). This research compared the results of applying two three-dimensional acquisition techniques using patients with Brugada Syndrome who underwent to cardiac mapping with CardioInsight. (Cluitmans et al., 2018; Cheniti et al., 2019). The 15 reconstructions were feasible in a relatively short time. The purpose of this research was also to evaluate the potential of the photogrammetric technique to be employed in cardiology with several potential applications. In this paper CT scan was used as gold standard. Comparing CT with photogrammetry is important considering the limitations and advantages of both techniques (Markose et al., 2009; Donato et al., 2020). The accuracy of the photogrammetric technique was evaluated and the results obtained are very encouraging. Photogrammetric reconstruction measurements were very close to those obtained from measurements made on CT-based reconstructions. These results confirmed that the photogrammetric technique is useful for three-dimensional reconstruction of digital images.

Moreover, it is essential to consider the impact on results if the hereby presented procedure is not followed. If Masquerade is not used, since it is a reconstruction of a digital object, the Zephir software makes incomplete and erroneous reconstructions. On the other hand, the reconstruction of virtual objects has the advantage of not requiring any manipulation of the mesh once it has been created. The aforementioned manipulation, in classical cases, is due to the presence of the support in which the object is placed for image acquisition.

For current practical use it is important to select a myriad of polygons that compose the mesh, as it will be possible to manipulate the mesh with greater precision. In fact, the software develops a line between the mesh to be preserved and the area to be removed. This line will automatically follow the perimeter of the polygons that make up the mesh. The smaller the number of polygons, the more precise the line will be. By doing so, a more precise edge to the removed area can be defined. If the number of polygons is too low, the accuracy of the line will be reduced (Donato et al., 2020), thus causing possible errors. It is essential to control the number of polygons that make up the mesh also in order to obtain a more accurate result during the measurements. Therefore, the experience of the operator is of fundamental importance to obtain accurate and reliable results.

CT will always be irreplaceable for the visualization of the structures within the area of interest; however, photogrammetry represents an excellent tool for the study and analysis of the surface also in the case of reproduction of digital objects. BrS is a primary arrhythmia syndrome with no structural changes. However, electrical changes (conduction delay) occur and they can be investigated with ECGi (Pannone et al., 2022b). One disadvantage of standard imaging techniques as CT scan or photogrammetry is that neither CT scan alone, nor photogrammetry alone can evaluate the electrical abnormalities in BrS but ECGi can. Photogrammetry cannot image the electrical changes in BrS on its own but it has an important clinical role in extracting the relevant geometrical and electrical information from an ECGi map, making these data available for a 3D surgical guide building. Based on the feedback from an experienced cardiologist (C.d.A.) and an experienced cardiothoracic surgeon (M.L.M.), photogrammetry was considered feasible and of major clinical impact to extract relevant data from ECGi maps.

Conclusion

It is possible to reproduce digital objects by photogrammetry if the process described in this study is performed. The reconstruction of the ventricles from CT scan was very close to the values of the respective photogrammetric reconstruction. Future studies should be performed, with a larger number of samples and on different pathologies. In particular, assessing the reproducibility of the described approach for a 3D model in a large cohort is eagerly awaited. Future research is needed to test mechanical properties and further assess biocompatibility of materials for a 3D surgical guide. The response of the material to ablation should be also evaluated.

Data availability statement

The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author.

Author contributions

Conception and design of the work: GT, LP, BI, and CdeA. Substantial contributions to the acquisition of data for the work: LP, CM, GT, and RR Substantial contributions to the analysis of data for the work: LP, CM, EB, IC, and MC Substantial contributions to the interpretation of data for the work: MW, RR, ML, AG, GC, BI, and CdeA. Drafting the work: GT and LP Revising the draft of the work critically for important intellectual content: MW, RR, ML, AG, GC, BI, and CdeA. Final approval of the version to be published: GT, LP, CM, EB, IC, MC, MW, RR, ML, AG, GC, BI, and CdeA. Agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved: GT, LP, CM, EB, IC, MC, MW, RR, ML, AG, GC, BI, and CdeA.

Conflict of interest

ML is consultant for Atricure. GC received compensation for teaching purposes and proctoring from Medtronic, Abbott, Biotronik, Boston Scientific, Acutus Medical. CdeA receives research grants on behalf of the center from Biotronik, Medtronic, Abbott, LivaNova, Boston Scientific, AtriCure, Philips, and Acutus; CdeA received compensation for teaching purposes and proctoring from Medtronic, Abbott, Biotronik, Livanova, Boston Scientific, Atricure, Acutus Medical Daiichi Sankyo.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Bruschweiler, W., Braun, M., Dirnhofer, R., and Thali, M. J. (2003). Analysis of patterned injuries and injury-causing instruments with forensic 3D/CAD supported photogrammetry (FPHG): An instruction manual for the documentation process. Forensic Sci. Int. 132 (2), 130–138. doi:10.1016/s0379-0738(03)00006-9

PubMed Abstract | CrossRef Full Text | Google Scholar

Cheniti, G., Puyo, S., Martin, C. A., Frontera, A., Vlachos, K., Takigawa, M., et al. (2019). Noninvasive mapping and electrocardiographic imaging in atrial and ventricular arrhythmias (CardioInsight). Card. Electrophysiol. Clin. 11, 459–471. doi:10.1016/j.ccep.2019.05.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Cluitmans, M., Brooks, D. H., MacLeod, R., Dossel, O., Guillem, M. S., van Dam, P. M., et al. (2018). Validation and opportunities of electrocardiographic imaging: From technical achievements to clinical applications. Front. Physiol. 9, 1305. doi:10.3389/fphys.2018.01305

PubMed Abstract | CrossRef Full Text | Google Scholar

Covino, S. W., Mitnic, R. J., Shprintzen, R. J., and Cisneros, G. J. (1996). The accuracy of measurements of three dimensional computed tomography reconstructions. J. OralMaxillofacSurg 54 (8), 982–990. doi:10.1016/s0278-2391(96)90397-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Donato, L., Cecchi, R., Goldoni, M., and Ubelaker, D. H. (2020). Photogrammetry vs CT scan: Evaluation of accuracy of a low-cost three-dimensional acquisition method for forensic facial approximation. American Academy of Forensic Sciences, 1–6.

Google Scholar

Markose, E., Vikraman, B., and Veerabahu, M. (2009). Three dimensional CT reconstruction: A comparison between 2D, 3D CT and original anatomical structures. J. Maxillofac. Oral Surg. 8, 8–12. doi:10.1007/s12663-009-0002-2

PubMed Abstract | CrossRef Full Text | Google Scholar

Mitchell, H. L. (1995). Applications of digital photogrammetry to medical investigations. ISPRS J. Photogrammetry Remote Sens. 50, 27–36. doi:10.1016/0924-2716(95)91288-U

CrossRef Full Text | Google Scholar

Pannone, L., Monaco, C., Sorgente, A., Vergara, P., Calburean, P. A., Gauthey, A., et al. (2022a). High-density epicardial mapping in Brugada syndrome: Depolarization and repolarization abnormalities. Heart Rhythm 19 (3), 397–404. doi:10.1016/j.hrthm.2021.09.032

PubMed Abstract | CrossRef Full Text | Google Scholar

Pannone, L., Monaco, C., Sorgente, A., Vergara, P., Calburean, P., Gauthey, A., et al. (2022b). Ajmaline-induced abnormalities in Brugada syndrome: evaluation with ECGimaging. J. Am. Heart Assoc. 11 (2), e024001. doi:10.1161/JAHA.121.024001

PubMed Abstract | CrossRef Full Text | Google Scholar

Pannone, L., Monaco, C., Sorgente, A., Vergara, P., Gauthey, A., Calburean, P. A., et al. (2022c). SCN5A mutation in Brugada syndrome is associated with substrate severity detected by electrocardiographic imaging and high-density electroanatomic mapping. Heart Rhythm 19 (6), 945–951. doi:10.1016/j.hrthm.2022.01.034

PubMed Abstract | CrossRef Full Text | Google Scholar

Pappone, C., Brugada, J., Vicedomini, G., Ciconte, G., Manguso, F., Saviano, M., et al. (2017). Electrical substrate elimination in 135 consecutive patients with Brugada syndrome. CircArrhythmElectrophysiol 10, e005053. doi:10.1161/circep.117.005053

PubMed Abstract | CrossRef Full Text | Google Scholar

Rudy, Y., and Messinger-Rapport, B. J. (1988). The inverse problem in electrocardiography: Solutions in terms of epicardial potentials. CritRevBiomed Eng. 16 (3), 215–268.

PubMed Abstract | Google Scholar

Salghetti, F., de Asmundis, C., Sieira, J., Coutiño, H. E., Abugattas, J. P., Varnavas, V., et al. (2019). Hybrid thoracoscopic epicardial ablation of right ventricular outflow tract in patients with Brugada syndrome. Heart 16, 879–887. doi:10.1016/j.hrthm.2018.12.026

PubMed Abstract | CrossRef Full Text | Google Scholar

Zhang, P., Tung, R., Zhang, Z., Sheng, X., Liu, Q., Jiang, R., et al. (2016). Characterization of the epicardial substrate for catheter ablation of Brugada syndrome. Heart Rhythm 13, 2151–2158. doi:10.1016/j.hrthm.2016.07.025

PubMed Abstract | CrossRef Full Text | Google Scholar