Real-Time 2D MR Cine From Beam Eye’s View With Tumor-Volume Projection to Ensure Beam-to-Tumor Conformality for MR-Guided Radiotherapy of Lung Cancer

Purpose: To minimize computation latency using a predictive strategy to retrieve and project tumor volume onto 2D MR beam eye’s view (BEV) cine from time-resolved four-dimensional magnetic resonance imaging (TR-4DMRI) libraries (inhalation/exhalation) for personalized MR-guided intensity-modulated radiotherapy (IMRT) or volumetric-modulated arc therapy (VMAT).

Methods: Two time-series forecasting algorithms, autoregressive (AR) modeling and deep-learning-based long short-term memory (LSTM), were applied to predict the diaphragm position in the next 2D BEV cine to identify a motion-matched and hysteresis-accounted image to retrieve the tumor volume from the inhalation/exhalation TR-4DMRI libraries. Three 40-s TR-4DMRI (2 Hz, 3 × 80 images) per patient of eight lung cancer patients were used to create patient-specific inhalation/exhalation 4DMRI libraries, extract diaphragmatic waveforms, and interpolate them to f = 4 and 8 Hz to match 2D cine frame rates. Along a (40•f)-timepoint waveform, 30•f training timepoints were moved forward to produce 3×(10•f-1) predictions. The accuracy of position prediction was assessed against the waveform ground truth. The accuracy of tumor volume projections was evaluated using the center-of-mass difference (∆COM) and Dice similarity index against the TR-4DMRI ground truth for both IMRT (six beam angles, 30° interval) and VMAT (240/480 beam angles, 1.5°/0.75° interval, at 4/8 Hz, respectively).

Results: The accuracy of the first-timepoint prediction is 0.36 ± 0.10 mm (AR) and 0.62 ± 0.21 mm (LSTM) at 4 Hz and 0.06 ± 0.02 mm (AR) and 0.18 ± 0.06 mm (LSTM) at 8 Hz. A 10%–20% random error in prediction-library matching increases the overall uncertainty slightly. For both IMRT and VMAT, the accuracy of projected tumor volume contours on 2D BEV cine is ∆COM = 0.39 ± 0.13 mm and DICE = 0.97 ± 0.02 at 4 Hz and ∆COM = 0.10 ± 0.04 mm and DICE = 1.00 ± 0.00 at 8Hz.

Conclusion: This study demonstrates the feasibility of accurately predicting respiratory motion during 2D BEV cine imaging, identifying a motion-matched and hysteresis-accounted tumor volume, and projecting tumor volume contour on 2D BEV cine for real-time assessment of beam-to-tumor conformality, promising for optimal personalized MR-guided radiotherapy.

Introduction

One of the major advantages of magnetic resonance imaging integrated linear accelerator (MR-Linac) is to provide MR-guided radiotherapy (MRgRT), including real-time tumor motion management, making respiratory gating accurate and tumor tracking possible, especially for hypo-fractional stereotactic body radiotherapy (SBRT) (1–3). As MRgRT provides patient-specific imaging in real-time during treatment, adapting to inter-fractional and intra-fractional patient anatomic variations, it offers the best-personalized radiotherapy. Clinically, improved treatment outcomes have been reported showing the benefit of sparing critical organs at risk (OARs) so that the tumor lethal dose can be prescribed and delivered to a mobile target, including lung, liver, and pancreatic cancer (3, 4). Compared with conventional image-guided radiotherapy (IGRT), MRgRT offers many advantages, including real-time imaging with high soft-tissue contrast without ionization radiation. So far, the intensity-modulated radiotherapy (IMRT) technique is available in the MR-Linac and the volumetrically modulated arc therapy (VMAT) technique should be possible in the future.

Currently, dynamic 2D cine imaging in the sagittal and coronal views can be employed for MR-guided IMRT to monitor respiratory-induced tumor motion in real time covering the major motions in the superior–inferior (SI) and anterior–posterior (AP) directions. Although the two cine views infer a 3D tumor motion, they are partial and indirect views of a volumetric tumor related to the radiation beam. Moreover, potential through-plane tumor motion may interfere with motion interpretation. The more effective, optimal view for assessing the beam-to-tumor conformality should be the beam eye’s view (BEV), which is how the radiation beam sees the mobile tumor and only needs one cine scan (5–7). Previously, a 2D BEV cine technique with tumor volume projection has been reported feasible for better MRgRT (8). For IMRT treatment, adequate accuracy and performance have been achieved to identify and project a volumetric tumor onto the BEV by 2D–3D matching between the 2D tumor image on the BEV cine images and a time-resolved (TR) 4DMRI library containing volumetric images from multiple breathing cycles (9–12). For MR-guided VMAT, which may be available in the future, real-time communication and computation are required for 2D cine imaging with a rotating BEV and projecting tumor volume with minimal latency.

To overcome system latency, predictive strategies have been applied to provide a just-in-time prediction of tumor motion in the next 30–1,000 ms, using conventional Linac for tumor tracking, including adjusting the radiation beam, a multi-leaf collimator (MLC), or couch position to keep up with a tumor motion (13–15). Various predictive methods have been evaluated, including time-series-based (16), model-based (17), regression-based (18), and machine-learning-based (19, 20) prediction methods. Short-term motion prediction is an effective and efficient approach to overcome the system latency and reduce the frequency of x-ray imaging.

Additionally, respiratory motion hysteresis is a commonly occurring phenomenon in patients, resulting in variations in tumor motion trajectory, orientation, and shape between inhalation and exhalation (21, 22). Therefore, even at the same tumor displacement in the superior–inferior (SI) direction, the tumor shape, orientation, and anterior–posterior (AP) and left–right (RL) positions may vary due to the motion hysteresis (15, 21–24). Therefore, without differentiating the inhalation and exhalation processes, it may add uncertainties in retrieving the motion-matched tumor volume for BEV projection.

In this simulation study, we aimed to minimize the latency in matching a tumor volume by predicting the respiratory motion to identify and project tumor volume in parallel with the next 2D BEV cine acquisition. In addition, to minimize the motion hysteresis effect, two TR-4DMRI image libraries of exhalation and inhalation were created by grouping volumetric images based on their moving directions. Two predictive algorithms, a conventional autoregression (AR) modeling and a deep-learning-based long short-term memory (LSTM) neural network, were applied and evaluated with the known waveforms. The accuracy of tumor volume projection on the 2D BEV cine images was evaluated against the ground truth embedded in the TR-4DMRI datasets. This improved 2D BEV cine technique was evaluated for both MR-guided IMRT and VMAT treatments.

Methods

In this study, a predictive strategy was proposed and evaluated to identify and project tumor volume onto 2D BEV cine images in real time with minimized system latency and included respiratory motion hysteresis. The workflow of the strategy is shown in Figure 1.

FIGURE 1

Figure 1 The workflow of the predictive strategy to predict, identify, and project tumor volume onto 2D BEV cine, followed by the verification of projected tumor volume using the center of mass and Dice similarity index against the ground truth for MR-guided IMRT and VMAT. In future clinical applications, the cine waveform can be utilized as well (gray box).

Three Time-Resolved 4DMRI Image Datasets, Waveform Extraction, and Interpolation

Eight lung cancer patients were recruited to participate in an IRB-approved protocol study using TR-4DMRI for respirator-induced tumor motion simulation and assessment in a 3T MRI scanner (Ingenia, Philips Healthcare, Amsterdam, Netherlands). Eight patients were scanned at simulation with 3D cine in free breathing (FB) for 40 (s) at f = 2 Hz three times within one imaging session. A T1-weighted, multi-shot, turbo field echo pulse sequence was used with SENSE acceleration (6.0) and partial Fourier approximation (0.8), so a total of 3 × 80 volumetric 3D cine images with a voxel size of 5 × 5 × 5 mm³ were acquired. Using the same scan protocol with less acceleration (3.75), three 3D cines (2 × 2 × 2 mm³) in breath hold (BH) at an arbitrary stage were acquired within 20 s. The TR-4DMRI images were reconstructed based on the super-resolution approach that has been developed via deformable image registration between the low-resolution FB and high-resolution BH images. Detailed 4DMRI scanning and reconstruction methods and conditions can be found in the previous publications (9–11).

The diaphragmatic motion waveforms were extracted from TR-4DMRI using an in-house program in MATLAB (MathWorks, MA). A navigator box (3 × 3 × 6 cm³) drawn on the right diaphragmatic dome was used to calculate the average voxel intensity at the same SI positions, and the point with the largest gradient was determined as the diaphragm position. Over the 80 images from a 40-s scan, diaphragmatic motion trajectory in a scan series was used as a motion waveform in the superior–inferior (SI) direction. Each of the three waveforms at the frequency of f = 2 Hz was interpolated using the b-spline function to f = 4 Hz and f = 8 Hz to match the possible scanning rates of clinical 2D cine frame rate, containing a total of 40•f timepoints.

Just-in-Time Prediction to Overcome System Latency in the 2D BEV Cine Strategy

Two time-series forecasting algorithms were applied to predict diaphragm motion based on the motion waveforms: (1) a classical autoregressive (AR) modeling algorithm implemented in the MATLAB Econometric Toolbox™ that uses past values as inputs to a regression algorithm to predict future values and (2) a deep-learning long short-term memory (LSTM) recurrent neural network algorithm in the MATLAB Deep Learning Toolbox™ that processes input data by looping over the time steps and updating the network state containing information over previous time steps. Various parameters were tested for the best prediction accuracy and performance in the two predictive algorithms, and the optimal settings include using 30-s training data, 10 AR polynomial degrees, and 20 hidden layers in the neural network using the Adam (Adaptive Moment Estimation) optimizer with the maximum of 150 epochs.

In each of the interpolated 40-s waveforms, a 30-s waveform section with 30•f timepoints was applied as training data to predict the diaphragm position at the next time point in 125 ms at f = 8 Hz and 250 ms at f = 4 Hz. After a prediction, the training dataset was moved one timepoint forward by adding one new timepoint and removing one old timepoint. The remaining 10-s waveform served as the ground truth to assess the prediction accuracy of n = 10•f-1 predictions per motion waveform. For each patient, a total of 3 × (10•f-1) predictions were made and evaluated to assess both patient-specific and population-based accuracy of motion prediction.

Accounting Tumor Motion Hysteresis by Identifying Tumor With Predicted Motion Direction

The 3 × 80 TR-4DMRI images per patient were categorized as in the inhalation and exhalation processes depending on the diaphragm moving direction from the previous timepoint. Therefore, two TR-4DMRI image libraries of inhalation and exhalation were built with roughly 120 images each. The estimated displacement interval on the diaphragm was 30 mm/120 = 0.25 mm, and the exact interval may vary, depending on the motion range, speed, and number of images. Note that the motion interval can be reduced, as more 40-s TR-4DMRI series could be acquired and added from simulation.

When the next diaphragm position is predicted, the moving direction was first used to select the TR-4DMRI library (inhalation or exhalation), and the predicted amplitude was then applied to identify a matched diaphragm, and therefore the corresponding tumor volume in the library. As exact diaphragm matching may not be found in a library, a small uncertainty should be added on top of the prediction uncertainty for the diaphragm position displacement (∆D), namely,

$\begin{array}{ll}\Delta D^{matched}=\Delta D^{prediction}+\Delta D^{matching}& (1)\end{array}$

Once a volumetric image was identified with a matched diaphragm from a library, either for inhalation or for exhalation, the tumor volume projection was evaluated as the difference in the center of mass (∆COM) and the DICE similarity index, compared with the image ground truth embedded in the simulation dataset for accuracy assessment.

Accuracy of Tumor Volume Projection on 2D BEV Cine for MR-Guided IMRT and VMAT

For IMRT treatment, six fixed beam angles (0°–150°) with 30° intervals were used to assess the ∆COM and DICE of the projected tumor volumes between the predicted and ground truth as a function of time during treatment delivery. Mimicking a 6-beam IMRT plan, all TR-4DMRI images in the patient-specific libraries were segmented for tumor volume using an automated MATLAB program on all cuts parallel to the BEV with a 2-mm interval and ready to be projected to the 2D BEV images. The union of all projected contours was used as the final tumor volume projection.

For VMAT treatment, the full rotation was divided into 240 sections with 1.5° intervals in 4-Hz simulation and used to assess the ∆COM and DICE between the tumor volume projections from the predicted and ground truth. In the 8-Hz simulation, 480 sections per gantry rotation with 0.75° intervals were used. The results were analyzed as a function of gantry angle, assuming that the gantry rotates at a constant speed. It should be noted that clinically the gantry position is known within the MR-integrated Linac system and the beam angle will be updated. In the VMAT cases, much more beam angles were prepared with pre-segmented tumor volume ready to be used.

For both IMRT and VMAT cases, the results of tumor volume projection resulting from the predictions at 125 and 250 ms were evaluated using the ∆COM and Dice index based on 8- and 4-Hz interpolated waveforms, respectively. The two-tailed Student’s t test was used for all comparison, and a p-value of less than 0.05 was considered statistically significant different.

Results

Prediction Accuracy of the AR Modeling and LSTM Deep-Learning Network

Figure 2 illustrates the prediction and matching errors based on the 4- and 8-Hz waveforms. Only the first time point will be used as the diaphragm position in the next 2D BEV cine image for accuracy evaluation, while prediction errors for the first 10 time points are provided, which tend to level off around 1,000 ms at both frequencies. The prediction accuracy is higher at 8 Hz than at 4 Hz, primarily due to more training points being used. Furthermore, the AR method predicts more accurately (0.4 mm) and takes less time (0.4 s) than the LSTM prediction (0.6 mm, 1.5 s) at 4 Hz under the current computing conditions, as shown in Table 1. The same trend is observed using the 8-Hz waveform, as shown in Table 2.

FIGURE 2

Figure 2 Prediction accuracy of the AR (autoregressive) modeling and the LSTM (long short-term memory) neural network as a function of the time points ahead of training datasets (the 30s of the waveform) using motion waveforms at 4 Hz (A: 10 predicted timepoints in 2,500 ms) and at 8 Hz (B: 10 predicted timepoints in 1,250 ms). The mean differences increase slightly due to the matching errors.

TABLE 1

Table 1 Prediction accuracy with matching uncertainties based on 4Hz motion waveforms.

TABLE 2

Table 2 Prediction accuracy with matching uncertainties based on 8-Hz motion waveforms.

Tumor Motion Hysteresis and Compensation Using Exhalation and Inhalation Libraries

The motion of the diaphragm spends slightly more time (more images) in exhalation than inhalation phases, as shown in Table 1. Figures 3, 4 show a couple of examples of motion hysteresis of the tumors in the coronal view (BEV = 0°) and are compensated for by identifying the matched tumor volume using the appropriate inhalation or exhalation TR-4DMRI library.

FIGURE 3

Figure 3 Illustration of motion hysteresis of a posterior peripheral tumor (patient 2, with 20 breathing cycles in 40 s) during mid-inhalation (A) and mid-exhalation (B) with a similar diaphragm displacement at the zero-gantry angle (BEV = 0°). The isocenter (red cross) position is shown inside the in-plane tumor BEV contour (orange) and the projected volumetric tumor contour (white). Both the in-plane and projected tumor contour shapes and the centers of mass (COMs) are different between inhalation and exhalation. By selectively using either the inhalation or exhalation TR-4DMRI library based on the motion direction, the respiratory hysteresis effect is compensated.

FIGURE 4

Figure 4 Illustration of 2D BEV (beam angle = 0°) difference of in-plane tumor contours (orange) between mid-inhalation (A) and mid-exhalation (B) (patient 7, with 17 breathing cycles in 40 s), while projected tumor volume contours (white) are similar in the multi-cycle TR-4DMRI images for this patient. The diaphragm positions are similar in all cases. Due to respiratory hysteresis, including the AP motion, the in-plane tumor contours (orange) are distinctively different between inhalation and exhalation. In contrast, tumor volume projections may not be affected by a through-plane (AP) motion, presenting a stable projected tumor contour. This predictive strategy distinguishes hysteresis-caused tumor volume differences via selectively using either the inhalation or exhalation TR-4DMRI library based on the respiratory direction, while the previous method distinguishes them via searching for a match with the highest Dice similarity index (8).

Verification of 2D BEV Cine With Tumor Volume Projections for IMRT and VMAT Treatments

Table 3 shows the average and standard deviation of COM difference (∆COM) and Dice similarity of projected tumor volume between the identified and ground truth for IMRT with six fixed beam angles and for VMAT with rotating beam angles (1.5° interval at 4 Hz and 0.75° interval at 8 Hz). Figure 5 shows the Dice similarity index in VMAT as a function of beam angle for predictions using the 4- and 8-Hz waveform data.

TABLE 3

Table 3 The accuracy of the center of mass (∆COM) and shape (Dice similarity index) of the projected tumor volume onto the 2D BEV cine images.

FIGURE 5

Figure 5 The Dice similarity index of the projected tumor contours on the 2D BEV cine images between the predicted and the ground truth of the tumor volume as a function of beam angle (0°–360°). A constant gantry rotation is assumed for the two plots (Dice value display: 0.4–1.0). At both frequencies, the Dice index of patient 3 (yellow) shows the most variations among the eight patients. From 4 Hz (A) to 8 Hz (B) the average Dice index increases from 0.97 ± 0.02 to 1.00 ± 0.00 (Table 3) as the prediction accuracy increases, as shown in Tables 1, 2.

Discussion

Based on the previous 2D BEV cine approach for MR-guided IMRT (8), we have demonstrated the following three improvements of the approach in this simulation study: (1) the computation latency has been minimized by applying the prediction method in parallel with the 2D cine scan to identify a motion-matched tumor volume without searching from scratch, (2) respiratory-induced tumor motion hysteresis has been explicitely compensated for using the predicted motion to find a match in either inhalation or exhalation libraries depending on the motion direction, and (3) the feasibility of the 2D BEV cine approach to MR-guided VMAT has been tested. In the following, the advantages and limitations of the new 2D BEV cine approach will be discussed in depth.

Minimizing Computation Latency Using Just-in-Time Tumor Motion Prediction

The so-called just-in-time tumor motion prediction has been studied to combat the system latency in respiratory-gated or tumor-tracking radiotherapy, as it can predict several tens or hundreds of milliseconds (ms) ahead of time, allowing the system to start to act before an event occurs. In this study, the predictive strategy is applied to remove the computation latency in the 2D–3D library matching (8) for tumor volume projection. As the prediction can be started in parallel with the next BEV 2D cine acquisition, the predicted tumor position in the next cine frame should be available and ready to project tumor volume contour onto the 2D BEV cine image. Therefore, the computation latency can be reduced or eliminated if the prediction can be completed within 125 or 250 ms for 8- or 4-Hz cine acquisition, respectively.

Interestingly, the prediction accuracy and speed of the AR modeling method outperform the deep-learning-based LSTM neural network method, in both accuracy and performance. As the learning process is progressively reoccurring along with the moving training dataset, the LSTM has to relearn every time in the current algorithm. Therefore, the AR results will be used as the focus of the discussion. As shown in Tables 1, 2, the current AR computation times for prediction are 400 and 500 ms for 4- and 8-Hz training datasets, respectively. The computation time must be reduced to within 125–250 ms to remove the computation latency, and performance enhancement is achievable by optimizing the prediction code, using a more powerful computer, and/or employing parallel computing techniques, including the graphics processing unit (GPU) technique.

On average, the prediction accuracy is higher for the time point that is closer to the training dataset, as illustrated by the predictions of the next 125 ms (8 Hz) and 250 ms (4 Hz), as shown in Tables 1, 2. The higher the 2D cine frame rate, the higher the accuracy of prediction would be, and therefore the more accurate the tumor volume projection. Therefore, it is recommended to apply the highest cine frame rate, as long as the cine image quality is sufficient for tumor visualization. In this study, as the b-spline interpolation was applied to the waveform extracted from TR-4DMRI (2Hz), the waveforms are “smoother” than the actual, which contains random noise. However, even if a 2% random error was added to the interpolated motion waveform, the prediction error should still be at the sub-mm level. A random library matching error was present due to the limited motion interval in the TR-4DMRI library, and on average, the prediction results remain roughly the same (Figure 2).

Compensating for the Tumor Motion Hysteresis by Separating Inhalation and Exhalation

It is well known that respiratory-induced tumor motion is direction-dependent, meaning the path for inhalation is different from that of exhalation, namely, motion hysteresis (21, 24). Therefore, when identifying an image with a matching diaphragm or tumor displacement in the SI direction, the motion direction should be considered, in addition to amplitude. In this study, we split the TR-4DMRI library into inhalation and exhalation libraries and applied motion direction first to select the correct library and then found a match with the motion amplitude. Therefore, the hysteresis effect will be accounted for.

Using the current three-series TR-4DMRI, the number of images seems sufficient to have a small-enough motion interval in both inhalation and exhalation libraries, as the uncertainty from slight mismatching does not add too much error on average (Figure 2). However, for individual prediction, the finer interval should be helpful to identify a precise diaphragm or tumor position. The library sizes can be easily increased by acquiring more 40-s TR-4DMRI series as additional acquisitions. The 40-s limit is due to the MR scanner memory capacity. Therefore, the number of MR images in the exhalation and inhalation libraries can be increased substantially within a couple of minutes of TR-4DMIR acquisition.

The motion hysteresis has some variation, meaning the inhalation and exhalation path may not be very reproducible, as part of breathing irregularities. Therefore, there is an uncertainty in identifying a tumor volume from the libraries, either through previous 2D–3D library matching (8) or through current inhalation/exhalation library assignment.

Advantages and Limitations of Using BEV 2D Cine for MRgRT Treatments

It is a commonly accepted concept in image-guided radiotherapy that BEV imaging is the most useful guidance because it directly verifies if the radiation beam is targeting a mobile tumor and if beam-to-tumor conformality is acceptable by comparing the projected tumor volume in the planning BEV image and the verification BEV image. In a conventional Linac, an MV electronic portal imaging device (EPID) is used to take the BEV portal image, while in an MR-integrated Linac, a 2D BEV cine image can be acquired by changing the MR imaging orientation to be perpendicular to the radiation beam. Aside from other differences, the 2D BEV cine image provides a slice image, while the EPID produces a radiographic projection image. Therefore, the through-plane tumor motion will affect the MR cine image, and projecting the tumor volume onto the BEV cine becomes necessary. To retrieve tumor volumetric information, we proposed to identify the matched tumor volume in TR-4DMRI libraries via either previous 2D–3D library matching (8) or the predictive strategy to identify the motion-matched and hysteresis-compensated tumor volume for volume projection onto the 2D BEV cine image. Although both approaches work, the current approach has the advantage of no computational latency.

In this study, verification of the predicted tumor volume projection against the ground truth illustrates the accuracy of the improved 2D BEV cine strategy. A sub-mm accuracy in prediction and tumor COM has been achieved, while the contour shape similarity of the projected tumor volume is as high as >0.95 against the ground truth. Therefore, the 2D BEV cine with projected tumor volume is sufficiently accurate and reliable and can be viewed in real time as soon as the 2D BEV cine is acquired without computational latency. The current predictive code needs to be optimized on its performance to reduce the computation time from 400 ms to within 250 ms for 4 Hz, including using parallel computing techniques, such as GPU technology.

In this predictive strategy, a high correlation between the diaphragm and tumor motions was assumed, which is often true for a lung or liver tumor that is located near the diaphragm (25–27). If a tumor is away from the diaphragm, its motion amplitude should be only a fraction of that of the diaphragm. Therefore, the tumor volume can be found with an acceptable tolerance when the diaphragm position has a precise match. As the diaphragm is a large object, uncertainties in its position determination are present (28). The sub-mm COM accuracy results have validated the assumption and method of segmenting the diaphragm used in this study. Clinically, if a tumor volume is sufficiently large, the tumor motion trajectory waveform can be directly measured using the same image processing tool on TR-4DMRI images. Clinically, the tumor trajectory from the 2D BEV cine can be also used as the motion of the day (Figure 1). Therefore, the error from the imperfect diaphragm–tumor correlation is eliminated, and more accurate identification and projection of tumor volume can be achieved.

Breathing irregularities have been observed and investigated during radiotherapy treatment in comparison with the motion assessment at the simulation (29, 30). Therefore, the TR-4DMRI library may need more images under some “abnormal” breathing conditions, including slightly deeper breathing. Therefore, the TR-4DMRI library represents more possible variations, ensuring the finding of a motion-matched and hysteresis-compensated tumor volume for projection onto the 2D BEV cine image.

Currently, the 2D BEV cine approach with projected tumor volume has only been tested for motion monitoring of peripheral lung cancer; it remains a challenge for centrally located lung tumor, liver, and pancreatic cancer, for which the image contrast between tumor and surrounding normal tissue may not be as clear, making the automatic tumor delineation difficult, even for MR imaging with high soft-tissue contrast. Therefore, further investigation is needed to assess the feasibility of other disease sites.

Conclusion

This study has demonstrated an improved 2D BEV cine approach for MR-guided IMRT with minimal computation latency, ability to compensating for respiratory hysteresis, and the feasibility for MR-guided VMAT. The predictive method can achieve sub-mm accuracy to determine the diaphragm position, tumor position, and tumor projection for the next frame of the 2D BEV cine image. The AR algorithm outperforms the LSTM algorithm in the next-frame motion prediction at both 4- and 8-Hz 2D BEV cine frame rates. The potential respiratory hysteresis effect on tumor shape between inhalation and exhalation is accounted for by checking motion direction and using corresponding TR-4DMRI libraries. This approach allows real-time assessment of beam-to-tumor conformality for respiratory gating or tracking during IMRT or VMAT treatments. With further clinical testing, this 2D BEV cine approach has strong potential to serve as optimal personalized imaging guidance in current MR-guided IMRT or future VMAT treatments.

Data Availability Statement

The raw data supporting the conclusions of this article can be made available by the authors upon request without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved by the MSK Institutional Review Board (IRB). The patients/participants provided their written informed consent to participate in this study.

Author Contributions

Both authors have made substantial contributions to this work. XN conducted most of the coding and computation. GL proposed the idea, participated in the data acquisition and analysis, and wrote the paper. All authors contributed to the article and approved the submitted version.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

This research is in part supported by the MSK Cancer Center Support Grant/Core Grant (P30 CA008748). We are grateful to Dr. Andreas Rimner for patient consenting, MR therapists for patient scanning, and all patients who participated in the IRB-approved protocol study.

References

1. Schrenk O, Spindeldreier CK, Burigo LN, Hoerner-Rieber J, Pfaffenberger A. Effects of Magnetic Field Orientation and Strength on the Treatment Planning of Nonsmall Cell Lung Cancer. Med Phys (2017) 44(12):6621–31. doi: 10.1002/mp.12631

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Chin S, Eccles CL, McWilliam A, Chuter R, Walker E, Whitehurst P, et al. Magnetic Resonance-Guided Radiation Therapy: A Review. J Med Imaging Radiat Oncol (2020) 64(1):163–77. doi: 10.1111/1754-9485.12968

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Corradini S, Alongi F, Andratschke N, Belka C, Boldrini L, Cellini F, et al. MR-Guidance in Clinical Reality: Current Treatment Challenges and Future Perspectives. Radiat Oncol (2019) 14(1):92. doi: 10.1186/s13014-019-1308-y

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Das IJ, McGee KP, Tyagi N, Wang H. Role and Future of MRI in Radiation Oncology. Br J Radiology (2019) 92(1094):20180505. doi: 10.1259/bjr.20180505

CrossRef Full Text | Google Scholar

5. Mao W, Hsu A, Riaz N, Lee L, Wiersma R, Luxton G, et al. Image-Guided Radiotherapy in Near Real Time With Intensity-Modulated Radiotherapy Megavoltage Treatment Beam Imaging. Int J Radiat Oncol Biol Phys (2009) 75(2):603–10. doi: 10.1016/j.ijrobp.2009.04.068

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Li G, Cohen P, Xie H, Low D, Li D, Rimner A. A Novel Four-Dimensional Radiotherapy Planning Strategy From a Tumor-Tracking Beam's Eye View. Phys Med Biol (2012) 57(22):7579–98. doi: 10.1088/0031-9155/57/22/7579

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Inoue M, Shiomi H, Iwata H, Taguchi J, Okawa K, Kikuchi C, et al. Development of System Using Beam's Eye View Images to Measure Respiratory Motion Tracking Errors in Image-Guided Robotic Radiosurgery System. J Appl Clin Med Phys (2015) 16(1):5049. doi: 10.1120/jacmp.v16i1.5049

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Nie X, Rimner A, Li G. Feasibility of MR-Guided Radiotherapy Using Beam-Eye-View 2D-Cine With Tumor-Volume Projection. Phys Med Biol (2021) 66(4):045020. doi: 10.1088/1361-6560/abd66a

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Li G, Wei J, Kadbi M, Moody J, Sun A, Zhang S, et al. Novel Super-Resolution Approach to Time-Resolved Volumetric 4-Dimensional Magnetic Resonance Imaging With High Spatiotemporal Resolution for Multi-Breathing Cycle Motion Assessment. Int J Radiat Oncol Biol Phys (2017) 98(2):454–62. doi: 10.1016/j.ijrobp.2017.02.016

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Li G, Sun A, Nie X, Moody J, Huang K, Zhang S, et al. Introduction of a Pseudo Demons Force to Enhance Deformation Range for Robust Reconstruction of Super-Resolution Time-Resolved 4DMRI. Med Phys (2018) 45(11):5197–207. doi: 10.1002/mp.13179

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Nie X, Huang K, Deasy J, Rimner A, Li G. Enhanced Super-Resolution Reconstruction of T1w Time-Resolved 4DMRI in Low-Contrast Tissue Using 2-Step Hybrid Deformable Image Registration. J Appl Clin Med Phys (2020) 21(10):25–39. doi: 10.1002/acm2.12988

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Nie X, Saleh Z, Kadbi M, Zakian K, Deasy J, Rimner A, et al. A Super-Resolution Framework for the Reconstruction of T2-Weighted (T2w) Time-Resolved (TR) 4DMRI Using T1w TR-4DMRI as the Guidance. Med Phys (2020) 47(7):3091–102. doi: 10.1002/mp.14136

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Ruan D, Fessler JA, Balter JM. Real-Time Prediction of Respiratory Motion Based on Local Regression Methods. Phys Med Biol (2007) 52(23):7137–52. doi: 10.1088/0031-9155/52/23/024

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Kalet A, Sandison G, Wu H, Schmitz R. A State-Based Probabilistic Model for Tumor Respiratory Motion Prediction. Phys Med Biol (2010) 55(24):7615–31. doi: 10.1088/0031-9155/55/24/015

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Toftegaard J, Keall PJ, O'Brien R, Ruan D, Ernst F, Homma N, et al. Potential Improvements of Lung and Prostate MLC Tracking Investigated by Treatment Simulations. Med Phys (2018) 45(5):2218–29. doi: 10.1002/mp.12868

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Tolakanahalli RP, Tewatia DK, Tome WA. Time Series Prediction of Lung Cancer Patients' Breathing Pattern Based on Nonlinear Dynamics. Phys Med (2015) 31(3):257–65. doi: 10.1016/j.ejmp.2015.01.018

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Baumgartner CF, Kolbitsch C, McClelland JR, Rueckert D, King AP. Autoadaptive Motion Modelling for MR-Based Respiratory Motion Estimation. Med Image Anal (2017) 35:83–100. doi: 10.1016/j.media.2016.06.005

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Wang G, Li Z, Li G, Dai G, Xiao Q, Bai L, et al. Real-Time Liver Tracking Algorithm Based on LSTM and SVR Networks for Use in Surface-Guided Radiation Therapy. Radiat Oncol (2021) 16(1):13. doi: 10.1186/s13014-020-01729-7

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Bukovsky I, Homma N, Ichiji K, Cejnek M, Slama M, Benes PM, et al. A Fast Neural Network Approach to Predict Lung Tumor Motion During Respiration for Radiation Therapy Applications. BioMed Res Int (2015) 2015:489679. doi: 10.1155/2015/489679

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Chang P, Dang J, Dai J, Sun W. Real-Time Respiratory Tumor Motion Prediction Based on a Temporal Convolutional Neural Network: Prediction Model Development Study. J Med Internet Res (2021) 23(8):e27235. doi: 10.2196/27235

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Seppenwoolde Y, Shirato H, Kitamura K, Shimizu S, van Herk M, Lebesque JV, et al. Precise and Real-Time Measurement of 3D Tumor Motion in Lung Due to Breathing and Heartbeat, Measured During Radiotherapy. Int J Radiat Oncol Biol Phys (2002) 53(4):822–34. doi: 10.1016/S0360-3016(02)02803-1

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Keall PJ, Mageras GS, Balter JM, Emery RS, Forster KM, Jiang SB, et al. The Management of Respiratory Motion in Radiation Oncology Report of AAPM Task Group 76. Med Phys (2006) 33(10):3874–900. doi: 10.1118/1.2349696

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Paganelli C, Lee D, Greer PB, Baroni G, Riboldi M, Keall P. Quantification of Lung Tumor Rotation With Automated Landmark Extraction Using Orthogonal Cine MRI Images. Phys Med Biol (2015) 60(18):7165–78. doi: 10.1088/0031-9155/60/18/7165

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Poels K, Dhont J, Verellen D, Blanck O, Ernst F, Vandemeulebroucke J, et al. A Comparison of Two Clinical Correlation Models Used for Real-Time Tumor Tracking of Semi-Periodic Motion: A Focus on Geometrical Accuracy in Lung and Liver Cancer Patients. Radiother Oncol (2015) 115(3):419–24. doi: 10.1016/j.radonc.2015.05.004

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Wilms M, Werner R, Ehrhardt J, Schmidt-Richberg A, Schlemmer HP, Handels H. Multivariate Regression Approaches for Surrogate-Based Diffeomorphic Estimation of Respiratory Motion in Radiation Therapy. Phys Med Biol (2014) 59(5):1147–64. doi: 10.1088/0031-9155/59/5/1147

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Yang J, Cai J, Wang H, Chang Z, Czito BG, Bashir MR, et al. Is Diaphragm Motion a Good Surrogate for Liver Tumor Motion? Int J Radiat Oncol Biol Phys (2014) 90(4):952–8. doi: 10.1016/j.ijrobp.2014.07.028

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Jupitz SA, Shepard AJ, Hill PM, Bednarz BP. Investigation of Tumor and Vessel Motion Correlation in the Liver. J Appl Clin Med Phys (2020) 21(8):183–90. doi: 10.1002/acm2.12943

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Hindley N, Keall P, Booth J, Shieh CC. Real-Time Direct Diaphragm Tracking Using kV Imaging on a Standard Linear Accelerator. Med Phys (2019) 46(10):4481–9. doi: 10.1002/mp.13738

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Dhont J, Vandemeulebroucke J, Burghelea M, Poels K, Depuydt T, Van DenBegin R, et al. The Long- and Short-Term Variability of Breathing Induced Tumor Motion in Lung and Liver Over the Course of a Radiotherapy Treatment. Radiother Oncol (2018) 126(2):339–46. doi: 10.1016/j.radonc.2017.09.001

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Park S, Farah R, Shea SM, Tryggestad E, Hales R, Lee J. Simultaneous Tumor and Surrogate Motion Tracking With Dynamic MRI for Radiation Therapy Planning. Phys Med Biol (2018) 63(2):025015. doi: 10.1088/1361-6560/aaa20b

PubMed Abstract | CrossRef Full Text | Google Scholar