Leveraging intelligent optimization for automated, cardiac-sparing accelerated partial breast treatment planning

Background: Accelerated partial breast irradiation (APBI) yields similar rates of recurrence and cosmetic outcomes as compared to whole breast radiation therapy (RT) when patients and treatment techniques are appropriately selected. APBI combined with stereotactic body radiation therapy (SBRT) is a promising technique for precisely delivering high levels of radiation while avoiding uninvolved breast tissue. Here we investigate the feasibility of automatically generating high quality APBI plans in the Ethos adaptive workspace with a specific emphasis on sparing the heart.

Methods: Nine patients (10 target volumes) were utilized to iteratively tune an Ethos APBI planning template for automatic plan generation. Twenty patients previously treated on a TrueBeam Edge accelerator were then automatically replanned using this template without manual intervention or reoptimization. The unbiased validation cohort Ethos plans were benchmarked via adherence to planning objectives, a comparison of DVH and quality indices against the clinical Edge plans, and qualitative reviews by two board-certified radiation oncologists.

Results: 85% (17/20) of automated validation cohort plans met all planning objectives; three plans did not achieve the contralateral lung V1.5Gy objective, but all other objectives were achieved. Compared to the Eclipse generated plans, the proposed Ethos template generated plans with greater evaluation planning target volume (PTV_Eval) V100% coverage (p = 0.01), significantly decreased heart V1.5Gy (p< 0.001), and increased contralateral breast V5Gy, skin D0.01cc, and RTOG conformity index (p = 0.03, p = 0.03, and p = 0.01, respectively). However, only the reduction in heart dose was significant after correcting for multiple testing. Physicist-selected plans were deemed clinically acceptable without modification for 75% and 90% of plans by physicians A and B, respectively. Physicians A and B scored at least one automatically generated plan as clinically acceptable for 100% and 95% of planning intents, respectively.

Conclusions: Standard left- and right-sided planning templates automatically generated APBI plans of comparable quality to manually generated plans treated on a stereotactic linear accelerator, with a significant reduction in heart dose compared to Eclipse generated plans. The methods presented in this work elucidate an approach for generating automated, cardiac-sparing APBI treatment plans for daily adaptive RT with high efficiency.

1 Introduction

The incidence rate of early stage breast cancer is steadily increasing due to improved detection and screening strategies (1). Equivalent overall survival rates of lumpectomy followed by external beam radiation therapy (RT) compared to mastectomy have been shown (2), and post-lumpectomy pathologic analysis by Vicini et al. demonstrated that residual disease occurred within 1cm of the lumpectomy cavity for more than 90% of patients (3). Until recently, external beam accelerated partial breast irradiation (APBI) has been less preferred to brachytherapy APBI due to the large planning target volume (PTV) margins necessary to account for set-up uncertainty, resulting in increased healthy tissue exposure and inferior cosmetic outcomes relative to whole breast RT (4). However, technical improvements in patient immobilization, imaging, and dosimetry have more recently piqued interest in stereotactic body radiation therapy (SBRT), which allows for reduced margins and steeper dose fall-off outside of the target.

To that end, Vermeulen et al. observed no toxicities ≥ grade 3 for 46 stage 1 patients receiving supine SBRT treatment with a 2mm PTV expansion (5, 6). Additionally, Timmerman et al. published methods and cosmetic outcomes for a 75 patient, five arm dose-escalation SBRT trial in which high rates of good or excellent cosmesis were achieved (7, 8). Livi et al. demonstrated that compared to conventionally fractionated (50Gy in 25 fractions) breast treatment, intensity modulated radiation therapy (IMRT) based PBI resulted in significantly fewer short and long term toxicities and improved cosmetic satisfaction compared to whole breast RT using 1cm PTV margins (9). Based on these findings, our institution initiated the UAB RAD 1802 trial (Pilot Trial of LINAC Based Stereotactic Body Radiotherapy for Early Stage Breast Cancer Patients Eligible for Post-Operative Accelerated Partial Breast Irradiation (APBI); clinicaltrials.gov identifier NCT03643861). The purpose was to combine the SBRT techniques and accelerated fractionation schemes, which were previously exclusively utilized on the Cyberknife platform, with the IMRT capabilities of a traditional linear accelerator. Methods and preliminary findings for the first 23 patients (16 prone, 7 supine) have since been published (10).

While novel platforms such as RapidPlan and HyperArc (Varian Medical Systems) have provided a means of automating planning processes (11, 12), many institutions still heavily rely on iterative, manual planning (13, 14). Developing alternatives to manual planning would be ideal as the time required to train personnel and manually generate high-quality treatment plans remains costly (15). Furthermore, planning skill varies greatly by planner and site (16, 17), manual plan constraints and optimization structures are often inconsistent, and time limitations greatly impact the quality of manual plans. Thus, the aim of automation is to increase plan consistency, reduce planning time, and maintain or improve plan quality. Popular forms of automation include, but are not limited to, knowledge-based planning (KBP) (18–22), multi-criteria optimization (MCO) (23, 24), and template-based planning (25–28).

There have been few studies showing effective automated planning implementation for the Ethos system. The Ethos (Varian Medical Systems, Palo Alto, CA) is an independent treatment planning system (TPS) that utilizes a unique Intelligent Optimization Engine (IOE) that is designed to mimic the way a skilled planner generates treatment plans. The IOE attempts to minimize the impact of planner ability on final treatment plan quality, reducing interpatient plan quality variation (29). While the IOE was designed to reduce the need for the iterative planner interactions, we have found that plan quality can be highly heterogeneous without adequate tuning of a treatment planning template. To the authors’ knowledge, there is only a single study outlining planning for stereotactic radiation therapy using the IOE (30). This study by Byrne et al. focused on treatment in the brain and lungs, where the planning focus is on plan conformity and dose falloff. However, for stereotactic APBI planning, sparing of proximal organs-at-risk can be just as important as conformity and dose falloff. Because of this, the template proposed by Byrne et al. is not easily translatable for APBI planning. Thus, the primary endpoint of the proposed work is to develop a treatment planning template which creates clinically acceptable treatment plans for stereotactic APBI in a fully automated fashion, which can easily be disseminated in XML format to any institution wishing to treat APBI using the Ethos. Clinical acceptability will be judged by adherence to published clinical trial guidelines created with traditional planning techniques and via evaluation by radiation oncologists with experience treating linear-accelerator based APBI. As a secondary endpoint, the automatically-generated plans will be compared to previously treated plans using standard dose-volume histogram metrics used to evaluate overall plan quality

2 Methods and materials

2.1 Cohort description

29 patients (30 plans due to one patient with bilateral disease) previously receiving supine APBI treatment for early stage breast cancer (stages 0-2) at our institution between 2019 and 2022 were utilized in this Institutional Review Board (IRB-1207033005) approved study. Nine patients (10 plans) were used to iteratively tune an optimization template and an independent 20 patient cohort was utilized to validate the template via automatic replanning without intervention or reoptimization. Seven patients met RAD 1802 inclusion criteria and were simulated and contoured according to trial protocol. Inclusion criteria consisted of age ≥ 50, estrogen receptor (ER) positive, and negative margins of at least 2mm for invasive histology or 3mm for ductal carcinoma in situ, carcinoma in situ, or T1 disease. Patients receiving neoadjuvant chemotherapy or having multifocal cancer, pure invasive lobular histology, surgical margins< 2mm, a lumpectomy cavity within 5mm of the body contour, or unclear cavity delineation on the planning scan were excluded. Additionally, patients with evaluation PTV (PTV_Eval) volumes exceeding 124cc were excluded based on fat necrosis observed by Timmerman et al. above this threshold (8). 23 patients were not included in the RAD 1802 study, but were simulated, contoured, and planned with the same methods and intent. For all patients, an isotropic 1cm gross tumor volume (GTV) expansion was utilized for clinical target volume (CTV) generation and an isotropic 3mm CTV expansion was utilized for PTV generation. PTV_Eval volumes were created by carving out the PTV at anatomical boundaries (i.e., lung, rib, chest wall, and 5mm from the skin). PTV_Eval volume ranged from 28.6cc to 217.9cc, with an average of 85.2cc. Patients were prescribed 30Gy in five fractions, with an average 98.3% of the PTV_Eval receiving 30Gy in the original clinical plans. Patient characteristics are summarized in Table 1.

TABLE 1

Table 1 Patient cohort description.

2.2 Treatment planning

Nine patients previously treated on the Ethos were selected for our tuning cohort (one bilateral patient, four left breast plans, six right breast plans). The tuning cohort was used to establish an Ethos planning template that generated plans meeting RAD 1802 treatment planning goals (Table 2) through iterative planning and fine-tuning of the optimization objectives. A particular emphasis was placed on lowering heart dose to the extent possible while maintaining otherwise similar plan quality to clinical plans. Twenty patients originally receiving supine RT on a Varian TrueBeam Edge were assigned to the validation cohort (seven left breast, thirteen right breast), and were automatically planned using the template resulting from the tuning cohort. Clinically approved Eclipse contours were exported from Eclipse to Ethos and were used for plan generation without modification (i.e., the manually-generated Eclipse lung contour was used in optimization instead of the Ethos auto-contoured lung volume). Ethos validation cohort plans were not reoptimized or renormalized prior to evaluation and were thus evaluated “as-is”.

TABLE 2

Table 2 APBI planning goals utilized in this study.

Clinical Edge plans were originally calculated with Acuros XB (AXB version 15.5.11, Varian Medical Systems) with heterogeneity correction on and dose-to-water selected. Because Ethos automatically calculates with AXB, dose-to-medium (version 16.1.0), all 20 Edge plans were recalculated using dose-to-medium prior to plan comparison. Recalculations preserved beam geometries and field weightings, but plans were re-normalized to the clinically accepted PTV_Eval prescription isodose coverage. A 2.5mm grid was used for dose calculation in both TPS. The Varian TrueBeam Edge is a stereotactic linear accelerator equipped with a 10MV flattening filter free (FFF) beam, high definition MLCs (HDMLC: 0.25cm in the center, 0.50cm in the periphery), and a maximum dose rate of 2400 MU/min. The Ethos is a CBCT-guided adaptive capable rotational linear accelerator equipped with a 6MV FFF beam, dual stacked and staggered MLC banks as its primary form of collimation, and a maximum dose rate of 800 MU/min (29).

The Ethos pre-defined planning geometries selected for this work include equidistant 9- and 12-field IMRT plans, an ipsilateral 7-field IMRT plan, a 2 full-arc VMAT plan, and a 2 half-arc (180-degree arc span) VMAT plan. While Eclipse optimization is dictated by an internal cost function that varies with assigned priority number, Ethos plans are optimized according to the ascending order of planning objectives submitted in the dose preview workspace. The optimum plan geometry generated from each intent was selected by the reviewing physicist based on adherence to RAD 1802 objectives. Selected Ethos plans were exported to Eclipse, where they were benchmarked dosimetrically against clinically delivered Edge plans. Eclipse and Ethos objective metrics and dose volume histograms (DVH) were extracted via the Eclipse Scripting Application Programming Interface (version 16.1). In addition to presenting the RTOG CI (30), high dose spillage (8) and Paddick gradient index (GI) (31) values were calculated to enable a more holistic plan quality evaluation. The CI, high-dose spillage, and GI are defined in equations (1) – (3).

$\begin{array}{ll}\text{RTOG CI} =\frac{\text{PIV}}{\text{TV}}& (1)\end{array}$

$\begin{array}{ll}\text{High dose spillage} (\%) = 100\ast \frac{\text{PI}{\text{V}}_{105\%}-\text{T}{\text{V}}_{105\%}}{\text{TV}}& (2)\end{array}$

$\begin{array}{ll}\text{Paddick GI} =\frac{\text{PI}{V}_{50\%}}{\text{PIV}}& (3)\end{array}$

Here PIV and TV are the prescription isodose volume and treated volume (i.e., PTV_Eval volume), respectively. Subscripts specify the isodose volumes evaluated if different than 100%. The Wilcoxon paired, non-parametric test was utilized to test for significant difference between Eclipse and Ethos plan metrics. When conducting multiple tests on the same dependent variable, the likelihood of observing a significant result by pure chance increases. Thus, a Bonferroni correction was applied to adjust for multiple testing, and p< 0.004 is considered significant (α = 0.05/12). Statistical analyses were performed in the Python ScyPy library without removal of outliers.

2.3 Physician review

Two board certified radiation oncologists specializing in accelerated partial breast treatment qualitatively evaluated all twenty automatically generated Ethos validation cohort plans according to a previously-utilized in-house grading scheme, which is outlined in Table 3 (32). To avoid scoring bias, the physicians were not shown the Ethos optimization template before evaluation; in addition, the physicists did not provide feedback or respond to physicians during evaluation, nor were the physicians aware of the cardiac-sparing emphasis of this study. Rather, the physicians graded each plan based on their past clinical experience and their unique interpretation of the scoring criteria. Physicians were not provided case-specific information and performed evaluations solely with anonymous patient identifiers. In cases where plans selected by the physicist would require modification prior to treatment (i.e., a clinically unacceptable physician score of 1-3), the physicians were asked to evaluate their preferred alternative geometry plan for clinical acceptability. The proportion of planning intents that automatically generated at least one plan that the physician deemed clinically acceptable without re-optimization was then evaluated.

TABLE 3

Table 3 Physician qualitative review grading scheme.

3 Results

3.1 Planning template and intent

From each APBI planning intent submitted in Ethos, five plans with varying geometries were automatically generated. A total of 110 intent and intent revisions were created in this work, equating to the generation and evaluation of 550 unique APBI plans. Ninety intents were required to iteratively plan the nine patient tuning cohort, and the twenty validation cohort patients were each only planned with one unbiased intent. When plans were optimized solely using the RAD 1802 dosimetric objectives in Table 2, many plans failed to meet planning goals. Thus, the planning template in Table 4 was iteratively procured to maximize the likelihood of achieving all planning objectives. The left sided template is shown as an example, but the right sided template is included in Supplementary Material. Both templates in XML format are available upon request for easy reproduction of this work by other researchers.

TABLE 4

Table 4 Ethos left-sided APBI planning template. The skin was generated using a 3mm inward expansion of the body surface.

The template prioritizes GTV coverage the highest, followed by PTV coverage and heart avoidance. The contralateral lung V1.5Gy was given lower priority in the right-sided than in the left-sided template because heart metrics were more challenging to meet for left sided treatments. This lead the optimizer to spill low dose into the contralateral lung in the absence of a higher priority objective. The left and right templates were identical besides the contralateral lung V1.5Gy constraint. The PTV was cropped out of the ipsilateral breast to avoid conflicting objectives prior to optimization (i.e., asking the optimizer to irradiate the PTV but spare the breast + PTV). The entire ipsilateral breast (including PTV) was designated as a report only structure and was thus not optimized. The template contains three rings constituting seven objectives focused solely on conformity, fall-off, and limiting high dose spillage. The inner, middle, and outer rings are derived from (0 - 0.5)cm, (0.5 - 1.0)cm, and (1.0 – 3.0)cm PTV_Eval expansions inside of the Body, respectively.

3.2 Plan selection

The twenty patient validation cohort was originally treated on the Edge using 6-field (n = 1), 8-field (n = 1), and 9-field IMRT (n = 5), as well as 2 partial VMAT arcs (n = 13). Validation cohort plan geometries chosen by the physicist to benchmark against the clinical Edge plans are as follows: seven equidistant 9-field plans (35%), four equidistant 12-field plans (20%), five ipsilateral 7-field plans (25%), three VMAT plans with 2 partial arcs (15%), and one VMAT plan with 2 full-arcs (5%). Because sparing the heart is a primary emphasis of this study, Figure 1 shows Ethos and Eclipse axial, sagittal, and coronal dose distributions (1Gy – 38Gy) for the manually generated plan with the highest heart V1.5Gy metric. As is visually evident, the Ethos IOE automatically produces an equidistant 9-field IMRT plan with significant cardiac sparing relative to the manual lateral 6-field IMRT plan.

FIGURE 1

Figure 1 Axial, coronal, and sagittal dose distributions of the manual plan with the highest heart V1.5Gy metric for both Eclipse and Ethos. The Eclipse and Ethos plans utilize 6 lateral fields and 9 equidistant fields, respectively. The planning target volume and heart were contoured in red and pink, respectively.

3.3 Dosimetry evaluation

The proposed template automatically generated plans meeting all RAD 1802 objectives for 85% (17/20) of plans without reoptimization. Three initially-selected plans failed to meet the contralateral lung V1.5Gy constraint. No other constraints were violated in any validation cohort plan. 90% (18/20) of the manually generated clinical Edge plans met all objectives; one plan had less than 95% of the PTV receiving prescription dose and one plan exceeded the contralateral lung V1.5Gy constraint. Boxplots showing validation cohort metric summaries for Ethos and Eclipse are displayed in Figure 2. Ethos plans had greater PTV_Eval V100% coverage (p = 0.01), decreased heart V1.5Gy (p< 0.001), but increased contralateral breast V5Gy and skin D0.01cc. (p = 0.03 and p = 0.03 respectively). Although several metrics have medians and interquartile ranges (IQR) that differ, only the heart V1.5Gy distributions are significantly different when a Bonferroni correction is applied to adjust for multiple hypothesis testing. The Eclipse left sided heart V1.5Gy IQR and maximum value and are 21.7% and 29.3%, respectively, whereas they are 0.4% and 0.5% for Ethos. The minimum Eclipse right sided heart V1.5Gy metric is 1.4% while the maximum Ethos V1.5Gy metric is 0.6%.

FIGURE 2

Figure 2 Boxplots summarizing manual Eclipse and automated Ethos validation cohort planning metrics. Open and closed circles indicate outlier and mean values, respectively. Significance values for the difference between TPS metric distributions were obtained via the Wilcoxon signed rank test and are stratified as follows: ns (not significant): (0.004, p, 1.00]; *: (0, p, 0.004].

All Ethos and Eclipse plans easily met the 1.30 CI planning objective; one Ethos outlier was much greater than all other plans and one Eclipse plan had a CI of 0.95 due to 92.7% PTV_Eval coverage. The median Eclipse and Ethos CI were 1.05 and 1.06, respectively. 100% of the Eclipse and Ethos plans met the 15% high-dose spillage constraint planning suggested in the Timmerman study (8). There is little discernable difference in high-dose spillage and GI distributions between both TPS when outliers are excluded. Ethos plans generally had more compact high-dose spillage values, but a greater GI IQR. The median Ethos GI was lower, but mean values were similar. While Eclipse CI values were lower than Ethos (p = 0.01), there were no significant quality metric differences between both TPS.

Validation cohort mean DVHs with standard deviation bounds are presented for both TPS in Figure 3. The inferior/superior triangle tips illustrate planning objectives and the insets elucidate DVH difference between both TPS (i.e., Ethos volume minus Eclipse volume as a function of dose). Ethos had superior PTV coverage between approximately 29.5Gy and 31.50Gy, but a lower portion of the target received above 105% of prescription dose, which is generally preferred for SBRT. Ethos significantly spares the heart above 0.25Gy, and on average, the heart volume receiving 1Gy was 10.8% less for automated Ethos plans. All left sided Ethos plans were substantially below the right sided planning objective. While the ipsilateral breast DVH curves are similar for high doses, Ethos spares the breast below approximately 11Gy, with a reduction of 3% breast volume receiving 6Gy. Ethos automated plans had overall higher ipsilateral lung dose above 2.5Gy, but the discrepancy between plan types was at most 1.4%. The template presented here generated plans with generally inferior contralateral breast dose; 3.4% additional volume received 2.3Gy. Because the Ethos planning approach heavily spared the heart, automated planning also resulted in much lower contralateral lung dose, with 6.5% less volume receiving 1Gy on average.

FIGURE 3

Figure 3 Population DVH comparison of Eclipse manual and Ethos automatic plans. Shaded areas show the mean ± standard deviation of all validation cohort data, and the inferior/superior point of triangles illustrate RAD 1802 planning objectives. Insets show the difference between mean population DVHs (i.e., Ethos mean volume minus Eclipse mean volume). Inset axes were sized for optimal visualization.

3.4 Qualitative evaluation

The physician score summary for physicist-selected Ethos validation cohort plans is shown in Table 5. Physicians A and B considered 75% (15/20) and 90% (18/20) of plans clinically acceptable (scores of 4 or 5) without modification, respectively. 75% of the selected plans (15/20) received a clinically acceptable score from both reviewing physicians. The mode scores of physicians A and B are 4 and 5, respectively. When physicians scored the physicist-selected plan 3 or lower, they then evaluated the alternate plan geometries generated from the same treatment intent and scored the plan they favored most. The five plans receiving a score of 3 from physician A received one 4 and four 5s when alternate plans were evaluated. The two plans receiving a score of 3 from physician B received one 3 and one 4 when alternate plans were evaluated. Thus, at least one plan of treatable quality was automatically generated using the proposed planning approach for 100% of intents for physician A and 95% of intents for physician B.

TABLE 5

Table 5 Qualitative scoring summary of plans selected by the physicist for physicians A and B.

Four plans received a score of 3 from physician A due to the lateral extent of 15Gy streaking prevalent in IMRT plans. One plan received a 3 because physician A preferred the contralateral breast and lung V5Gy be further reduced given favorable patient anatomy. Both plans receiving a score of 3 from physician B were penalized due to lateral extent of 15Gy streaking. However, physician B further specified that they would have considered whole breast treatment over APBI for the plan receiving a score of 3 even after alternate plan evaluation, primarily due to challenging anatomy and target location.

4 Discussion

In this work, we evaluated APBI plans automatically generated from a standard planning approach in the Ethos adaptive workspace; nine patients (ten plans) were iteratively re-planned until desired quality was achieved and twenty validation cohort patients were only planned once using the resulting template. 85% of selected validation cohort plans met all planning objectives with significant reduction in heart dose, and physicians A and B scored 75% and 90% of physicist-selected plans as clinically acceptable, respectively. Physicians A and B deemed at least one automatically generated plan clinically acceptable, without modification, 100% and 95% of the time, respectively. This study showed that high quality APBI treatment plans can be created in an automated process with a well-tuned template. Although we do not measure planning times prospectively, our team estimates that creation of a clinically acceptable treatment plan takes between 1 and 4 hours of active planner time for one case. With the proposed template, a plan can be created from scratch with around 5 minutes of active time to set up and approximately an hour of passive time for optimization and dose calculation running as a background process. Additionally, we have shown that automated plans were of similar quality to manual plans while simultaneously reducing heart dose.

Four patients in this study with PTV_Eval volumes > 124cc (two in the tuning cohort, two in the validation cohort) received APBI treatment despite failing to meet RAD 1802 inclusion criteria. The treating physician for these cases, who is also the RAD 1802 principal investigator, was comfortable exceeding this threshold due to personal APBI experience, and because these patients had larger breasts or were receiving re-irradiation. Ipsilateral breast V30Gy and V15Gy objectives were achieved for all four patients.

While Ethos contralateral lung dose was on average significantly less than Eclipse dose due to heart avoidance, there were three outlier plans with high contralateral lung dose. 1/13 right-sided plans and 2/7 left-sided plans did not meet the V1.5Gy objective, suggesting that the template may be further improved by increasing the priority of contralateral lung planning goals, especially for the left-sided template. However, the effects of this change in priority require further dosimetric investigation and physician evaluation, as this may affect dose contribution to the ipsilateral lung or contralateral breast, or both. The priority adjustment described above should be considered if contralateral lung dose constraints are exceeded during clinical implementation. Additionally, as this template is being used for adaptive radiation therapy, it is possible that the dose the patient receives to the contralateral lung is lower than the initial plan based on changes in daily anatomy

Significant effort has been dedicated to sparing the heart in lung RT due to high levels of proximal dose (33, 34), but it has also been observed that breast RT induces cardiac toxicity linearly with no apparent dose threshold. Increased risk in major coronary events between 7.4% and 19% per additional Gy of mean heart dose have been reported for breast RT (35–37). The initial clinically-treated plans for comparison in this work were of high-quality, with cardiac dose levels below the UAB RAD 1802 objectives for all 20 validation cohort plans. Drawing from the data published by Darby et al, which showed that risk of major coronary events increased linearly with mean heart dose with no apparent threshold (35), it becomes imperative that the planner continue to minimize heart dose, even well below acceptable levels (i.e., V1.5Gy< 5% and 40% for right and left-sided targets, respectively (8, 10)), so long as the net effect on target coverage and sparing of other OARs is not detrimental. To that end, the authors argue that leveraging the Ethos to spare even 1Gy is clinically meaningful, so long as other Ethos plan characteristics are similar in quality to manual Eclipse plans. As shown in Figure 3, the template-generated plans led to reductions in heart dose above 0.5 Gy relative to the Eclipse plans. It should also be mentioned that the OAR dose being spared would be greater were the 6Gy x 5 hypo-fractionated scheme converted to 2Gy equivalent fractions. Furthermore, the Ethos platform allows for RT plan adaption based on daily cone beam CT (CBCT) anatomy (38, 39), which could allow for further reduced doses due to daily re-optimization.

Ethos plans were slightly, but consistently, less conformal than Eclipse plans. While some of this discrepancy may be attributed to template design and optimizer differences, it is due at least in part to tertiary collimation width. The double banked, 10mm width Ethos MLC bank is staggered, effectively producing 5mm width MLCs. The Edge has 2.5mm central HDMLC leaves, resulting in twice the collimation resolution. It is reasonable to assume that Ethos plans would see some measurable reduction in CI and high-dose spillage were the MLC width halved. However, Automated Ethos plans had superior CI values (1.07± 0.05) compared to the 30Gy arm published by Timmerman et al. using the Cyberknife (1.22 ± 0.10) (7). It is also important to note that the mean Ethos validation cohort target volume was smaller than the mean 30Gy arm Cyberknife cohort target volume (Ethos: 77.6cc; Cyberknife: 80.9cc), and CI typically decreases with increasing target size. Thus, the authors argue that the automated plans presented here, while slightly less conformal than Edge plans, are still of high-quality. Further studies are required to deconflate the effects of the different collimators and optimization engine on Ethos plan quality.

The upper Ethos outlier for CI and high dose spillage originated from one plan. This plan presented challenging and abnormal patient anatomy which elucidates a fundamental limitation of this study: fixed beam geometries. The target of interest was the smallest PTV_Eval in the validation cohort and located medially in the upper, inner breast quadrant. The standard field geometries failed to address the patient-specific anatomy; the 2 partial arc and lateral IMRT field geometries span angles from 0° to 180°, clockwise, and the equidistant 9 and 12-field IMRT geometries only space fields every 40° and 30°, respectively. Given the very medial nature of this target, it would have benefitted from partial arcs or densely placed lateral IMRT fields ranging from -90° to 90°. This example highlights that the proposed template does not negate the need for dosimetrist involvement or patient-specific anatomy review; it is expected that abnormal target location or anatomy will require beam geometry modification prior to planning in some instances.

Both reviewing physicians performed a slice-by-slice evaluation of all validation cohort plans. Physicians considered disease extent and location, anatomy favorability, dose distribution shape, and PTV undercoverage in addition to verifying satisfactory DVH metrics. IMRT plan geometries tended to have comparable or even improved GI relative to VMAT, leading the reviewing physicist to select many IMRT plans for further evaluation. However, physician A strongly preferred the consolidated shape of VMAT 15Gy isodose lines compared to IMRT, which tended to exhibit greater lateral extent but similar volume. Physician B was not as opposed to 15Gy streaking, except in more serious cases. This highlights the role of personal preference when reviewing plans qualitatively. While we observed stylistic differences in plan evaluation between the two physician raters, the template provides a mechanism to standardize practices across practitioners, resulting in a large majority of evaluated plans considered acceptable during qualitative review. A future prospective analysis will elucidate if any changes are made after the proposed template is clinically commissioned for use outside of this study.

Artificial intelligence (AI) promises to revolutionize every aspect in radiation oncology care, and has already made a profound impact in enabling the clinical implementation of online adaptive radiotherapies (40, 41). From automated contouring (42–44) to radiotherapy dose estimations (45–47), AI applications are playing a key role increasing efficiency and, often times, improving quality of care through more consistent radiotherapy (48). For example, studies have shown that auto-contouring can significantly save contouring time, providing the critical time savings needed to minimize patient motion during online adaptive treatment design and delivery (49). While most clinical applications currently focus on efficiency improvements, we can expect that in the near future clinical teams will be supported by various AI-driven clinical support systems to compliment decision-making during adaptive treatment’s design and delivery. In the current study, we evaluate radiotherapy treatment plans generated using Varian’s IOE, which uses an artificial intelligence driven optimization process to automatically generate radiotherapy treatment plans. Our study shows that this novel optimization engine provides high-quality APBI treatment plans for a large majority of cases (with no planner interaction) after defining a robust planning template through a data-driven iterative approach.

APBI treatments were transitioned from the Edge to the Ethos in 2021 at our institution, and APBI treatment for 17 patients has been successfully completed in the Ethos adaptive workspace. During the first course of adaptive treatment on the Ethos, we noticed that the GTV location, volume, and shape changed from simulation to first fraction, and between each subsequent fraction. Consequently, adapted plans significantly spared OARs compared to scheduled plans (i.e., initial treatment-approved plans recalculated onto daily CBCT anatomy). Therefore, even though automated Ethos plans are overall similar in quality to manual Eclipse plans, the added benefit of daily CBCT based adaption vastly outweighs whatever slight deficiencies might exist in the proposed Ethos planning approach (i.e., higher Ethos contralateral breast dose). The impact of daily adaptation on both plan quality and patient outcomes warrants further investigation. Other future projects include implementing the APBI template presented here into our clinical workflow and continuing to generate planning templates for other sites.

The manuscript presented here, including study design and analysis, was developed for consistency with recently published RATING guidelines for generating high-quality planning studies (RAdiotherapy Treatment plannINg study Guidelines) (50). The authors’ self-assessment score was 94% (195/207) and the accompanying grading template is added to the Supplementary Material.

Although APBI planning is challenging due to proximity of many OARs and the need for conformity and steep dose gradients, the Ethos templates investigated in this work automatically generate high-quality left- or right-sided APBI plans. Ethos plans had similar target coverage, reduced heart dose, and otherwise similar OAR dose to manual Eclipse plans. 85% of validation cohort plans met all planning objectives, and only the contralateral lung V1.5Gy objective was violated for any plan. Physicians A and B scored at least one plan from each intent of clinically acceptable quality, without reoptimization, 100% and 95% of the time, respectively. Therefore, the approach summarized here enables consistent and high-quality generation of Ethos APBI plans with a specific emphasis on minimizing heart dose.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving human participants were reviewed and approved by the Institutional Review Board 1207033005. Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.

Author contributions

JP conceptualized the study, generated treatment plans, analyzed data, and prepared the manuscript. CC conceptualized the study, anonymized patient data, and prepared the manuscript. YC analyzed data and prepared the manuscript. RP conceptualized the study and prepared the manuscript. MS and DB conceptualized the study, scored treatment plans, and prepared the manuscript. DS conceptualized the study, generated treatment plans, and prepared the manuscript. JH conceptualized the study, analyzed data, and prepared the manuscript. All authors contributed to the article and approved the submitted version.

Conflict of interest

RP’s institution, University of Alabama at Birmingham, has product evaluation agreements and research grants with Varian Medical Systems. He has a patent licensed by UAB Research Foundation to Varian Medical Systems. He has received honoraria for presentations on behalf of Varian Medical Systems. He has received a stipend to speak at Sun Nuclear meetings. Varian Medical Systems provides equipment to UAB as a part of a product evaluation agreement. DB has received research support from Varian including for this study, speaker honoraria, and research support from Novocure. DS received honoraria from Varian medical systems to present on their behalf and is an educational consultant speaker for Varian Medical Systems; He received support through a clinical trial sponsored by Varian Medical Systems.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fonc.2023.1130119/full#supplementary-material

References

1. Ahmad A. Breast cancer statistics: Recent trends. Adv Exp Med Biol (2019) 1152:1–7. doi: 10.1007/978-3-030-20301-6_1

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Fisher B, Anderson S, Bryant J, Margolese RG, Deutsch M, Fisher ER, et al. Twenty-year follow-up of a randomized trial comparing total mastectomy, lumpectomy, and lumpectomy plus irradiation for the treatment of invasive breast cancer. New Engl J Med (2002) 347:1233–41. doi: 10.1056/NEJMoa022152

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Vicini FA, Kestin LL, Goldstein NS. Defining the clinical target volume for patients with early-stage breast cancer treated with lumpectomy and accelerated partial breast irradiation: A pathologic analysis. Int J Radiat Oncol Biol Phys (2004) 60:722–30. doi: 10.1016/j.ijrobp.2004.04.012

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Olivotto IA, Whelan TJ, Parpia S, Kim DH, Berrang T, Truong PT, et al. Interim cosmetic and toxicity results from RAPID: A randomized trial of accelerated partial breast irradiation using three-dimensional conformal external beam radiation therapy. J Clin Oncol (2013) 31:4038–45. doi: 10.1200/JCO.2013.50.5511

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Vermeulen S, Cotrutz C, Buchanan C, Dawson P, Morris A, Porter B, et al. Accelerated partial breast irradiation: Using the CyberKnife as the radiation delivery platform in the treatment of early breast cancer. Front Oncol (2011) 1. doi: 10.3389/fonc.2011.00043

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Vermeulen SS, Haas JA. CyberKnife stereotactic body radiotherapy and CyberKnife accelerated partial breast irradiation for the treatment of early breast cancer. Transl Cancer Res (2014) 3:295–302. doi: 10.3978/j.issn.2218-676X.2014.07.06

CrossRef Full Text | Google Scholar

7. Rahimi A, Morgan HE, Kim DW, Zhang Y, Leitch M, Wooldridge R, et al. Cosmetic outcomes of a phase 1 dose escalation study of 5-fraction stereotactic partial breast irradiation for early stage breast cancer. Int J Radiat Oncol Biol Phys (2021) 110:772–82. doi: 10.1016/j.ijrobp.2021.01.015

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Rahimi A, Thomas K, Spangler A, Rao R, Leitch M, Wooldridge R, et al. Preliminary results of a phase 1 dose-escalation trial for early-stage breast cancer using 5-fraction stereotactic body radiation therapy for partial-breast irradiation. Int J Radiat Oncol Biol Phys (2017) 98:196–205.e2. doi: 10.1016/j.ijrobp.2017.01.020

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Livi L, Meattini I, Marrazzo L, Simontacchi G, Pallotta S, Saieva C, et al. Accelerated partial breast irradiation using intensity-modulated radiotherapy versus whole breast irradiation: 5-year survival analysis of a phase 3 randomised controlled trial. Eur J Cancer (2015) 51:451–63. doi: 10.1016/j.ejca.2014.12.013

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Liu Y, Veale C, Hablitz D, Krontiras H, Dalton A, Meyers K, et al. Feasibility and short-term toxicity of a consecutively delivered five fraction stereotactic body radiation therapy regimen in early-stage breast cancer patients receiving partial breast irradiation. Front Oncol (2022) 12:901312. doi: 10.3389/fonc.2022.901312

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Popple RA, Brown MH, Thomas EM, Willey CD, Cardan RA, Covington EL, et al. Transition from manual to automated planning and delivery of volumetric modulated arc therapy stereotactic radiosurgery: Clinical, dosimetric, and quality assurance results. Pract Radiat Oncol (2021) 11:e163–71. doi: 10.1016/j.prro.2020.10.013

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Thomas EM, Phillips H, Popple RA, Fiveash JB. Development of a knowledge based model (RapidPlan) for brain metastasis stereotactic radiosurgery and validation with automated non-coplanar treatment planning (HyperArc). Int J Radiat Oncol (2017) 99:E727–8. doi: 10.1016/j.ijrobp.2017.06.2353

CrossRef Full Text | Google Scholar

13. Kisling KD, Ger RB, Netherton TJ, Cardenas CE, Owens CA, Anderson BM, et al. A snapshot of medical physics practice patterns. J Appl Clin Med Phys (2018) 19:306–15. doi: 10.1002/acm2.12464

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Petragallo R, Bardach N, Ramirez E, Lamb JM. Barriers and facilitators to clinical implementation of radiotherapy treatment planning automation: A survey study of medical dosimetrists. J Appl Clin Med Phys (2022) 23:e13568. doi: 10.1002/acm2.13568

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Winkel D, Bol GH, van Asselen B, Hes J, Scholten V, Kerkmeijer LG, et al. Development and clinical introduction of automated radiotherapy treatment planning for prostate cancer. Phys Med Biol (2016) 61:8587–95. doi: 10.1088/1361-6560/61/24/8587

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Moore KL, Brame RS, Low DA, Mutic S. Quantitative metrics for assessing plan quality. Semin Radiat Oncol (2012) 22:62–9. doi: 10.1016/j.semradonc.2011.09.005

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Nelms BE, Robinson G, Markham J, Velasco K, Boyd S, Narayan S, et al. Variation in external beam treatment plan quality: An inter-institutional study of planners and planning systems. Pract Radiat Oncol (2012) 2:296–305. doi: 10.1016/j.prro.2011.11.012

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Ge Y, Wu QJ. Knowledge-based planning for intensity-modulated radiation therapy: A review of data-driven approaches. Med Phys (2019) 46:2760–75. doi: 10.1002/mp.13526

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Li N, Carmona R, Sirak I, Kasaova L, Followill D, Michalski J, et al. Highly efficient training, refinement, and validation of a knowledge-based planning quality-control system for radiation therapy clinical trials. Int J Radiat Oncol Biol Phys (2017) 97:164–72. doi: 10.1016/j.ijrobp.2016.10.005

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Tambe NS, Pires IM, Moore C, Cawthorne C, Beavis AW. Validation of in-house knowledge-based planning model for advance-stage lung cancer patients treated using VMAT radiotherapy. Br J Radiol (2020) 93:20190535. doi: 10.1259/bjr.20190535

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Olanrewaju A, Court LE, Zhang L, Naidoo K, Burger H, Dalvie S, et al. Clinical acceptability of automated radiation treatment planning for head and neck cancer using the radiation planning assistant. Pract Radiat Oncol (2021) 11:177–84. doi: 10.1016/j.prro.2020.12.003

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Rhee DJ, Jhingran A, Kisling K, Cardenas C, Simonds H, Court L. Automated radiation treatment planning for cervical cancer. Semin Radiat Oncol (2020) 30:340–7. doi: 10.1016/j.semradonc.2020.05.006

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Kierkels RG, Visser R, Bijl HP, Langendijk JA, van ‘t Veld AA, Steenbakkers RJ, et al. Multicriteria optimization enables less experienced planners to efficiently produce high quality treatment plans in head and neck cancer radiotherapy. Radiat Oncol (2015) 10:87. doi: 10.1186/s13014-015-0385-9

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Naccarato S, Rigo M, Pellegrini R, Voet P, Akhiat H, Gurrera D, et al. Automated planning for prostate stereotactic body radiation therapy on the 1.5 T MR-linac. Adv Radiat Oncol (2022) 7:100865. doi: 10.1016/j.adro.2021.100865

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Cilla S, Ianiro A, Romano C, Deodato F, Macchia G, Buwenge M, et al. Template-based automation of treatment planning in advanced radiotherapy: a comprehensive dosimetric and clinical evaluation. Sci Rep (2020) 10:423. doi: 10.1038/s41598-019-56966-y

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Vanderstraeten B, Goddeeris B, Vandecasteele K, van Eijkeren M, De Wagter C, Lievens Y. Automated instead of manual treatment planning? a plan comparison based on dose-volume statistics and clinical preference. Int J Radiat Oncol Biol Phys (2018) 102:443–50. doi: 10.1016/j.ijrobp.2018.05.063

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Han EY, Cardenas CE, Nguyen C, Hancock D, Xiao Y, Mumme R, et al. Clinical implementation of automated treatment planning for whole-brain radiotherapy. J Appl Clin Med Phys (2021) 22:94–102. doi: 10.1002/acm2.13350

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Huang K, Das P, Olanrewaju AM, Cardenas C, Fuentes D, Zhang L, et al. Automation of radiation treatment planning for rectal cancer. J Appl Clin Med Phys (2022) 23:e13712. doi: 10.1002/acm2.13712

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Lim TY, Dragojevic I, Hoffman D, Flores-Martinez E, Kim GY. Characterization of the Halcyon(TM) multileaf collimator system. J Appl Clin Med Phys (2019) 20:106–14. doi: 10.1002/acm2.12568

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Shaw E, Kline R, Gillin M, Souhami L, Hirschfeld A, Dinapoli R, et al. Radiation therapy oncology group: radiosurgery quality assurance guidelines. Int J Radiat Oncol Biol Phys (1993) 27:1231–9. doi: 10.1016/0360-3016(93)90548-A

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Paddick I, Lippitz B. A simple dose gradient measurement tool to complement the conformity index. J Neurosurg (2006) 105 Suppl:194–201. doi: 10.3171/sup.2006.105.7.194

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Pogue JA, Cardenas CE, Harms J, Soike MH, Kole AJ, Schneider CS, et al. Design and validation of an automated radiation therapy treatment planning approach for locally advanced lung cancer. medRxiv (2022). doi: 10.1101/2022.09.30.22280584

CrossRef Full Text | Google Scholar

33. Ferris MJ, Martin KS, Switchenko JM, Kayode OA, Wolf J, Dang Q, et al. Sparing cardiac substructures with optimized volumetric modulated arc therapy and intensity modulated proton therapy in thoracic radiation for locally advanced non-small cell lung cancer. Pract Radiat Oncol (2019) 9:e473–81. doi: 10.1016/j.prro.2019.04.013

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Harms J, Zhang J, Kayode O, Wolf J, Tian S, McCall N, et al. Implementation of a knowledge-based treatment planning model for cardiac-sparing lung radiation therapy. Adv Radiat Oncol (2021) 6:100745. doi: 10.1016/j.adro.2021.100745

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Darby SC, Ewertz M, McGale P, Bennet AM, Blom-Goldman U, Bronnum D, et al. Risk of ischemic heart disease in women after radiotherapy for breast cancer. N Engl J Med (2013) 368:987–98. doi: 10.1056/NEJMoa1209825

PubMed Abstract | CrossRef Full Text | Google Scholar

36. van den Bogaard VA, Ta BD, van der Schaaf A, Bouma AB, Middag AM, Bantema-Joppe EJ, et al. Validation and modification of a prediction model for acute cardiac events in patients with breast cancer treated with radiotherapy based on three-dimensional dose distributions to cardiac substructures. J Clin Oncol (2017) 35:1171–8. doi: 10.1200/JCO.2016.69.8480

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Laugaard Lorenzen E, Christian Rehammar J, Jensen MB, Ewertz M, Brink C. Radiation-induced risk of ischemic heart disease following breast cancer radiotherapy in Denmark, 1977-2005. Radiother Oncol (2020) 152:103–10. doi: 10.1016/j.radonc.2020.08.007

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Mao W, Riess J, Kim J, Vance S, Chetty IJ, Movsas B, et al. Evaluation of auto-contouring and dose distributions for online adaptive radiation therapy of patients with locally advanced lung cancers. Pract Radiat Oncol (2022) 12:e329–38. doi: 10.1016/j.prro.2021.12.017

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Sibolt P, Andersson LM, Calmels L, Sjostrom D, Bjelkengren U, Geertsen P, et al. Clinical implementation of artificial intelligence-driven cone-beam computed tomography-guided online adaptive radiotherapy in the pelvic region. Phys Imaging Radiat Oncol (2021) 17:1–7. doi: 10.1016/j.phro.2020.12.004

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Chapman JW, Lam D, Cai B, Hugo GD. Robustness and reproducibility of an artificial intelligence-assisted online segmentation and adaptive planning process for online adaptive radiation therapy. J Appl Clin Med Phys (2022) 23:e13702. doi: 10.1002/acm2.13702

PubMed Abstract | CrossRef Full Text | Google Scholar

41. El Naqa I, Haider MA, Giger ML, Ten Haken RK. Artificial intelligence: Reshaping the practice of radiological sciences in the 21st century. Br J Radiol (2020) 93:20190855. doi: 10.1259/bjr.20190855

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Cardenas CE, Yang J, Anderson BM, Court LE, Brock KB. Advances in auto-segmentation. Semin Radiat Oncol (2019) 29:185–97. doi: 10.1016/j.semradonc.2019.02.001

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Harms J, Lei Y, Tian S, McCall NS, Higgins KA, Bradley JD, et al. Automatic delineation of cardiac substructures using a region-based fully convolutional network. Med Phys (2021) 48:2867–76. doi: 10.1002/mp.14810

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Wong J, Huang V, Wells D, Giambattista J, Giambattista J, Kolbeck C, et al. Implementation of deep learning-based auto-segmentation for radiotherapy planning structures: a workflow study at two cancer centers. Radiat Oncol (2021) 16. doi: 10.1186/s13014-021-01831-4

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Gotz TI, Schmidkonz C, Chen S, Al-Baddai S, Kuwert T, Lang EW. A deep learning approach to radiation dose estimation. Phys Med Biol (2020) 65:035007. doi: 10.1088/1361-6560/ab65dc

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Gronberg MP, Gay SS, Netherton TJ, Rhee DJ, Court LE, Cardenas CE. Technical note: Dose prediction for head and neck radiotherapy using a three-dimensional dense dilated U-net architecture. Med Phys (2021) 48:5567–73. doi: 10.1002/mp.14827

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Nguyen D, Long T, Jia X, Lu W, Gu X, Iqbal Z, et al. A feasibility study for predicting optimal radiation therapy dose distributions of prostate cancer patients from patient anatomy using deep learning. Sci Rep (2019) 9:1076. doi: 10.1038/s41598-018-37741-x

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Cornell M, Kaderka R, Hild SJ, Ray XJ, Murphy JD, Atwood TF, et al. Noninferiority study of automated knowledge-based planning versus human-driven optimization across multiple disease sites. Int J Radiat Oncol Biol Phys (2020) 106:430–9. doi: 10.1016/j.ijrobp.2019.10.036

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Vaassen F, Hazelaar C, Vaniqui A, Gooding M, van der Heyden B, Canters R, et al. Evaluation of measures for assessing time-saving of automatic organ-at-risk segmentation in radiotherapy. Phys Imaging Radiat Oncol (2020) 13:1–6. doi: 10.1016/j.phro.2019.12.001

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Hansen CR, Crijns W, Hussein M, Rossi L, Gallego P, Verbakel W, et al. Radiotherapy treatment plannINg study guidelines (RATING): A framework for setting up and reporting on scientific treatment planning studies. Radiother Oncol (2020) 153:67–78. doi: 10.1016/j.radonc.2020.09.033

PubMed Abstract | CrossRef Full Text | Google Scholar