- 1Imaging and Radiation Oncology Core-Rhode Island, Department of Radiation Oncology, UMass Chan Medical School, Lincoln, RI, United States
- 2Imaging and Radiation Oncology Core-Ohio, Department of Radiology, The Ohio State University, Columbus, OH, United States
- 3Imaging and Radiation Oncology Core-Houston, Division of Radiation Oncology, University of Texas, MD Anderson, Houston, TX, United States
- 4Imaging and Radiation Oncology Core Philadelphia, Department of Radiation Oncology, University of Pennsylvania, Philadelphia, PA, United States
- 5Imaging and Radiation Oncology Core Philadelphia, Department of Radiology, University of Pennsylvania, Philadelphia, PA, United States
- 6Department of Radiation Oncology, Washington University, St Louis, MO, United States
- 7Department of Biomedical Informatics, University of Arkansas, Little Rock, AR, United States
- 8Department of Biomedical Informatics, Stony Brook University, Stony Brook, NY, United States
- 9Department of Biomedical Informatics, Emory University, Atlanta, GA, United States
Clinical trials have become the primary mechanism to validate process improvements in oncology clinical practice. Over the past two decades there have been considerable process improvements in the practice of radiation oncology within the structure of a modern department using advanced technology for patient care. Treatment planning is accomplished with volume definition including fusion of multiple series of diagnostic images into volumetric planning studies to optimize the definition of tumor and define the relationship of tumor to normal tissue. Daily treatment is validated by multiple tools of image guidance. Computer planning has been optimized and supported by the increasing use of artificial intelligence in treatment planning. Informatics technology has improved, and departments have become geographically transparent integrated through informatics bridges creating an economy of scale for the planning and execution of advanced technology radiation therapy. This serves to provide consistency in department habits and improve quality of patient care. Improvements in normal tissue sparing have further improved tolerance of treatment and allowed radiation oncologists to increase both daily and total dose to target. Radiation oncologists need to define a priori dose volume constraints to normal tissue as well as define how image guidance will be applied to each radiation treatment. These process improvements have enhanced the utility of radiation therapy in patient care and have made radiation therapy an attractive option for care in multiple primary disease settings. In this chapter we review how these changes have been applied to clinical practice and incorporated into clinical trials. We will discuss how the changes in clinical practice have improved the quality of clinical trials in radiation therapy. We will also identify what gaps remain and need to be addressed to offer further improvements in radiation oncology clinical trials and patient care.
1 Introduction
Radiation oncology has undergone significant change in patient care treatment processes during the past two decades. Modern radiation oncology training programs have limited resemblance to the programs designed by our mentors. Today, patients are planned with volumetric metrics with image fusion to optimize target definition based on the integration of multiple datasets including response adaptive radiation therapy (RT) planning. Multiple components of motion are managed through several pathways including four-dimensional treatment planning and optical tracking of patient positioning to provide security in daily treatment reproduction. With an increasing number of patients being treated with curative intent or re-treated with definitive fractionation, RT treatment planning is increasingly complex with the majority of patient treatments delivered with intensity modulation and advanced technology therapy techniques. In many clinical situations, higher daily doses are becoming more commonplace as our community becomes increasingly confident in our technology and reproducibility of daily treatment. Plans receive double checks by physics/dosimetry staff and often patients are imaged and not treated during the initial visit in order to confirm that all planning objectives are met, and the positioning is accurate. RT prescriptions include not only daily and total dose but now require a written strategy for dose, a document indicating normal tissue constraints and a written directive for how to apply daily image guidance. Smaller targets with normal tissue exclusion can be successfully treated with compressed fractionation and stereotactic techniques. Imaging tools incorporated into accelerator function serve as both an image positioning validation tool and can also function as a dosimeter confirming daily dose. These can be reviewed daily to ensure the quality and safety of each treatment. Because of the nimble nature of data acquisition through several informatics formats and the ability to display objects through multiple media, intra-departmental physician peer review of contours with image integration can be performed in real time. Therefore, data required for onsite peer review can be acquired and managed through facile onsite tool management strategies.
Many of these improvements are imbedded in protocols currently active in the National Cancer Institute (NCI) National Clinical Trials Network (NCTN) and other sponsored clinical trials and are applied at an enterprise level on a worldwide basis. Therefore, department processes used to manage quality and safety are re-purposed for quality management of clinical trials. In this chapter, the objective is to review how these process improvements are written into clinical trials and managed as part of the clinical trial data acquisition and management process and how these improvements ensure confidence in clinical trial outcome. Challenges remain with clinical trial processes as incomplete datasets and titration of data used to manage the clinical trial can often influence trial outcome. Issues likewise remain to be reconciled in trial management with real time adjustments in target location and how this should be managed in the context of the study. These improvements enhance the quality of treatment and improve patient care (1–17).
2 History of radiation oncology clinical trials
Although many disease areas in medicine do not have a structured mechanism for clinical trial function and management, oncology clinical trials have developed over the past half century with the NCTN. In the late 1960’s, seminal investigators began to design clinical trials in both adult and pediatric oncology to make progress in clinical care. RT was a nascent discipline, nevertheless important questions required consensus review including strategies for dose computation algorithms and radiation dose and fractionation for both tumor control and normal tissue tolerance. The Radiation Therapy Oncology Group (RTOG) developed clinical trials addressing these points in disease areas seminal to RT. The Gynecologic Oncology Group (GOG) integrated RT trials including brachytherapy (6, 14, 15). In multiple other cooperative groups RT would begin as an informal committee but developed voice as radiation was recognized as an important component to patient care. By the mid 1970’s investigators recognized the importance of data acquisition and review of treatment objects to ensure consistent application of RT to patients on trial. There was no established mechanism for this to occur, however early mentors in clinical trials imbedded data transfer strategies into the trials in order to establish a platform to review protocol compliance. The Quality Assurance Review Center (QARC) placed emphasis on data acquisition and data management including the collection of images to validate field placement for radiation oncology (18). QARC became an important feature in the clinical trials process as RT objects including dose computation, planning images and treatment portal images were collected in hard copy to be used for retrospective review of treatment for protocol compliance. QARC established a mechanism for on treatment review of objects whereas treatment plans could be reviewed as therapy was initiated in order to ensure the plan was compliant to study objectives. This significantly improved the quality of the trial as the study deviation rate was considerably improved by this process. QARC was complimented by the efforts of the Radiological Physics Center (RPC) (19–28). The RPC played an essential role in clinical trials by ensuring that dose computation and execution algorithms were compliant to standard (23). This was accomplished by multiple vehicles including the use of phantoms for treatment and review of RT planning dosimetry. Dose was measured by the RPC through multiple vehicles including thermoluminescence dosimetry (TLD) strategically placed in phantoms at critical locations. This would ensure that dose to both target and normal tissue was compliant to a national guideline. Because RT treatment planning was performed on fluoroscopy without the benefit of volumetric planning and validation imaging was performed with megavoltage, the perception in the first iteration of clinical trials was that quality of treatment was driven largely by the consistency of computational analytics and physics calibration. As time matured and volumetric imaging became available, imaging became an important vehicle to validate the target volume of interest and ensure that targets were treated in a protocol compliant manner and tumor was fully treated without exaggeration of dose to normal tissue. Up until this point however, there was no precise manner to validate specific dose to target from a volumetric perspective that could be trusted.
This changed with the development of RT digital planning tools. The Imaged Guided Therapy Quality Assurance (QA) Center became the leaders in this field and their effort and expertise brought RT clinical trials to a new level of performance (29). Prostate cancer became one of the first disease areas to be managed through this mechanism. The process of radiation treatment simulation for patients pivoted on this point and permanently changed for both the radiation clinic and clinical trials. Targets were drawn, slice by slice, on a planning computer tomography study. Normal tissues were contoured, and volumes of tumor and normal tissue were constructed. Radiation planning was now conducted through dose volumes and written directives were developed to specify dose to a volume. Constraints to normal tissue were applied to limit dose to a specified volume of normal tissue. Inverse planning tools for intensity modulation followed and fluence profiles within the beam could be modified to optimize both dose to tumor targets and limit dose to normal tissues. It was assumed, at this phase of development, that the imaging acquired for RT treatment planning was sufficient to determine protocol compliance. This major development permanently altered the approach to planning patients for RT as well as the management of clinical trials.
Jim Purdy and Jim Deye were essential contributors to the development of these tools, and both were kind to share these tools with QA centers. The cooperative groups associated with QARC, especially within the pediatric groups, had strong imaging committees and the RT committees had strong interest in the application of imaging to RT treatment planning. Although tools for image fusion of imaging objects into RT treatment plans had not yet matured at this time, images were collected at QARC for both assessment of response and validation of the targets chosen for RT. This became of particular importance in the early management of clinical protocols evaluating treatment strategies for Hodgkin lymphoma and have grown of increasing importance today as most clinical trials are now dependent on modern imaging to response assessment and outcome evaluation (30–43). In a series of early clinical trials involving patients of intermediate and high-risk disease, protocols were designed to test the utility of post chemotherapy RT including response adaptation of therapy in the mediastinum. In the Pediatric Oncology Group clinical trial 8725, imaging and RT objects were acquired as part of the data management process for the conduct of the study and reviewed at study closure. RT post-chemotherapy was the randomization point. For the entire study population, there appeared to be no benefit to the addition of RT to this patient population. However, because imaging and RT objects were onsite for secondary analysis, review of the data revealed that if patients received RT in a protocol compliant manner, there was a statistically significant improvement in survival and disease-free survival. The protocol had specific areas to target, and all disease sites noted at presentation had to be treated to protocol dose with appropriate dose titration to critical normal tissue structures including cardiac, pulmonary, and renal volumes. This implied that the quality of the application of RT had direct impact on patient and study outcomes and that images played an essential role in defining targets and assessing response. The challenge became moving this peer review process away from retrospective management and apply peer review in a more dedicated real time pre-therapy format in order to make certain trials were conducted in a protocol compliant manner. Pre-therapy review of imaging and RT objects in medulloblastoma clinical trial Children’s Oncology Group (COG) 9961 provided an unanticipated advantage of identifying patients who were not protocol eligible and assigned them to an appropriate study as those assigned to COG 9961 who were not eligible due to a higher stage and tumor burden had a statistically significant decrease in survival (44–50).
Pre-review of RT treatment objects was initially accomplished in early stage and intermediate stage Hodgkin lymphoma protocols COG 9425 and 9426. These studies included a component of response-adaptive therapy titration based on response to chemotherapy prior to the initiation of RT. Compliance to RT was significantly improved with the use of pre-review of objects, however, interestingly, there was a 50% discrepancy between central and site review of response to chemotherapy, therefore the need to review imaging objects for response as part of protocol management became the next process improvement in clinical trial management (45, 46).
The increasing use of digital data transfer permitted images to be acquired and reviewed by QA teams in a real time same-day format. This is important for trial management as it provided common ground review by site and study investigators as images could be reviewed by all involved in a simultaneous format and conflicts could be addressed and resolved before therapy could be applied. This was and remains important for adaptive studies. These studies are more complex to execute, and consistent interpretation of response ensures confidence in study conduct and outcome. Today, this is standard practice and cases from anywhere in the world can be reviewed on a same day basis for protocol management (45, 46, 51, 52).
Simultaneous real time review of imaging and RT objects as part of protocol management was accomplished in intermediate risk Hodgkin lymphoma study COG AHOD0031. This study was designed with an adaptive strategy with therapy titration including exclusion of RT in selected patients with initial rapid early response to chemotherapy and complete imaging response as defined by the study criteria. There was also a therapy augmentation component for patients who did not experience a rapid early response to initial chemotherapy. Therefore, consistent interpretation of response to therapy and application of RT treatment objects was essential for this trial which accrued nearly 1,800 patients. Review of anatomic and metabolic imaging by investigators was accomplished at several time points for each patient on study including at study entry for eligibility and after each two cycles of chemotherapy. RT treatment objects were reviewed pre therapy and outcome imaging was acquired per protocol. Outcome imaging has been an important component for this study and has generated many important publications including review of response to therapy for pleural effusions and bone. The study demonstrated that imaging datasets and RT objects could be managed and reviewed in a real time format in a manner identical to management of patients in a modern radiation oncology department (47, 48).
Because of the success in real time data management for clinical trials, this approach has been applied to most NCTN and other sponsored clinical trials and has been used in highly complex formats. For advanced stage Hodgkin lymphoma patients, RT is applied to areas of incomplete response or residual areas of disease measuring greater than 2.5 cm. For exceptionally young patients with minimal tumor burden removed by surgery, protocols include careful observation without additional therapy. Therefore, interpretation of imaging by study and site investigators is exceptionally important to ensure protocol objectives are met and the correct intended sites receive RT (51, 52).
Process improvements in RT with technology advancement, dose computation, image fusion, and clinical validation have been re-purposed and directly applied into NCTN and other sponsored RT clinical trials. In the next section, we will review how these improvements are applied in the daily workflow for the Imaging and Radiation Oncology Core (IROC). IROC serves as the imaging and radiation oncology core service for data acquisition and data management for the NCTN and other sponsored clinical trials. IROC credentials institutions and investigators for clinical trials and provides knowledge datasets to ensure that the sites can transfer the correct information to IROC. The knowledge tests often include contouring of tumor and normal tissues with generation of a dose plan in a protocol compliant manner.
3 QA process
As radiation oncology has matured with improved RT technology and image integration, the processes have matured and have been incorporated into the QA workflow. The segments of the program include site qualification, clinical trial design with support, site and individual credentialing, clinical trial management, case review, and support for trial analysis. The process is harmonized between multiple offices nationwide under a single integrated grant called the IROC. The office at IROC Houston manages site qualification and credentialing while the offices at IROC Rhode Island and IROC Philadelphia provide protocol case data acquisition and management with shared resources as needed for clinical trial support. The integrated informatics infrastructure has been provided by the American College of Radiology (ACR) and is called TRIAD (53).
4 Site qualification
Institutions and departments of radiation oncology can achieve recognition for being credentialed through multiple venues including but not limited to the American College of Radiology (ACR). Through a not dissimilar process, sites qualify for participation in clinical trials based on personnel, equipment, and the ability to apply technology according to the specific study. Generally, for protocol management, qualification includes completion of a questionnaire and validation of beam output for each unit treating clinical trial patients. This service verifies that a site has the basic resources and abilities to participate in NCI supported clinical trials. International sites must meet the same qualification requirements as North American sites. As of 2018, site qualification services are provided to 1,837 centers participating in clinical trials (including 140 international centers in 25 countries). The facility questionnaire is an electronic web-based form that is linked to the IROC database. This form collects information on the site’s demographics, staffing, treatment planning and delivery capabilities, and QA procedures. Since the facility questionnaire is linked to the IROC RT facility database, previous information regarding the site is filled in, requiring the site to simply add or modify existing information. Sites are required to review and edit their facility questionnaire annually. The questionnaire also collects information on RT imaging capabilities and supplies current information to the IROC roster used by the NCI Clinical Trial Support Unit (CTSU) and the Cancer Therapeutics Evaluation Program (CTEP). On an annual basis, all megavoltage photon beams, proton beams, and a selection of electron beams at every participating RT site have their reference beam output calibration measured remotely through a mailed TLD/optically stimulated luminescence dosimetry (OSLD) program to verify that beam output is within ±5%. High dose rate (HDR) (192Ir) brachytherapy and small field output factor OSLD/TLD remote audit tools have been developed and are in the initial stages of use (54). This output verification program is notable for its simplicity and would be similar to onsite QA measures performed by institutions as part of their internal program. When the TLD/OSLD measurement disagrees with an institution’s stated dose by more than 5%, IROC resolves this discrepancy through communication and procedural reviews. If those are unsuccessful, an on-site dosimetry visit may be performed.
Prior to a proton site enrolling patients on trials, they must be approved by IROC. In addition to the questionnaire and reference beam output verification, this process also includes successful completion of electronic transfer of proton treatment plans, irradiation of IROC’s baseline proton phantoms, and an on-site dosimetry review visit. Each proton delivery technique (passive scatter, intensity modulated proton therapy-(IMPT), etc.) used must be individually reviewed and approved.
5 Clinical trials support
Modern RT departments share or individually house the ability to generate clinical trials within the department or a trial generated by the cancer program involving RT. IROC offers expertise to help NCTN Groups develop new protocols, focusing on those sections relating to RT delivery, QA, and data collection. In a manner similar to an in-house data management group and Institutional Review Board (IRB), IROC reviews proposals at initial discussions including concept development through finalization of the protocol with efforts completed in a timely manner to not impede trial development. This early interaction is critical when new technologies or novel treatment techniques (e.g., developing QA strategies for radiopharmaceutical trials) are being introduced into trials. Because concept proposals typically lack detailed RT and imaging information, IROC provides a questionnaire to the concept Principal Investigator to gather the necessary information and facilitate further interaction as required. This in turn can help improve onsite investigator institutional trials mature with consistent treatment guidelines compliant to CTEP and support the conduct of the trial. After concept approval, IROC assists in the entire protocol development process. IROC contributes to developing and providing protocol RT section templates aimed to ensure quality trial data by standardizing structure name nomenclature, definition of dose volume analysis parameters, and radiotherapy processes in general. IROC’s protocol support includes but is not limited to 1) imaging procedures for target and organ at risk (OAR) definition, 2) dose prescription, 3) protocol compliance conditions, 4) RT treatment planning instructions; 5) QA procedures and their implementation, 6) image-guided RT (IGRT), and 7) the data submission process. By avoiding any unclear or ambiguous wording in the protocol and utilizing a consistent format, we ensure ease of understanding and implementation of the protocol.
6 Credentialing
Credentialing is the process of verifying that a specific site and/or clinician/physicist have the knowledge, resources, and capability to meet the protocol specifications. This process is analogous to independent peer review, and thus provides a basis for confidence in the institution and processes imbedded in the institution. Credentialing is distinct from ongoing periodic QA activities required for certification. Credentialing is designed and implemented through multiple pathways depending on the trial requirements. Credentialing may verify treatment planning, dose distributions, structure contouring, and/or image guidance. Credentialing tests may be systematic tests that assess the capabilities of the site or test the knowledge or skill of the investigator or planner. The success of credentialing is clear in that major deviation rates have decreased dramatically since credentialing has been required by protocols. While failure to meet the criteria may prevent a site’s participation in a specific protocol, the goal of IROC is not to restrict participation but to assist sites in any required remedial actions so that they meet the protocol criteria. Credentialing requirements are typically made generalizable so that if an institution successfully completes a credentialing requirement for one protocol, they do not need to repeat it for a subsequent protocol unless the new trial has a novel component. IROC has well established guidelines on when new credentialing is required and when an institution or investigator can be grandfathered through credentialing. To simplify the credentialing process, the presentation of protocol credentialing requirements has been condensed to a single table with links to the details of each requirement and implemented across all Groups. Additionally, an automated credentialing status inquiry system has been developed that maximizes clarity and ease of institutional inquiries and also provides credentials electronically to the CTSU’s Regulatory Support System (RSS) portal. Credentialing requirements may include completion of knowledge assessment/benchmark case or phantom irradiation depending on the specific protocol but are responsive to evolving protocol requirements including demonstrating successful site transfer of data to the QA centers. A meaningful minority of institutions still fail to meet the relatively lenient acceptability criteria. Importantly, these failures appear to be related to systematic problems at the institution, indicating that these errors will also affect protocol patients and potentially study outcomes. This highlights the importance of resolving any issues with the institution, and also highlights the value of this independent QA to supplement an institution’s internal QA program. Image guidance credentialing ensures that sites have an image guidance process that allows implementation of an appropriate imaging technique/image registration algorithm for the protocol and disease site. To minimize the burden on participating institutions, IROC has streamlined the IGRT credentialing to be based generically on bony registration or soft tissue registration. Institutions need therefore only pass two credentialing tests to be IGRT credentialed for all protocols (54–59).
7 Clinical trial management
The initial step in the QA evaluation process is review of the integrity of the submitted data (pre-review data management). This includes verifying that the institution has submitted accurate and complete protocol patient data to IROC. Incomplete and/or inaccurate files require additional IROC efforts and communication. To optimize the process, IROC has continual ongoing efforts to automate the QA of the data submission process. Clinical trials require that patients be treated as specified by the protocol. The purpose of case review is to verify that the sites planned and delivered the RT as required. A case review evaluates technical factors such as the dose distribution and fractionation and clinical factors such as the prescription, diagnostic imaging, tumor target/OAR contouring, and field placement. The case review service has been established so that NCI Groups have dedicated contacts within IROC for both imaging and RT. Standardizations for structure name nomenclature and dose volume parameter definitions are fully compliant with the American Association of Physicists in Medicine (AAPM) recommendations and templates for dose volume evaluation for different systems are published for each trial to aid institutions’ compliance. IROC has greatly increased the efficiency for dose volume analysis by creating the mechanism for automation of analysis and data upload. For institutional use, this would optimize individual case planning. For evaluation of submitted cases to IROC, scripts are developed for each trial to extract dose-volume histogram (DVH) data points and automatically format the data for direct upload to Medidata Rave (Rave), the NCI data management system. Institutions function in a similar manner with templates used for specification of image guidance and RT dose volume constraints. IROC has also implemented and developed knowledge-based tools and models to assess plan quality which can be re-purposed for institutional use. This has been done by evaluating and implementing knowledge-based systems/tools to assess the quality of RT treatment plans. These are commonly used for NCTN, and other sponsored trials (60).
There are three types of case review during the active period of the protocol: Pre, On, and Post-treatment review. These differ in the time when they are initiated relative to the start of patient treatment. All reviews evaluate the cases for protocol compliance. IROC will evaluate the effectiveness of case reviews using the large amount of information relating to institution performance in complying with protocols. Pretreatment review occurs prior to the start of treatment so cases can be modified to meet protocol objectives. The challenge of this review is that it requires coordinated timing to have the case reviewed promptly by IROC to avoid delay of the start of the patient’s treatment. Pre-treatment reviews also provide an interactive forum between study and site investigators. This process helps to determine if protocol amendments are needed. For institutions, this process resembles goals and objectives achieved in chart rounds and peer review. The on-treatment review is performed within seven days from the start of RT and results are communicated to the site for the purpose of improving overall treatment. Post treatment review records the details of the treatment given (including changes that are made after the initial review). The process evaluates technical aspects as well as verification that the RT data elements captured are accurate. Upon completion of case accrual for a trial, or for interim analysis, IROC provides information as requested by the NCTN Group for analysis. This process is similar to processes performed by institutions for quality metrics or publication.
8 Data management
IROC is responsible for holding the Digital Imaging and Communications in Medicine (DICOM) imaging and RT data for all NCTN Group clinical trials that use RT for treatment. These datasets are cross-referenced to the case data entered into the NCI information system RAVE. These data are used for protocol endpoint analysis including tools to obtain crucial data points from dose volume submissions not included in the initial design of the protocol. IROC is the custodian of the data on behalf of the NCI Groups, and use of the data will be determined by the NCI and the NCTN data resource sharing plan for each group. IROC can support data analyses by 1) gathering, packaging, and forwarding de-identified data to the requesting organization or group meeting 2) providing analysis relating to dose distributions and structure contouring for secondary analysis requests 3) providing services relating to tumor response and critical structure complications modeling and 4) provide data to support reviews not anticipated at the time of study initiation. IROC may also use these data directly for the purpose of quality improvement and effectiveness research (14, 15, 54–69).
9 Conclusion
The RT QA processes and procedures offered by IROC are diverse, dynamic and subject to continuous review and adaptation, similar to process improvements seen in daily department function. IROC endeavors to ensure that the QA services are as efficient and effective as possible in order to reasonably handle the clinical trial workload from NCI Group studies and industry trials. The systems developed, data collected, and services provided by IROC offer opportunities for collaborations with the NCI Groups, both nationally and internationally. As a service organization, IROC achieves the highest quality patient data allowing the NCI to meet its clinical trial goals. This is important for primary trial analysis as well as analysis of both secondary and tertiary trial objectives. A complete and accurate dataset increases the likelihood the outcome can be trusted and moved into clinical practice. Processes in the QA of clinical trials can be re-purposed by individual institutions through real time peer review and ensure compliance to standard between colleagues. Many tools exist through multiple vendors which can display image objects and radiation targets during a chart rounds session which can mimic the process of real time review for clinical trial activity. There are multiple checks for the computational component of our work and chart completeness. Coupling this effort with physician peer review will harmonize activity within a department and bring the physicians closer to the departmental QA process and into compliance standards for regulatory review. Research platforms can be built through this prism.
Each patient treated with RT can be part of a clinical trial. The trials can be investigator driven with endpoints following many pathways including outcome analysis, normal tissue tolerance, patient support, physics, and nursing. To do so at an enterprise manner would require infrastructure similar to The Cancer Imaging Archive (TCIA) imbedded into daily department activities and workflow with transfer of onsite objects to TCIA as seen appropriate by the TCIA steering group. The information technology infrastructure of TCIA is PRISM and this platform houses patient objects including pathology/pathomics, radiology, RT treatment objects, medical oncology information, and relevant clinical information. All of this information can be re-purposed for modern projects including but not limited to artificial intelligence. Many departments now house TCIA informatics infrastructure as part of internal data management services to use on a daily basis for translational research. As data including pathology evolves in digital format, research can be completed with nimble query tools to help move our translational science forward (66–77).
The future of our discipline is bright. We have to be disciplined in our science place emphasis on the quality of our data. If we do so, our work will be well perceived and colleagues can trust our results.
Author contributions
All authors listed have made a substantial, direct, and intellectual contribution to the work and approved it for publication.
Funding
The work done at QARC and within the IROC program were supported in part by U10CA029511, U24CA180803 and U24CA215109.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
1. Munro AJ. Hidden danger, obvious opportunity: Error and risk in the management of cancer. Br J Radiol (2007) 80:955–66. doi: 10.1259/bjr/12777683
2. Bogdanich W. Radiation offers new cures, and ways to do harm. In: The new York times: The radiation room (New York, NY: The New York Times Company) (2010).
3. Folio LR, Nelson CJ, Benjamin M, Ran A, Engelhard G, Bluemke DA. Quantitative radiology reporting in oncology: Survey of oncologists and radiologists. AJR Am J Roentgenol (2015) 205:W233–43. doi: 10.2214/AJR.14.14054
4. Peters LJ, O’Sullivan B, Giralt J, Fitzgerald TJ, Trotti A, Bernier J, et al. Critical impact of radiotherapy protocol compliance and quality in the treatment of advanced head and neck cancer: Results from TROG 02.02. J Clin Oncol (2010) 28:2996–3001. doi: 10.1200/JCO.2009.27.4498
5. Weber DC, Tomsej M, Melidis C, Hurkmans CW. QA makes a clinical trial stronger: Evidence-based medicine in radiation therapy. Radiother Oncol (2012) 105(1):4–8. doi: 10.1016/j.radonc.2012.08.008
6. Ohri N, Shen X, Dicker AP, Doyle LA, Harrison AS, Showalter TN. Radiotherapy protocol deviations and clinical outcomes: A meta-analysis of cooperative group clinical trials. J Natl Cancer Inst (2013) 105(6):387–93. doi: 10.1093/jnci/djt001
7. Fairchild A, Straube W, Laurie F, Followill D. Does quality of radiation therapy predict outcomes of multicenter cooperative group trials? a literature review. Int J Radiat Oncol Biol Phys (2013) 87(2):246–60. doi: 10.1016/j.ijrobp.2013.03.036
8. Barthelemy-Brichant N, Sabatier J, Dewé W, Albert A, Deneufbourg JM. Evaluation of frequency and type of errors detected by a computerized record and verify system during radiation treatment. Radiother Oncol (1999) 53(2):149–54. doi: 10.1016/S0167-8140(99)00141-3
9. McGill AR. Book review: Normal accidents: Living with high-risk technologies. In: Perrow C, editor. Basic books, vol. 23 (1984). p. 434–6.
10. Macklis RM, Meier T, Weinhous MS. Error rates in clinical radiotherapy. J Clin Oncol (1998) 16(2):551–6. doi: 10.1200/JCO.1998.16.2.551
11. Huang G, Medlam G, Lee J, Billingsley S, Bissonnette JP, Ringash J, et al. Error in the delivery of radiation therapy: Results of a quality assurance review. Int J Radiat Oncol Biol Phys (2005) 61(5):1590–5. doi: 10.1016/j.ijrobp.2004.10.017
12. Pettersen MN, Aird E, Olsen DR. Quality assurance of dosimetry and the impact on sample size in randomized clinical trials. Radiother Oncol (2008) 86(2):195–9. doi: 10.1016/j.radonc.2007.07.001
13. Bentzen SM, Bernier J, Davis JB, Horiot JC, Garavaglia G, Chavaudra J, et al. Clinical impact of dosimetry quality assurance programmes assessed by radiobiological modelling of data from the thermoluminescent dosimetry study of the European organization for research and treatment of cancer. Eur J Cancer. (2000) 36(5):615–20. doi: 10.1016/S0959-8049(99)00336-6
14. Younge KC, Marsh RB, Owen D, Geng H, Xiao Y, Spratt DE, et al. Improving quality and consistency in NRG oncology radiation therapy oncology group 0631 for spine radiosurgery via knowledge-based planning. Int J Radiat Oncol Biol Phys (2018) 100(4):1067–74. doi: 10.1016/j.ijrobp.2017.12.276
15. Zhong H, Wang J, Soest JV, Geng H, Huang M, Cheng C, et al. The evidence driven dosimetric constraints from outcome analysis of h & n patients’ data from NRG oncology RTOG 0522 trial. Int J Radiat Oncol Biol Phys (2017) 99(2):S137. doi: 10.1016/j.ijrobp.2017.06.275
16. Boyer AL, Schultheiss T. Effects of dosimetric and clinical uncertainty on complication-free local tumor control. Radiother Oncol (1988) 11(1):65–71. doi: 10.1016/0167-8140(88)90046-1
17. Kry SF, Dromgoole L, Alvarez P, Leif J, Molineu A, Taylor P, et al. Radiation therapy deficiencies identified during on-site dosimetry visits by the imaging and radiation oncology core Houston quality assurance center. Int J Radiat Oncol Biol Phys (2017) 99(5):1094–100. doi: 10.1016/j.ijrobp.2017.08.013
18. Fitzgerald TJ, Arvin S. Glicksman, MD 1924 to 2020. Pract Radiat Oncol (2020) 10 5:301–3. doi: 10.1016/j.prro.2020.05.003
19. Alvarez P, Kry SF, Stingo F, Followill D. TLD and OSLD dosimetry systems for remote audits of radiotherapy external beam calibration. Radiat Meas. (2017) 106:412–5. doi: 10.1016/j.radmeas.2017.01.005
20. Followill DS, Urie M, Galvin JM, Ulin K, Xiao Y, Fitzgerald TJ. Credentialing for participation in clinical trials. Front Oncol (2012) 2:198. doi: 10.3389/fonc.2012.00198
21. Molineu A, Hernandez N, Nguyen T, Ibbott G, Followill D. Credentialing results from IMRT irradiations of an anthropomorphic head and neck phantom. Med Phys (2013) 40(2):022101. doi: 10.1118/1.4773309
22. Ibbott GS, Haworth A, Followill DS. Quality assurance for clinical trials. Front Oncol (2013) 3:311. doi: 10.3389/fonc.2013.00311
23. Leif J, Roll J, Followill D, Ibbott G. The value of credentialing. Int J Radiat Oncol Biol Phys (2006) 66(3):S716. doi: 10.1016/j.ijrobp.2006.07.1312
24. Leif J, Roll J, Davis C, Ibbott G. Rapid reviews reduce the rate of protocol deviations. Int J Radiat Oncol Biol Phys (2009) 75(3):S617. doi: 10.1016/j.ijrobp.2009.07.1411
25. Ibbott GS, Hanson WF, O’Meara E, Kuske RR, Arthur D, Rabinovitch R, et al. Dose specification and quality assurance of radiation therapy oncology group protocol 95-17; a cooperative group study of iridium-192 breast implants as sole therapy. Int J Radiat Oncol Biol Phys (2007) 69(5):1572–8. doi: 10.1016/j.ijrobp.2007.08.011
26. Kerns JR, Followill DS, Lowenstein J, Molineu A, Alvarez P, Taylor PA, et al. Agreement between institutional measurements and treatment planning system calculations for basic dosimetric parameters as measured by the imaging and radiation oncology core-Houston. Int J Radiat Oncol Biol Phys (2016) 95(5):1527–34. doi: 10.1016/j.ijrobp.2016.03.035
27. Ibbott GS, Followill DS, Molineu HA, Lowenstein JR, Alvarez PE, Roll JE. Challenges in credentialing institutions and participants in advanced technology multi-institutional clinical trials. Int J Radiat Oncol Biol Phys (2008) 71(1S):S71–75. doi: 10.1016/j.ijrobp.2007.08.083
28. Taylor PA, Kry SF, Alvarez P, Keith T, Lujano C, Hernandez N, et al. Results from the imaging and radiation oncology core houston’s anthropomorphic phantoms used for proton therapy clinical trial credentialing. Int J Radiat Oncol Biol Phys (2016) 95(1):242–8. doi: 10.1016/j.ijrobp.2016.01.061
29. Purdy JA. Quality assurance issues in conducting multi-institutional advanced technology clinical trials. Int J Radiat Oncol Biol Phys (2008) 71(1S):S66–70. doi: 10.1016/j.ijrobp.2007.07.2393
30. Ellingson BM, Kim E, Woodworth DC, Marques H, Boxerman JL, Safriel Y, et al. Diffusion MRI quality control and functional diffusion map results in ACRIN 6677/RTOG 0625: A multicenter, randomized, phase II trial of bevacizumab and chemotherapy in recurrent glioblastoma. Int J Oncol (2015) 46(5):1883–92. doi: 10.3892/ijo.2015.2891
31. Kurland BF, Muzi M, Peterson LM, Doot RK, Wangerin KA, Mankoff DA, et al. Multicenter clinical trials using 18F-FDG PET to measure early response to oncologic therapy: Effects of injection-to-acquisition time variability on required sample size. J Nucl Med (2016) 57(2):226–30. doi: 10.2967/jnumed.115.162289
32. Levy MA, Rubin DL. Tool support to enable evaluation of the clinical response to treatment. AMIA Annu Symp Proc (2008) 2008:399–403.
33. Press OW, Li H, Schöder H, Straus DJ, Moskowitz CH, LeBlanc M, et al. US Intergroup trial of response-adapted therapy for stage III to IV Hodgkin lymphoma using early interim fluorodeoxyglucose-positron emission tomography imaging: Southwest oncology group S0816. J Clin Oncol (2016) 34(17):2020–7. doi: 10.1200/JCO.2015.63.1119
34. Cui Y, Galvin JM, Straube WL, Bosch WR, Purdy JA, Li XA, et al. Multi-system verification of registrations for image-guided radiotherapy in clinical trials. Int J Radiat Oncol Biol Phys (2011) 81(1):305–12. doi: 10.1016/j.ijrobp.2010.11.019
35. Buckler AJ, Boellaard R. Standardization of quantitative imaging: The time is right, and 18F-FDG PET/CT is a good place to start. J Nucl Med (2011) 52(2):171–2. doi: 10.2967/jnumed.110.081224
36. Curran S, Muellner A, Schwartz LH. Imaging response assessment in oncology. Cancer Imaging. (2006) 6:S126–S130. doi: 10.1102/1470-7330.2006.9039
37. Saltz J, Almeida J, Gao Y, Sharma A, Bremer E, DiPrima T, et al. Towards generation, management, and exploration of combined radiomics and pathomics datasets for cancer research. AMIA JT Summits Transl Sci Proc (2017) 2017:85–94.
38. Gillies RJ, Kinahan PE, Hricak H. Radiomics: Images are more than pictures, they are data. Radiology. (2016) 278(2):563–77. doi: 10.1148/radiol.2015151169
39. Wolchok JD, Hoos A, O’Day S, Weber JS, Hamid O, Lebbé C, et al. Guidelines for the evaluation of immune therapy activity in solid tumors: Immune-related response criteria. Clin Cancer Res (2009) 15(23):7412–20. doi: 10.1158/1078-0432.CCR-09-1624
40. Seymour L, Bogaerts J, Perrone A, Ford R, Schwartz LH, Mandrekar S, et al. iRECIST: Guidelines for response criteria for use in trials testing immunotherapeutics. Lancet Oncol (2017) 18(3):e143–52. doi: 10.1016/S1470-2045(17)30074-8
41. Fahey FH, Kinahan PE, Doot RK, Kocak M, Thurston H, Poussaint TY. Variability in PET quantitation within a multicenter consortium. Med Phys (2010) 37(7):3660–6. doi: 10.1118/1.3455705
42. Scheuermann JS, Reddin JS, Opanowski A, Kinahan PE, Siegel BA, Shankar LK, et al. Qualification of national cancer institute-designated cancer centers for quantitative PET/CT imaging in clinical trials. J Nucl Med (2017) 58(7):1065–71. doi: 10.2967/jnumed.116.186759
43. Smilowitz JB, Das IJ, Feygelman V, Fraass BA, Kry SF, Marshall IR, et al. AAPM medical physics practice guideline 5.A.: Commissioning and QA of treatment planning dose calculations - megavoltage photon and electron beams. J Appl Clin Med Phys (2015) 16(5):14–34. doi: 10.1120/jacmp.v16i5.5768
44. Weiner MA, Leventhal B, Brecher ML, Marcus RB, Cantor A, Gieser PW, et al. Randomized study of intensive MOPP-ABVD with or without low-dose total-nodal radiation therapy in the treatment of stages IIb, IIIa2, IIIb, and IV hodgkin’s disease in pediatric patients: A pediatric oncology group study. J Clin Oncol (1997) 15(8):2769–79. doi: 10.1200/JCO.1997.15.8.2769
45. FitzGerald TJ. What we have learned: The impact of quality from a clinical trials perspective. Semin Radiat Oncol (2012) 22(1):18–28. doi: 10.1016/j.semradonc.2011.09.004
46. FitzGerald TJ, Urie M, Ulin K, Laurie F, Yorty J, Hanusik R, et al. Processes for quality improvements in radiation oncology clinical trials. Int J Radiat Oncol Biol Phys (2008) 71(1S):S76–9. doi: 10.1016/j.ijrobp.2007.07.2387
47. Friedman DL, Chen L, Wolden S, Buxton A, McCarten K, FitzGerald TJ, et al. Dose-intensive response-based chemotherapy and radiation therapy for children and adolescents with newly diagnosed intermediate-risk Hodgkin lymphoma: A report from the children’s oncology group study AHOD0031. J Clin Oncol (2014) 32(32):3651–8. doi: 10.1200/JCO.2013.52.5410
48. Dharmarajan KV, Friedman DL, FitzGerald TJ, McCarten KM, Constine LS, Chen L, et al. Radiotherapy quality assurance report from children’s oncology group AHOD0031. Int J Radiat Oncol Biol Phys (2015) 91(5):1065–71. doi: 10.1016/j.ijrobp.2014.11.034
49. Parzuchowski A, Bush R, Pei Q, Friedman DL, FitzGerald TJ, Wolden SL, et al. Patterns of involved-field radiation therapy protocol deviations in pediatric versus adolescent and young adults with Hodgkin lymphoma: A report from the children’s oncology group AHOD0031. Int J Radiat Oncol Biol Phys (2018) 100(5):1119–25. doi: 10.1016/j.ijrobp.2018.01.002
50. Michalski JM, Janss A, Vezina G, Gajjar A, Pollack I, Merchant TE, et al. Results of COG ACNS0331: A phase III trial of involved-field radiotherapy (IFRT) and low dose craniospinal irradiation (LD-CSI) with chemotherapy in average-risk medulloblastoma: A report from the children’s oncology group. Int J Radiat Oncol Biol Phys (2016) 96(5):937–8. doi: 10.1016/j.ijrobp.2016.09.046
51. Rosen M, Kinahan PE, Gimpel JF, Opanowski A, Siegel BA, Hill GC, et al. Performance observations of scanner qualification of NCI-designated cancer centers: Results from the centers of quantitative imaging excellence (CQIE) program. Acad Radiol (2017) 24(2):232–45. doi: 10.1016/j.acra.2016.09.025
52. Xiao Y, Rosen M. The role of imaging and radiation oncology core for precision medicine era of clinical trial. Transl Lung Cancer Res (2017) 6(6):621–4. doi: 10.21037/tlcr.2017.09.06
53. Giaddui T, Yu J, Manfredi D, Linnemann N, Hunter J, O’Meara E, et al. Structures’ validation profiles in transmission of imaging and data (TRIAD) for automated national clinical trials network (NCTN) clinical trial digital data quality assurance. Pract Radiat Oncol (2016) 6(5):331–3. doi: 10.1016/j.prro.2016.01.007
54. Casey KE, Alvarez P, Kry SF, Howell RM, Lawyer A, Followill D. Development and implementation of a remote audit tool for high dose rate (HDR) ir-192 brachytherapy using optically stimulated luminescence dosimetry. Med Phys (2013) 40(11):112102. doi: 10.1118/1.4824915
55. Aguirre J, Alvarez P, Amador C, Tailor A, Followill D, Ibbott G. We-D-BRB-08: Validation of the commissioning of an optically stimulated luminescence (OSL) system for remote dosimetry audits. Med Phys (2010) 37(6):3428. doi: 10.1118/1.3469396
56. Ulin K, Urie MM, Cherlow JM. Results of a multi-institutional benchmark test for cranial CT/MR image registration. Int J Radiat Oncol Biol Phys (2010) 77(5):1584–9. doi: 10.1016/j.ijrobp.2009.10.017
57. Urie M, FitzGerald TJ, Followill D, Laurie F, Marcus R, Michalski J. Current calibration, treatment, and treatment planning techniques among institutions participating in the children’s oncology group. Int J Radiat Oncol Biol Phys (2003) 55(1):245–60. doi: 10.1016/S0360-3016(02)03827-0
58. Carson ME, Molineu A, Taylor PA, Followill DS, Stingo FC, Kry SF. Examining credentialing criteria and poor performance indicators for IROC houston’s anthropomorphic head and neck phantom. Med Phys (2016) 43(12):6491. doi: 10.1118/1.4967344
59. Kerns JR, Stingo F, Followill DS, Howell RM, Melancon A, Kry SF. Treatment planning system calculation errors are present in most imaging and radiation oncology core-Houston phantom failures. Int J Radiat Oncol Biol Phys (2017) 98(5):1197–203. doi: 10.1016/j.ijrobp.2017.03.049
60. Mayo CS, Moran JM, Bosch W, Xiao Y, McNutt T, Popple R, et al. American Association of physicists in medicine task group 263: Standardizing nomenclatures in radiationoOncology. Int J Radiat Oncol Biol Phys (2018) 100(4):1057–66. doi: 10.1016/j.ijrobp.2017.12.013
61. Yu J, Straube W, Mayo C, Giaddui T, Bosch W, Ulin K, et al. Radiation therapy digital data submission process for national clinical trials network. Int J Radiat Oncol Biol Phys (2014) 90(2):466–7. doi: 10.1016/j.ijrobp.2014.05.2672
62. Giaddui T, Li N, Curry K, Moore K, Mell L, Leath C, et al. Su-F-T-351: Establishing a workflow for IMRT pre-treatment reviews for NRG-GY006 clinical trial. Med Phys (2016) 43(6):3544. doi: 10.1118/1.4956536
63. Followill DS, Kry SF, Qin L, Lowenstein J, Molineu A, Alvarez P, et al. The radiological physics center’s standard dataset for small field size output factors. J Appl Clin Med Phys (2012) 13(5):3962. doi: 10.1120/jacmp.v13i5.3962
64. Taylor PA, Kry SF, Followill DS. Pencil beam algorithms are unsuitable for proton dose calculations in lung. Int J Radiat Oncol Biol Phys (2017) 99(3):750–6. doi: 10.1016/j.ijrobp.2017.06.003
65. Kry SF, Molineu A, Kerns JR, Faught AM, Huang JY, Pulliam KB, et al. Institutional patient-specific IMRT QA does not predict unacceptable plan delivery. Int J Radiat Oncol Biol Phys (2014) 90(5):1195–201. doi: 10.1016/j.ijrobp.2014.08.334
66. Amador C, Keith T, Nguyen T, Molineu A, Followill D. Su-E-P-02: Imaging and radiation oncology core (IROC) Houston QA center (RPC) credentialing. Med Phys (2014) 41:127. doi: 10.1118/1.4887940
67. Followill DS, Molineu H, Lafratta RG, Ibbott GS. The IROC Houston quality assurance program: Potential benefits of 3D dosimetry. J Phys: Conf Series. (2017) 847. doi: 10.1088/1742-6596/847/1/012029
68. Prior F, Smith K, Sharma A, Kirby J, Tarbox L, Clark K, et al. The public cancer radiology imaging collections of the cancer imaging archive. Sci Data. (2017) 4:170124. doi: 10.1038/sdata.2017.124
69. Taylor RE, Donachie PH, Weston CL, Robinson KJ, Lucraft H, Saran F, et al. Impact of radiotherapy parameters on outcome for patients with supratentorial primitive neuro-ectodermal tumours entered into the SIOP/UKCCSG PNET 3 study. Radiother Oncol (2009) 92(1):83–8. doi: 10.1016/j.radonc.2009.02.017
70. Eich HT, Engenhart-Cabillic R, Hansemann K, Lukas P, Schneeweiss A, Seegenschmiedt H, et al. Quality control of involved field radiotherapy in patients with early-favorable (HD10) and early-unfavorable (HD11) hodgkin’s lymphoma: An analysis of the German Hodgkin study group. Int J Radiat Oncol Biol Phys (2008) 71(5):1419–24. doi: 10.1016/j.ijrobp.2007.12.002
71. Dixon P, O’Sullivan B. Radiotherapy quality assurance: Time for everyone to take it seriously. Eur J Cancer. (2003) 39(4):423–9. doi: 10.1016/S0959-8049(02)00744-X
72. Toita T, Kato S, Ishikura S, Tsujino K, Kodaira T, Uno T, et al. Radiotherapy quality assurance of the Japanese gynecologic oncology group study (JGOG1066): A cooperative phase II study of concurrent chemoradiotherapy for uterine cervical cancer. Int J Clin Oncol (2011) 16(4):379–86. doi: 10.1007/s10147-011-0196-4
73. Jullumstrø E, Wibe A, Lydersen S, Edna TH. Violation of treatment guidelines – hazard for rectal cancer patients. Int J Colorectal Dis (2012) 27(1):103–9. doi: 10.1007/s00384-011-1283-8
74. Gaze MN, Boterberg T, Dieckmann K, Hörmann M, Gains JE, Sullivan KP, et al. Results of a quality assurance review of external beam radiation therapy in the international society of paediatric oncology (Europe) neuroblastoma group’s high-risk neuroblastoma trial: A SIOPEN study. Int J Radiat Oncol Biol Phys (2013) 85(1):170–4. doi: 10.1016/j.ijrobp.2012.05.004
75. Gains JE, Stacey C, Rosenberg I, Mandeville HC, Chang YC, D’Souza D, et al. Intensity-modulated arc therapy to improve radiation dose delivery in the treatment of abdominal neuroblastoma. Future Oncol (2013) 9(3):439–49. doi: 10.2217/fon.12.199
76. Müller RP, Eich HT. The development of quality assurance programs for radiotherapy within the German Hodgkin study group (GHSG). introduction, continuing work, and results of the radiotherapy reference panel. Strahlenther Onkol. (2005) 181(9):557–66. doi: 10.1007/s00066-005-1437-0
Keywords: oncology, clinical trials, patient care, quality, radiation therapy
Citation: Smith K, Ulin K, Knopp M, Kry S, Xiao Y, Rosen M, Michalski J, Iandoli M, Laurie F, Quigley J, Reifler H, Santiago J, Briggs K, Kirby S, Schmitter K, Prior F, Saltz J, Sharma A, Bishop-Jodoin M, Moni J, Cicchetti MG and FitzGerald TJ (2023) Quality improvements in radiation oncology clinical trials. Front. Oncol. 13:1015596. doi: 10.3389/fonc.2023.1015596
Received: 09 August 2022; Accepted: 06 January 2023;
Published: 26 January 2023.
Edited by:
John C Roeske, Loyola University Medical Center, United StatesReviewed by:
Zhitao Dai, Chinese Academy of Medical Sciences and Peking Union Medical College, ChinaFrancesco Ricchetti, Sacro Cuore Don Calabria Hospital (IRCCS), Italy
Paul Harari, University of Wisconsin-Madison, United States
Copyright © 2023 Smith, Ulin, Knopp, Kry, Xiao, Rosen, Michalski, Iandoli, Laurie, Quigley, Reifler, Santiago, Briggs, Kirby, Schmitter, Prior, Saltz, Sharma, Bishop-Jodoin, Moni, Cicchetti and FitzGerald. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Thomas J. FitzGerald, VEouRml0ekdlcmFsZEB1bWFzc21lbW9yaWFsLm9yZw==