Core functions and forms of Bright IDEAS: A multi-methods evaluation of the adoption of an evidence-based psychosocial training program through iterative adaptation

Background: Despite efforts to widely disseminate interventions designed to increase access to quality supportive care to pediatric cancer patients and their families, many of these interventions fail to meet expectations once deployed in real-life clinical settings. This study identifies the functions and forms of Bright IDEAS: Problem-Solving Skills Training, an evidence based psychosocial intervention for caregivers of children recently diagnosed with cancer, to identify pragmatic program adaptations in its real-world clinical implementation. We compare intervention adoption before and after adaptations to the Bright IDEAS training program as part of a national training program designed to disseminate the intervention.

Methods: 209 pediatric psychosocial oncology practitioners representing 134 unique institutions were trained during 10 in-person 8-hour workshops (2015–2019). Functions and forms of Bright IDEAS were identified, and adaptations made to the training agenda and curriculum based on practitioner feedback following implementation in local institutions. Mixed method evaluation included longitudinal surveys at 6- and 12-months post training; and qualitative interviews among a subgroup of practitioners (N = 47) to understand and compare perspectives on intervention adoption and barriers to implementation before and after adaptations to the Bright IDEAS training program. The RE-AIM framework was used to guide dissemination evaluation.

Results: A total of four adaptations were tailored to the identified forms of the intervention: case studies; pre-training reading materials; training videos; and letters of institutional support from primary supervisor. Pre- and post-training adaptations to the Bright IDEAS training program were mapped to RE-AIM constructs. Quantitative findings demonstrate that adaptations appeared to improve adoption and usage overall.

Conclusion: This study provides insight into how contextual factors influence psychosocial practitioners' capacity to adopt, implement, and maintain Bright IDEAS in the clinical setting. This study demonstrates the use of real-time stakeholder feedback to guide intervention translation from research to practice settings.

Background

Bright IDEAS

Problem-Solving Skills Training (Bright IDEAS) is an evidence-based cognitive behavioral therapy that has over 25 years of empirical evidence demonstrating a decrease in negative affectivity (mood, depression, post-traumatic stress symptoms) in mothers of children with recently diagnosed cancer (1–4). Problem-solving therapy (PST) is a cognitive-behavioral approach developed to treat depression and anxiety in adults (5). The decision to call the intervention Problem-Solving Skills Training rather than therapy was aimed at making the intervention acceptable to distressed parents, who did not feel that they required “therapy.”

Multisite randomized controlled trials (RCTs) funded by the NIH/NCI over 25 years showed that learning the 5-step Bright IDEAS paradigm improves problem-solving skills and improved problem-solving skills led to decreased depression, improved mood, and fewer symptoms of posttraumatic stress (1, 4). Specifically, when compared to a nonspecific behavioral intervention, which provided the same time and attention from research assistants and focused on non-judgmental support and expression of feelings, participants of Bright IDEAS, at the 3-month follow up (T3) showed significant improvements in mood (−2.78 vs. −9.33, p ≤ 0.009), anxiety (−0.14 vs. −0.54, p ≤ 0.001), and post-traumatic stress (−2.27 vs. −4.01, p = 0.12) (6). Additionally, Bright IDEAS, when compared to control, had the greatest impact on improving constructive problem solving, accounting for 40% of the difference in mood scores between the two groups (1). In a two-arm randomized clinical trial of usual psychosocial care (UPC) as the control condition vs. UPC + Bright IDEAS as the intervention condition, mothers that received UPC + Bright IDEAS reported significantly enhanced problem-solving skills and significantly decreased negative affectivity (2).

The Bright IDEAS intervention is designed to empower individuals to manage adverse situations by using constructive coping strategies. It is a five-step cognitive-behavioral intervention based on the theoretical underpinnings of established problem-solving therapy (PST) (5, 7). Bright IDEAS represents a mnemonic (Figure 1). Bright signifies the concept of optimism (i.e., positive problem orientation), which is essential to successful problem-solving. The letters in the word “IDEAS” each stand for one of the five steps in the problem-solving process: I (Identify the problem), D (Define possible options), E (Evaluate your options—pros and cons of each option), A (Act—create an action plan based on D and E and do it), and S (See if it worked). If the plan did not work, review the options and devise a Plan B.

FIGURE 1

Figure 1. Bright IDEAS pneumonic.

No constraints are placed on the type of problem or challenge that Bright IDEAS can address. Of note, the majority of problems selected by the caregivers who participated in Bright IDEAS efficacy studies were not related to pediatric cancer (1–3). Optimum engagement is gained by focusing on problems the caregiver identifies as particularly relevant to him or her and walking through each of the steps. Clinical trials have shown that Bright IDEAS is acceptable to caregivers when taught in 6–8 and 30–60-min face-to-face sessions (1) and more effective over time than one of the most common forms of psychosocial support, non-directive supportive counseling (6).

The National Cancer Institute (NCI) designated Bright IDEAS an Evidence-Based Cancer Control Program (EBCCP; formerly, Research-Tested Intervention Program), in 2010. The EBCCP is a public-facing searchable database of evidence-based cancer control programs designed to provide program planners and health professionals easy and immediate access to research-tested materials. NCI tracking statistics indicate the Bright IDEAS webpage received 370 views on average over an 8-year period (averaging 2.7 min per page); and 40 CDs of the intervention materials were requested to be mailed. These findings suggest that, despite NCI endorsement, public availability, and considerable evidence supporting its efficacy, the leap from research protocol to standard clinical care was minimal.

There is increasing urgency to address the gap between the generation of new knowledge and empirical evidence, and its application to routine clinical care (8, 9). This urgency is fueled, in part, by the many interventions that fail to meet expectations once deployed in real-life clinical settings. Typically, interventions are tested under controlled conditions that are unlike the clinical practice settings in which they are deployed. As a result, many interventions lack full consideration of the local and contextual factors that ultimately affect intervention implementation (9, 10).

To overcome intervention failure in clinical settings, the field of dissemination and implementation (D&I) science has called attention to the importance of adapting existing evidence-based interventions to improve their fit in new contexts (11, 12). A critical first step in adaptation is to identify core functions (purposes) and forms (activities). Core functions represent the central purpose of the change processes that the health intervention seeks to facilitate. The forms of an intervention are the specific strategies or activities that can be customized to local contexts to carry out the core functions. Core functions should be considered unchallengeable as they are the essential mechanisms responsible for intervention efficacy. Adaptation at the form level, however, allows flexibility for organizations to tailor an intervention to their specific setting and situtaion (11, 12). Ideally, an intervention's core functions and forms align with health system and patient needs at the clinical level to ensure both the integrity of the intervention and its successful implementation.

Despite efforts to scale up cancer control interventions, there are limited data assessing the adoption of NCI-recognized survivorship and supportive care EBCCPs into clinical practice (13). Specifically, adaptation of Bright IDEAS had not been considered previously. This study examines the core functions and forms of Bright IDEAS and the impact of adaptations to the training program on its real-world clinical use based on feedback from participants in this natural experiment of an NCI-supported dissemination training grant. The goal of the grant was to increase national awareness of Bright IDEAS, train providers on how to deliver Bright IDEAS, and facilitate adoption amongst the approximately 200 pediatric oncology centers operating in North America. The multi-methods evaluation presented in this paper, involving both survey and qualitative assessment, seeks to elucidate barriers to adoption, implementation, and maintenance of a psychosocial intervention in diverse real-world pediatric oncology practice settings.

Methods

Training format

An NCI training grant (R25 CA65520) was awarded to train 200 pediatric psychosocial oncology professionals throughout the United States by conducting 10 interactive in-person training workshops between October 2015 and September 2019. Practitioner recruitment for the workshops was conducted in partnership with national professional organizations intimately involved in pediatric oncology: Children's Oncology Group (COG), Association of Pediatric Oncology Social Workers (APOSW), the Association of Pediatric Hematology/Oncology Nurses (APHON), and the Society of Pediatric Psychology (SPP).

The 1½-day training workshops were held in conjunction with association national meetings and endorsed through co-advertising. The workshops included summary information about the three large multi-site randomized controlled trials conducted to date demonstrating the efficacy of Bright IDEAS; role plays to observe and practice administration of the intervention; and in-depth discussions about implementation at an attendee's specific home institution. The original training agenda was modeled after the research training protocol used in clinical trials, which included a clinician's manual that detiled the basic approach and discrete steps of Bright IDEAS, a brief user's manual summarizing three steps, and worksheets (14). In the pre-adaptation phase we delivered the training curriculum based on the research protocol used in clinical trials. Post-adaptation phase we delivered the training curriculum that was informed by real-world clinical practice feedback from participants. Adaptations to the training program were made to mitigate perceived barriers to clinical application of Bright IDEAS in institutional settings. Workshop participants received up to $1,000 to reimburse expenses associated with travel and lodging.

Study population

All training participants (N = 209; pediatric oncology psychosocial professionals representing 134 unique institutions) were electronically surveyed at 6- and 12-months post workshop training. Survey response rate was 85.6% (n = 179) at 6 months and 72.2% (n = 149) at 12 months.

In addition, a subset of trainees were purposively sampled (N = 47, 24.4% of participants) and interviewed between January 2017 and March 2020. Interviews represented a range of post-training workshop experience before and after adaptations were made to the training workshop agenda: Pre-adaptation: Wave 1- more than 12 months since training (N = 11), Wave 2- between 6 and 12 months (N = 9), and Wave 3- less than 6 months (N = 10). Post-adaptation: Wave 1- more than 12 months since training (N = 6), Wave 2- between 6 and 12 months (N = 4), and Wave 3- less than 6 months (N = 4). The pre-adaptation group participated in the workshops that delivered the original training curriculum as used in the clinical research studies. The post-adaptation group participated in the workshops that delivered the adapted training curriculum designed to be more relevant for real-world clinical practice. We define the core intervention as the IDEAS psychosocial behavioral intervention pneumonic. The intervention materials were streamlined in their delivery, not in the content they conveyed.

Practitioners were contacted via email and invited to participate in a 30-minute telephone interview. In total, 106 professionals were contacted, 68 responded to the study invitation (64% response), and 44 were scheduled (65% participation) for interviewing allowing up to three contact attempts. A $25 gift card was offered for participation.

The project was approved by the Colorado Combined Institutional Review Board (COMIRB).

Evaluation methods

The RE-AIM framework, which is recognized by the NCI as a leading implementation framework in cancer control research, was used to guide the evaluation process (15). We intentionally focused on three of the five dimensions of RE-AIM given the clinical and translational stage of the Bright IDEAS program: adoption, implementation, and maintenance. We did not focus on reach in this study as the goal of the training grant was directed at providers (target adopters) of the intervention and dissemination reach was not focused on the beneficiaries of the intervention. Effectiveness was not assessed because it had been previously established as Bright IDEAS has been an NCI EBCCP for greater than 10 years (1–3, 6). Therefore, facilitating and understanding adoption, implementation, and maintenance of Bright IDEAS were the primary objectives of the training grant.

Quantitative survey outcome measures and qualitative codes were aligned with constructs of the three RE-AIM dimensions: adoption, implementation, and maintenance. Outcome measures included: intervention satisfaction (e.g., likelihood I will recommend Bright IDEAS to a colleague); barriers to adoption (e.g., lack time in clinic); implementation (e.g., lack of opportunity (clients); and maintenance of Bright IDEAS in clinical practice (e.g., reimbursement and/or insurance issues).

Supplementary materials provide the survey instrument and semi-structured interview guide. The guide was pilot tested with a small sample of psychosocial providers (N = 5) and changes were made based on feedback. Semi-structured interviews (range: 24–47 min per interview) were conducted over the telephone by the first author (DMM) who had no prior relationship with any of the respondents. All interviews were audio-recorded and transcribed verbatim.

Data analyses

Survey responses were coded into REDCap (16) secure web application for building and managing online surveys and databases and analyzed using SAS. Descriptive analyses were performed to summarize demographic characteristics of training participants and outcome measures of Bright IDEAS use, satisfaction, and implementation barriers at 6- and 12-months post training. Outcome measures were stratified into pre-adaptation and post-adaptation time periods and compared using Chi-square and two sample t-tests statistical test.

Analyses of the in-depth interviews were completed using data analysis package ATLAS.ti 8.0 (Scientific Software Development GmbH, Berlin, Germany) for coding by study authors (DMM, SB) who are PhD and PharmD trained researchers with experience in qualitative methods, health services research, and D&I science. All the transcripts were double coded. The coders familiarized themselves with the data by carefully reading the transcripts. They then deductively coded the data using the constructs of the three RE-AIM dimensions: adoption, implementation, and maintenance (17, 18). Discrepancies were resolved through discussion. Interviewer and analytic biases were managed during regular analysis meetings among all authors. Two study authors (DMM, SB) engaged in regular discussion of cases throughout the data analysis phase to ensure rigor. Transcribed interviews were coded by marked text with phrases indicating content of the discussions (19).

The Consolidated Criteria for Reporting Qualitative Research framework was used to guide the reporting of findings (20). Additionally, criteria for credibility, transferability, and confirmability were used to ensure rigor of this study (21, 22). Strategies used to address credibility included recording interviews and transcribing them; authors frequently discussing findings; encouraging participants to pursue their own line of thinking; and searching the data for conflicting patterns (21, 23). Confirmability was addressed by rigorous review of interview transcripts, the codes used to identify them, and drafts and revisions of the findings (23).

Results

Quantitative findings

The core functions and forms of the Bright IDEAS training workshop were identified as part of continuous program evaluation to determine adaptable components for local and clinical context needs (Table 1). The study team took a learning health system approach and conducted an in-person midpoint review in study year three. Based on pre-adaptation evaluation survey findings (N = 159) and in-depth interviews (N = 33) from workshop participants, implementation barriers or facilitators were identified, and new implementation strategies developed to mitigate barriers identified by study participants. As a result, four adaptations (Table 2) were made to the forms of the training program to facilitate transition from a research context to a clinical care context. Adaptations are described in Table 2 and involved the added requirement of institutional support to attend the training; changes in pre-workshop study materials; case study role playing in the training workshop; and added guidance to improve clinical workflow integration.

TABLE 1

Table 1. Core functions and forms of Bright IDEAS.

TABLE 2

Table 2. Bright IDEAS training adaptation for pediatric oncology practice.

Bright IDEAS training participants were primarily female (91%) and from academic medical centers (82%). The majority of practitioners were social workers (47%) or psychologists (39%) (Table 3). Measures of Bright IDEAS use and satisfaction through 12 months following training suggest that professionals who received the training would recommend Bright IDEAS to a colleague remained strong through 12 months and improved after adaptations to the Bright IDEAS training program [9.11 vs. 8.38 (p < 0.001) on a 1 to 10 scale, where 10 = “extremely likely”]. Intervention usage, as measured by the mean number of clients to whom Bright IDEAS was delivered, also improved following adaptations to the Bright IDEAS training program at both 6 months (5.67 vs. 4.01, p < 0.001) and at 12 months (9.04 vs. 6.31, p = ns; Table 4).

TABLE 3

Table 3. Characteristics of Bright IDEAS training participants.

TABLE 4

Table 4. Adoption of Bright IDEAS in pediatric oncology practice among training participants.

The most common situations endorsed as barriers were: “I lack time”, “Incorporating Bright IDEAS into my clinical workflow”, and “Client compliance issues”. “Lack of consensus of professional guidelines”, “Reimbursement and/or insurance issues”, and “Lack of experience” were reported as barriers in less than 10% of trainees (Table 5). Overall, the rank order of surveyed barriers and their perceived magnitude did not change appreciably after adaptation. The exceptions were: “I lack time” which decreased post-adaptation at the 12-month assessment (from 63 to 41% reporting as a barrier, p < 0.05); and “Lack of experience” which increased (from 3 to 15%, p < 0.05).

TABLE 5

Table 5. Reported barriers to Bright IDEAS adoption, implementation and maintenance in pediatric oncology practice.

Qualitative findings

Table 6 summarizes perceptions from the in-depth interviews about implementing Bright IDEAS into pediatric oncology practice. Data appeared to become redundant following the 23rd interview during the pre-adaptive phase and following the 11th interview during the post-adaptive phase. All authors agreed that no unique responses were emerging within the data and that saturation had been reached (18, 24). As practitioners had already agreed to participate, seven more interviews were completed during the pre-adaptation phase and three more during the post-adaptation phase. As no interview data was omitted, reported results reflects all the interview data. Representative quotes are provided to support the rationale for program adaptation of identified forms of Bright IDEAS to better align the program with clinical practice. The following further compares intervention perceptions before, and after, adaptations to the Bright IDEAS training program using three key constructs of the RE-AIM framework: adoption, implementation, and maintenance.

TABLE 6

Table 6. Perceptions about implementing the Bright IDEAS program in pediatric oncology practice among training participants before and after program adaptation.

Bright IDEAS adoption

Overall, there was general agreement that Bright IDEAS (referred to as “BI” in the quotes) was initially adopted, or not, at the independent discretion of the practitioner, with no institutional oversight. This resulted in practitioners creating their own methods for identifying clients they thought appropriate for the intervention. For example, a social worker described the type of client for the intervention this way: “I pretty much just think about how they're dealing with particular problems they seem to be having and deciding on my own whether or not I think BI would be a good intervention for them.” Additionally, as one psychologist noted, “some of it is based on how the family presents, and how they buy in [BI].”

After adaptations to the Bright IDEAS training program, the revised intervention materials were noted as useful with their clients and helped to organize the practitioner's clinical work. For example, a social worker stated, “I usually keep the worksheets and I have them take a picture with their phone of the action plan.” The profile of the ideal patient was identified by one psychologist as:

“Patients that have a lot of stressors that tends to be ongoing. So, that could either be a diagnosis and they've just started treatment, or that could be longstanding, strained relationships with their family or their partner. In addition to that, I would say families that don't have a lot of social support specifically, family or social support are good candidates for BI.”

Bright IDEAS implementation

In general, practitioners found difficulty with providing numerous written intervention materials with clients and sometimes “forgot” about using BI. For example, clients seemed overwhelmed with the new cancer diagnosis and unable to process a new resource, demonstrated by one psychologist's experience from a client, “Oh my gosh. Are you kidding me? They just gave me a calendar for medication and now you want me to write some things down?”

Additionally, it was observed that use of BI was inconsistent across clients. For example, a social worker stated, “I haven't been able to use this (BI) as frequently as I would have hoped.” Comparatively, a psychologist stated, “I have been offering it (BI). I go, introduce myself to families at the time of diagnosis and introduce the program as a support tool, and then follow-up after 4 weeks, you know the next time they're admitted, and then kind of offer the program at that point and time. It's standard just offering it to everybody regardless.” However, some practitioners found it difficult to fully deliver the prescribed five steps of BI as noted by one psychologist, “I think when we were trying to track things at the very beginning and be able to report back every month what was happening it felt just so much more rigid and made it difficult for the family to keep up with all of it.”

After adaptations to the Bright IDEAS training program, Bright IDEAS tended to be used in a greater variety of clinical situations. Specifically, a psychologist reported, “I have been able to implement BI in all different settings. So, I have done it inpatient, I have done it on the outpatient side and, and certainly done it in clinic as well. It is possible.” Similarly, a social worker recounted, “I work for a nonprofit and we actually are a community-based organization. So, we go into the patient's house…and meet with them in their environment to discuss their problems…using BI.”

Bright IDEAS maintenance

Plans to maintain the use of Bright IDEAS over time varied between practitioners; variability was partially explained by the local health system context. One psychologist noted,

“Our division of labor, will all be changing because we are one division that serves two hospitals and so once we all are under one program, the way that we provide psychosocial services will be changing, and our goal is that BI in the long term becomes part of a process where we offer it [BI] to everybody but that families who are identified as higher risk factors for all sorts of issues associated with the diagnosis, managing the diagnosis, et cetera, will be offered that program with a little bit more of a push.”

There was also a psychologist that took the initiative to train other practitioners, demonstrated by the exemplar,

“I have actually trained all of my students here. I trained my counterpart at our center as well as the social worker over there, and then we had other staff members here at our children's hospital, like we had a child life specialist ask to sit in on training, social work asked to sit in on training, so they have all been trained here already, and then additionally, we'll be kind of continually training students as they come through with us and I have also trained our child and adolescent psychiatry team who function in an outpatient mental health clinic, so they – we trained them as well.”

Comparatively, some practitioners mentioned being the “only one” at their institution trained in the intervention and recognized the difficulty in maintaining use of Bright IDEAS in their absence.

After adaptations to the Bright IDEAS training program, practitioners discussed ways in which they plan to continue to use Bright IDEAS over time. For example, one social worker shared, “I just consider it [BI] to be another very useful tool in my toolbox to use. I plan to just keep using it for families that clearly will benefit from it.” The sentiment was also expressed as, “I feel like it [BI] is something I'm going to always continue to use. If I notice that there are certain participants or patients, I have that would really benefit from having the goals.” Additionally, there were examples of planned internal training, “So, on my team there's two other people, a social worker, and a counselor. The plan is to teach them BI.”

Discussion

To the authors' knowledge, this is the first systematic multi-methods evaluation of the functions and forms of an NCI EBCCP-recognized intervention and its dissemination into clinical oncology practice. This study identifies the core functions and forms of a psychosocial intervention to address barriers to adoption, implementation, and maintenance of Bright IDEAS in real-world settings. Based on feedback from trainees, adaptations to the Bright IDEAS training program were made to foster a more pragmatic approach to intervention delivery and sustainment. The new training model fostered expanded use and acceptance of Bright IDEAS at the individual level.

The goal of the grant was to increase national awareness of Bright IDEAS, train providers on how to deliver Bright IDEAS, and facilitate adoption amongst the approximately 200 pediatric oncology centers operating in North America, as well as elucidate barriers to adoption, implementation, and maintenance in diverse real-world pediatric oncology practice settings. There was a clear distinction between the two training groups, indicating a positive response to the adaptations. We believe the difference between the two groups could be partially explained by the fact that behavioral interventions are more difficult to define and standardize because of the inherent interactivity with local client characteristics, preferences, and behaviors (25). Although Bright IDEAS has been proven to be efficacious for more than 20 years, factors affecting real-world application had not been studied.

Historically, the translation of tested interventions into clinical practice is limited by the inherent tension between intervention development and efficacy testing in the context of tightly-controlled explanatory trials and implementation in real-world settings (26). However, successful implementation of evidence-based interventions requires flexibility in treatment delivery based on clinical context. Scaling and sustainability of evidence-based programs often requires a trade-off between fidelity to the trial protocol and feasibility in a real-world clinical setting (27). To address this gap, stakeholder-informed adaptations to the training program as the intervention was being nationally scaled promoted flexibility in intervention delivery and feasibility based on dynamics of diverse clinical settings.

Implementation science has historically endorsed intervention permanence – i.e., once the evidence-base has been established for an intervention then practitioners can directly proceed to implementation, scale-up and sustainability. However, Chambers and Norton (28) posit that concerns around program drift and requirements for intervention permanence have not served implementation sciences well because it may hinder translation into real-world clinical practice (28). This study is an exemplar of an iterative approach to advance implementation science which responds to the call by revisiting the training protocol and delivery of Bright IDEAS in the clinical setting and acknowledging that interventions are not static events, rather they are dynamic in nature requiring continuing adaptations to meet the numerous demands of clinicians and the ever-changing context of the setting in which they are deployed.

This innovative approach to identifying standard core functions and how the forms were adapted to match practice characteristics was key in understanding practitioner needs and environmental factors. For example, we advised that Bright IDEAS not be formally introduced to families with newly diagnosed children until at least 4 weeks later, precisely because of a “not now” response upon initial implementation (1, 2, 4, 6). Lau et al. (29) observed similar results when examining adolescents and young adults' perspectives on facilitators and barriers to utilization of psychosocial programs and found that “starting something new” could be a significant barrier to utilization.

The current study revealed moderating factors that may affect adoption more broadly. This finding is not surprising as Greenhalgh et al. (30) noted that standard attributes of the intervention will not ensure adoption alone (30). Rather, the interaction among the intervention, intended adopters, and a particular context is what determines adoption rate (30). In this study, adaptations to the Bright IDEAS training program considered both practitioner experience and local setting. While there was improved adoption with adaptation, there is still opportunity for progress. Chambers and Norton (28) describe the necessary fit between interventions and their settings and suggest ongoing learning about optimal intervention delivery over time (28). Gathering feedback across diverse clinical settings should be planned as an iterative process that accounts for evolving methods of care and practice settings (31, 32).

The field has an enormous opportunity within the context of dissemination and implementation research to elucidate a full science of intervention adaptation (28). This study adds to empirical evidence by systematically collecting information on the impact of adaptation to practitioners and used this information to extend the knowledge base of implementation of evidence-based practices as well as ongoing improvement of the Bright IDEAS.

The path toward sustained maintenance of Bright IDEAS in clinical settings remains to be identified. While there were practitioners that intentionally trained colleagues or fellows, such training efforts were rarely supported by institutional leadership beyond attending training. Generalizable lessons learned underscore the importance of continual stakeholder engagement and administrative assistance to ensure long-term maintenance.

Limitations

While the aim of this study was to provide lessons learned to inform dissemination and sustainability planning for other psychosocial interventions, there are limitations that should be noted. First, results may be difficult to generalize to other diseases as practitioners were recruited from national organizations with a focus on pediatric and adolescent cancer. Another limitation is the possibility of social desirability bias. That is, some providers may have responded to questions in a manner they thought consistent with the research aims of the project. Future research with other stakeholders, such as institutional leadership and members of patient treatment teams, would be valuable to understanding factors that affect the dissemination and implementation process in the clinical setting.

The adaptations made to the psychosocial intervention training workshop presented in this study can help to bridge the science-to-service gap in mental health care and may provide important information regarding facilitators and barriers to implementation for other mental health researchers and implementation scientists. Moreover, may also be effective for the implementation of other psychosocial interventions or innovations in psychiatric care for patients with cancer, survivors of cancer, and for caregivers of those with cancer.

Summary

This multi-methods evaluation of a national training program highlights some of the issues psychosocial providers face when translating a new evidence-based intervention from research to practice settings, and the steps that can be taken to improve implementation. Further, attention to the fit between characteristics of an intervention and the clinical setting and the availability of resources, and knowledge of potential implementers is critical for informing an implementation process that capitalizes on facilitators and “works around” barriers. For busy psychologists and social workers, we found that a blend of strategies that helps to increase compatibility with existing organizational structures is critical for implementation.

Future pediatric oncology-based psychosocial interventions should build on the current focus of addressing adoption, implementation, and maintenance issues at the design stage of interventions when trials are first initiated (33). In addition, teams should explore adaptive dissemination strategies that aim to evolve to meet the dynamic nature of the clinical environment. Maintenance requires integration of research-tested protocols tailored for easy incorporation into routine clinical workflow. Longitudinal follow-up post training is imperative to ensuring the maintenance of an intervention; otherwise, “out of sight, out of mind” is inevitable.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving human participants were reviewed and approved by the Colorado Combined Institutional Review Board (COMIRB). Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.

Author contributions

OS, RN, and EM conceived and designed the experiment. DM, MV, and DF performed the experiments. DM, SB, and DF analyzed the data. OS, RN, MV, and DF contributed materials/analysis tools. DM and EM wrote the paper. OS and RN obtained the funding. All authors contributed to the article and approved the submitted version.

Funding

This study was funded by the National Institutes of Health and the National Cancer Institute for funding NIH Grant R01 CA 159013 and R25 CA65520.

Acknowledgments

Acknowledgment is extended to the National Institutes of Health and the National Cancer Institute for funding NIH Grant R01 CA 159013 and R25 CA65520. Megan Morris and Juliana Barnard of the Adult and Child Consortium for Health Outcomes Research and Delivery Science (ACCORDS), University of Colorado Anschutz Medical Campus are acknowledged for their qualitative methodological expertise and review. Allison Kempe, Director of the Adult and Child Center for Health Outcomes Research and Delivery Science (ACCORDS), University of Colorado Anschutz Medical Campus is acknowledged for overall project support.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/frhs.2022.928580/full#supplementary-material

Abbreviations

APHON, Association of Pediatric Hematology/Oncology Nurses; APOSW, Association of Pediatric Oncology Social Workers; COG, Children's Oncology Group; COMIRB, Colorado Combined Institutional Review Board (COMIRB); EBCCP, Evidence-Based Cancer Control Programs; NCI, National Cancer Institute; SPP, Society of Pediatric Psychology.

References

1. Sahler OJ, Varni JW, Fairclough DL, Butler RW, Noll RB, Dolgin MJ, et al. Problem-solving skills training for mothers of children with newly diagnosed cancer: a randomized trial. J Dev Behav Pediatr. (2002) 23:77–86. doi: 10.1097/00004703-200204000-00003

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Sahler OJ, Fairclough DL, Phipps S, Mulhern RK, Dolgin MJ, Noll RB, et al. Using problem-solving skills training to reduce negative affectivity in mothers of children with newly diagnosed cancer: report of a multisite randomized trial. J Consult Clin Psychol. (2005) 73:272–83. doi: 10.1037/0022-006X.73.2.272

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Sahler OJZ, Roghmann KJ, Mulhern RK, Carpenter PJ, Sargent JR, Copeland DR, et al. Sibling Adaptation to Childhood Cancer Collaborative Study: The association of sibling adaptation with maternal well-being, physical health, and resource use. J Dev Behav Pediatr. (1997) 18:233–43. doi: 10.1097/00004703-199708000-00003

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Dolgin MJ, Devine KA, Tzur-Bitan D, Askins MA, Fairclough DL, Katz ER, et al. Responsivity to problem-solving skills training in mothers of children with cancer. J Pediatr Psychol. (2020) 46:413–21. doi: 10.1093/jpepsy/jsaa117

PubMed Abstract | CrossRef Full Text | Google Scholar

5. D'Zurilla TJ, Goldfried MR. Problem solving and behavior modification. J Abnorm Psychol. (1971) 78:107–26. doi: 10.1037/h0031360

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Sahler OJ, Dolgin MJ, Phipps S, Fairclough DL, Askins MA, Katz ER, et al. Specificity of problem-solving skills training in mothers of children newly diagnosed with cancer: results of a multisite randomized clinical trial. J Clin Oncol. (2013) 31:1329–35. doi: 10.1200/JCO.2011.39.1870

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Nezu AM, D'Zurilla TJ. Problem-Solving Therapy: A Positive Approach to Clinical Intervention. Berlin: Springer Publishing Company (2006).

Google Scholar

8. Gordan VV. How to bridge research results to everyday clinical care? Oper Dent. (2017) 42:1–9. doi: 10.2341/16-154-B

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Morris ZS, Wooding S, Grant J. The answer is 17 years, what is the question: understanding time lags in translational research. J R Soc Med. (2011) 104:510–20. doi: 10.1258/jrsm.2011.110180

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Gyani A, Shafran R, Myles P, Rose S. The gap between science and practice: how therapists make their clinical decisions. Behav Ther. (2014) 45:199–211. doi: 10.1016/j.beth.2013.10.004

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Perez Jolles M, Lengnick-Hall R, Mittman BS. Core functions and forms of complex health interventions: a patient-centered medical home illustration. J Gen Intern Med. (2019) 34:1032–8. doi: 10.1007/s11606-018-4818-7

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Kirk MA, Haines ER, Rokoske FS, Powell BJ, Weinberger M, Hanson LC, et al. case study of a theory-based method for identifying and reporting core functions and forms of evidence-based interventions. Transl Behav Med. (2021) 11:21–33. doi: 10.1093/tbm/ibz178

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Sherman EJ, Primack BA. What works to prevent adolescent smoking? A systematic review of the national cancer institute's research-tested intervention programs. J School Health. (2009) 79:391–9. doi: 10.1111/j.1746-1561.2009.00426.x

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Varni JW, Sahler OJ, Katz ER, Mulhern RK, Copeland DR, Noll RB, et al. Maternal problem-solving therapy in pediatric cancer. J Psychosoc Oncol. (1999) 16:41–71. doi: 10.1300/J077v16n03_04

CrossRef Full Text | Google Scholar

15. Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Pub. Health. (1999) 89:1322–7. doi: 10.2105/AJPH.89.9.1322

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG, et al. metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. (2009) 42:377–81. doi: 10.1016/j.jbi.2008.08.010

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Bazeley P. Analysing qualitative data: More than ‘identifying themes'. Malaysian J Q Res. (2009) 2:6–22.

Google Scholar

18. Creswell JW, Hanson WE, Clark Plano VL, Morales A. Qualitative research designs: Selection and implementation. Couns Psychol. (2007) 35:236–64. doi: 10.1177/0011000006287390

CrossRef Full Text | Google Scholar

19. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. (2006) 3:77–101. doi: 10.1191/1478088706qp063oa

CrossRef Full Text | Google Scholar

20. Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Q Health Care. (2007) 19:349–57. doi: 10.1093/intqhc/mzm042

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Carnevale FA. Authentic qualitative research and the quest for methodological rigour. Can J Nurs Res Archive. (2016) 34:2.

PubMed Abstract | Google Scholar

22. Chiovitti RF, Piran N. Rigour and grounded theory research. J Adv Nurs. (2003) 44:427–35. doi: 10.1046/j.0309-2402.2003.02822.x

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Ritchie J, Lewis J, Nicholls CM, Ormston R. Qualitative Research Practice: A Guide for Social Science Students and Researchers. Thousand Oaks, CA: SAGE Publishing (2013).

Google Scholar

24. Wicks D. The coding manual for qualitative researchers. Q Res Org Manage Int J. (2017) 12:169–70. doi: 10.1108/QROM-08-2016-1408

CrossRef Full Text | Google Scholar

25. Glasgow RE, Lichtenstein E, Marcus AC. Why don't we see more translation of health promotion research to practice? Rethinking the efficacy-to-effectiveness transition. Am J Pub Health. (2003) 93:1261–7. doi: 10.2105/AJPH.93.8.1261

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Neta G, Johnson KE. Informing real-world practice with real-world evidence: the value of PRECIS-2. BMC Med. (2018) 16:1–3. doi: 10.1186/s12916-018-1071-1

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Lee SJ, Altschul I, Mowbray CT. Using planned adaptation to implement evidence-based programs with new populations. Am J Commun Psychol. (2008) 41:290–303. doi: 10.1007/s10464-008-9160-5

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Chambers DA, Norton WE. The adaptome: advancing the science of intervention adaptation. Am J Prev Med. (2016) 51:S124–31. doi: 10.1016/j.amepre.2016.05.011

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Lau N, Parsa AG, Walsh C, Yi-Frazier JP, Weiner BJ, Curtis JR, et al. Facilitators and barriers to utilization of psychosocial care in adolescents and young adults with advanced cancer: integrating mobile health perspectives. J Adolesc Young Adult Oncol. (2020) 10:476–92. doi: 10.1089/jayao.2020.0129

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. (2004) 82:581–629. doi: 10.1111/j.0887-378X.2004.00325.x

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Shelton RC, Chambers DA, Glasgow RE. An extension of RE-AIM to enhance sustainability: addressing dynamic context and promoting health equity over time. Front Public Health. (2020) 8:134. doi: 10.3389/fpubh.2020.00134

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Luke DA, Calhoun A, Robichaux CB, Elliott MB, Moreland-Russell S. The program sustainability assessment tool: a new instrument for public health programs. Prev Chronic Dis. (2014) 11:130184. doi: 10.5888/pcd11.130184

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Kwan BM, Brownson RC, Glasgow RE, Morrato EH, Luke DA. Designing for dissemination and sustainability to promote equitable impacts on health. Annu Rev Public Health. (2022) 43:331–53. doi: 10.1146/annurev-publhealth-052220-112457

PubMed Abstract | CrossRef Full Text | Google Scholar