Evaluating the implementation of a multi-level mHealth study to improve hydroxyurea utilization in sickle cell disease

Background: Sickle Cell Disease (SCD) is a progressive genetic disease that causes organ damage and reduces longevity. Hydroxyurea is an underutilized evidence-based medication that reduces complications and improves survival in SCD. In a multi-site clinical trial, part of the NIH-funded Sickle Cell Disease Implementation Consortium (SCDIC), we evaluate the implementation of a multi-level and multi-component mobile health (mHealth) patient and provider intervention to target the determinants and context of low hydroxyurea use. Given the complexity of the intervention and contextual variability in its implementation, we combined different behavioral and implementation theories, models, and frameworks to facilitate the evaluation of the intervention implementation. In this report, we describe engagement with stakeholders, planning of the implementation process, and final analytical plan to evaluate the implementation outcomes.

Methods: During 19 meetings, a 16-member multidisciplinary SCDIC implementation team created, conceived, and implemented a project that utilized Intervention Mapping to guide designing an intervention and its evaluation plan. The process included five steps: (1) needs assessment of low hydroxyurea utilization, (2) conceptual framework development, (3) intervention design process, (4) selection of models and frameworks, and (5) designing evaluation of the intervention implementation.

Results: Behavioral theories guided the needs assessment and the design of the multi-level mHealth intervention. In designing the evaluation approach, we combined two implementation frameworks to best account for the contextual complexity at the organizational, provider, and patient levels: (1) the Consolidated Framework for Implementation Research (CFIR) that details barriers and facilitators to implementing the mHealth intervention at multiple levels (users, organization, intervention characteristics, broader community), and (2) the Technology Acceptance Model (TAM), a conceptual model specific for explaining the intent to use new information technology (including mHealth). The Reach Effectiveness Adoption Implementation and Maintenance (RE-AIM) framework was used to measure the outcomes.

Discussion: Our research project can serve as a case study of a potential approach to combining different models/frameworks to help organize and plan the evaluation of interventions to increase medication adherence. The description of our process may serve as a blueprint for future studies developing and testing new strategies to foster evidence-based treatments for individuals living with SCD.

Introduction

Sickle cell disease (SCD) is a devastating genetic disease where progressive organ damage leads to premature death (1). Only a few evidence-based treatments exist for SCD, such as the disease-modifying agent hydroxyurea, and all are vastly underutilized. Uptake of disease-modifying therapies by patients with SCD is impacted by socio-economic barriers that create health inequities for this population (2–4). To accelerate the equitable translation of evidence-based treatments into clinical care for individuals with SCD, the Sickle Cell Disease Implementation Consortium (SCDIC) was created in 2016 (5). The SCDIC is a cooperative research program funded by the NHLBI and composed of eight academic and clinical sites and one data coordinating center. SCDIC members include clinicians, epidemiologists, implementation scientists, government representatives, behavioral scientists, and patient partners. A primary goal of the SCDIC is to develop and test the effectiveness of interventions aimed at increasing the uptake of evidence-based therapies for SCD while seeking input from key stakeholders during the entire implementation strategy development process (5).

As the first FDA-approved evidence-based drug for SCD, daily oral hydroxyurea is a medication recommended by guidelines (6) and with robust evidence for reducing acute and chronic disease complications, lowering acute-care utilization, and improving survival among individuals with SCD (7–11). In the U.S., <50% of patients with SCD appropriately utilize hydroxyurea (12–15), severely limiting its population impact. SCDIC consortium members thus chose to focus on strategies to increase hydroxyurea uptake. The consortia members and stakeholder partners worked together to develop a multi-level/multi-component mobile health (mHealth) intervention to increase provider prescribing practices and patient medication adherence to hydroxyurea. The implementation of this novel mHealth intervention is currently being tested in a multicenter study (16, 17).

During the development of the multi-level/multi-component mHealth intervention, a major challenge within the SCDIC was to construct an evaluation approach to identify how each contextual level (i.e., at the patient, provider, and clinical setting) and intervention component (i.e., the different mHealth features) could influence implementation and effectiveness outcomes, both as individual factors and interactively. For instance, patients, providers, and institutional characteristics can influence the implementation and distal effectiveness outcomes. Still, implementation outcomes can also interact across multiple levels (e.g., provider adoption might impact patient reach), modifying the effectiveness outcomes. To address this challenge, we used the intervention mapping methodology (18), which allows for strong collaboration among researchers and patient partners to develop the multi-level/multi-component mHealth intervention and to guide the design of the evaluation plan.

In this paper, we will report the process development of an intervention evaluation plan that considers what and how contextual factors influence implementation outcomes while anticipating possible interrelationships (e.g., synergy) among the intervention components. This paper will discuss different theories, models, and frameworks and how they are optimally combined to evaluate the implementation of a multi-level/multi-component mHealth intervention to improve hydroxyurea utilization in SCD (clinicaltrials.gov NCT03344900) (17). We describe the rationale, process, and application of the implementation process and the engagement with stakeholders to develop the analytical plan of the intervention and define implementation outcomes. The description of our design process may serve as a blueprint for future studies developing and testing new strategies to foster the use of evidence-based treatments for individuals living with SCD.

Methods

This descriptive narrative of the process selects and blends theories, models, and frameworks to design the clinical trial evaluation that tests a new multi-level/multi-component mHealth intervention to improve hydroxyurea adherence among patients with SCD (17).

Settings and population

This multi-center study included seven SCDIC clinical sites, all academic institutions in urban, suburban, and rural areas, with some degree of organizational and population variability. All sites have experienced SCD providers (adult and pediatric hematologists and advanced care practitioners, all trained in SCD management) and trainees (e.g., residents and fellows). The prevalence of eligible patient participants (i.e., patients with SCD treated with hydroxyurea who were not receiving monthly blood transfusions and not using mobile apps to improve adherence) ranged from 40 to 60% of the patient population at each site. Among eligible participants, approximately 70% were covered under government health plans (primarily Medicaid). Most patients were aged >25 years, although, in two sites, about half of eligible patients were adolescents. Among eligible patients, approximately 70% had a severe SCD genotype (HbSS and HbSβ⁰-thalassemia).

Research team

Our SCDIC team comprises 16 members, including implementation scientists, hematologists, health science researchers, behavioral scientists, research coordinators, biostatisticians, patient partners, clinicians, and epidemiologists who collaborated on the study throughout 19 meetings from December 2017 to October 2019. In addition to the research team, the SCDIC steering committee (including principal site investigators, NHLBI representatives, patient partners, data coordinating center staff, and study coordinators) and the SCDIC Implementation Research Committee (composed of implementation scientists) offered an additional layer of scientific input, suggested changes to the study, and voted to approve the study in its final form.

Summary of activities

Our activities began by providing level-setting knowledge about SCD and dissemination and implementation research to all members of the SCDIC. As the research team was diverse and background knowledge was often non-overlapping, much of the initial work centered on the generation of a shared vision, i.e., establishing agreement on the goals of the program and the process whereby a research question was developed and addressed. The design of the intervention components followed a comprehensive needs assessment phase examining the barriers and enablers to hydroxyurea utilization in the SCD population at multiple levels (Activity 1). Next, a conceptual model evolved using Intervention Mapping (Activity 2) to guide the development of the interventions and the planning of the implementation strategies (Activity 3), the selection of the models and frameworks (Activity 4), and the evaluation plan (Activity 5). We used an inclusive consensus approach throughout all activity phases and defined and selected priorities, designed interventions, and measured outcomes. Our study is fully accrued and is on track to complete data analysis by March 2023. Next, we will present the details of each activity.

Activity 1: Needs assessment of hydroxyurea utilization

Our needs assessment focused on care redesign to improve hydroxyurea uptake and considered intervention targets, modalities, and strategy mechanisms. Barriers and enablers to hydroxyurea utilization in the SCD population were conducted through literature synthesis (19–22), population-level claims analysis (13, 23), patient and provider surveys (24, 25), semi-structured interviews, and focus groups (16, 26). To synthesize and organize the needs assessment findings at the patient level, the Health Belief Model (HBM) (27) was used, and the Social Cognitive Theory (SCT) was used as the behavioral change model (28). The HBM has been broadly used to evaluate the acceptance of health services among people with SCD, including attendance to clinic visits (29), sepsis (30), and stroke screening (31). The constructs within the HBM align well with the disease characteristics and demographics (e.g., intermittent acute exacerbations align with perceived susceptibility), therefore, was selected for this project. In alignment with our behavioral models, our needs assessment findings identified the main determinants of poor hydroxyurea adherence among patients with SCD: perceived high susceptibility, high disease severity, low motivation to take medications, memory deficit (leading to poor habituation), low understanding of hydroxyurea benefit (i.e., medication knowledge), and low self-efficacy with taking hydroxyurea (16, 26). Among providers of patients with SCD, the main determinants of low rates of prescribing hydroxyurea are: limited knowledge of the drug, lack knowledge of the national care guidelines, and low self-efficacy in dosing hydroxyurea (24, 32). Following this formative evaluation, our team identified two priority target levels for intervention: 1) patient hydroxyurea adherence and 2) provider prescribing of hydroxyurea.

Activity 2: Development of the conceptual model

Informed by the needs assessment, we developed a comprehensive conceptual model centered on Intervention Mapping. To organize the components of the intervention at both the patient and the provider levels, we used Intervention Mapping to identify specific methods and practical applications to create change in determinants, performance objectives, and behavioral change outcomes. Intervention Mapping is a protocol based on using theory and evidence for developing effective behavior change interventions (18). A primary aim of the consortia was to develop a research protocol that included a consensus framework. Drawing on the team's expertise and experience in using different implementation frameworks, we selected frameworks based on overarching aims (33). For instance, to understand what influences implementation, we selected a determinant framework, and to evaluate the implementation we chose an evaluation framework. We thus combined different models and frameworks to 1) identify determinants of poor hydroxyurea utilization at multiple levels, 2) guide the design of the interventions and strategies to be used at different intervention levels, and 3) plan the implementation evaluation. This comprehensive process allowed us to identify the salient targets to increase the use of the evidence-based treatment for SCD, hydroxyurea, while identifying and prioritizing important influencers of the implementation at the contextual level.

Activity 3: Intervention design process

SCDIC investigators noted the widespread use of technology among patients with SCD (>90% own smartphones, 91% use them regularly for communication, and 87% rate the highest possible comfort levels (34, 35) and the desire of providers to receive SCD guideline information in a mobile platform (36). The research team arrived at the consensus that to best “package” all necessary functions of the intervention and deliver it to the two targets (patients and providers), mHealth was the ideal conduit. mHealth intervention refers to the use of mobile technology for medical and public health practices (37) and here it serves as the channel for effecting behavioral modifications.

Using user-centered design principles, we co-created (with patients and providers) a two-level intervention: (1) InCharge Health app, which incorporated features that acted on the modifiable determinants of patients' poor hydroxyurea adherence (e.g., low cue to action, low self-efficacy in taking medication) (Figure 1), and (2) HU Toolbox app for providers, which incorporated features that acted on modifiable determinants of clinician failure to prescribe hydroxyurea (e.g., low disease knowledge, low self-efficacy in prescribing hydroxyurea) (Figure 1). User-centered design is an approach to designing and developing products that grounds its process on the information about the people who will ultimately use the products to improve usability and user experience (38, 39). Patients used InCharge Health within their own lived environment, while providers used HU Toolbox in the clinic/office. Because in clinical practice, providers prescribe and counsel patients on the benefits of hydroxyurea during regular visits, providers were both the actors and targets of our multi-level intervention (Figure 1) (40).

FIGURE 1

Figure 1. Multi-level/multi-component intervention mapping and implementation outcomes. The intervention components (mHealth activities) address the determinants of hydroxyurea utilization at the patient and provider levels. Performance outcomes depict the actions taken by the targets of the intervention. Provider-appropriate prescribing influences patients’ hydroxyurea adherence, promoting increased hydroxyurea utilization, reduced organ damage, and improved quality of life.

A menu of implementation strategies was used to increase the implementation outcomes of both InCharge Health and HU Toolbox. The “train and educate stakeholders strategy” (41, 42) included ongoing consultation and training of patients and providers on how to use both interventions, both as one-on-one activities (patient or provider individual meetings while in clinic) and group activities (provider educational meetings during staff and faculty meetings). “Support of the clinicians”” strategy (41, 42) was utilized during the study as reminders to clinicians to use the HU Toolbox and the facilitation of relay of clinical data as a function of the HU Toolbox app (i.e., guidance on correct prescribing of hydroxyurea app function). Other implementation strategies included understanding barriers and facilitators to digital health interventions, a thorough understanding of the implementation context, and patient feedback. These strategies were prospectively tracked and documented by each participating site.

Activity 4: Selection of implementation models and frameworks

Our challenge in organizing patient, provider, and clinic-level contextual factors within SCDIC included the selection of implementation frameworks that would appropriately represent the multiple dimensions of the intervention (two target levels and several mHealth features) while accounting for the complex context where the population received care and where we sought to enhance the impact of our intervention. For instance, unmet social needs can lead to health disparities (43) and may significantly influence how patients with SCD engage with the intervention. Recruitment efforts that lead to equitable reach across the patient population and equity in the adoption and delivery of the intervention among providers are of paramount importance in the SCD population, given that the majority belong to underserved groups in the United States. Additionally, SCD characteristics (e.g., the disease severity) and co-morbidities might also affect response to the intervention (i.e., response heterogeneity) and lead to a lack of robust effects. Given the variability in patient characteristics across the participating sites, this was of particular concern.

SCDIC investigators considered several models and frameworks to define theory-based domains associated with contextual variables and the overall robustness of the mHealth innovation. Our goal was to examine the influences of patient, provider, and clinical setting level characteristics on study outcomes and the qualitative examination of barriers and enablers of the mHealth innovation. Given that different models and frameworks identify other metrics for evaluating implementation success, the group decided to combine the Consolidated Framework for Implementation Research (CFIR) (44) and the Reach Effectiveness Adoption Implementation and Maintenance (RE-AIM) (45) frameworks. CFIR is a determinant framework with five domains (inner setting, outer setting, intervention, process, and individuals involved) that systematically assesses potential barriers and facilitators (the determinants) of an implementation. RE-AIM is a versatile planning and evaluation framework, in which dimensions (reach effectiveness, maintenance, adoption, and implementation) systematically capture the outcomes of the implementation while assessing the equitability of the implementation (46). Because we planned qualitative data collection as part of our formative evaluation of the implementation process, we used RE-AIM Qualitative Evaluation for Systematic Translation (RE-AIM QuEST) to develop the interview questions (47).

In addition to CFIR and RE-AIM/RE-AIM QuEST, we also used the Technology Acceptance Model (TAM) (48, 49) to design and measure the outcomes specifically associated with mHealth use. Users’ acceptance of new technology, including new mHealth innovations, impacts its successful adoption. TAM is a conceptual model that explains the intent to use new information technology (including mHealth) or information science among users, including medical providers. TAM has five constructs: perceived usefulness, perceived ease of use, compatibility, mobile health care systems self-efficacy, and technical support and training. However, perceived ease of use and perceived usefulness are the two dominant determinants of technology use. mHealth care systems self-efficacy is the health care professional's perception of their ability to use health care systems to accomplish the health care task and must be accounted for when new technology is implemented. The combination of implementation frameworks RE-AIM/RE-AIM QuEST, CFIR, and TAM informed the planning of the study while complementing each other in evaluating the complexity of the influential multi-level factors on implementation (Figure 2). TAM was used to measure the specific constructs related to the intervention/m-Health tool. CFIR was used in specific quantitative measures of intervention characteristics, characteristics of individuals, inner settings, and processes. The five RE-AIM domains (Reach, Effectiveness, Adoption, Implementation, Maintenance) were used to evaluate implementation outcomes with quantitative and qualitative assessments.

FIGURE 2

Figure 2. Interrelation of theories, models, and frameworks. The Health Belief Model (HBM) and Social Cognitive Theory (SCT) informed the design of the InCharge Health app. Technology Acceptance Model (TAM) informed the creation of the HU Toolbox. The Consolidated Framework for Implementation Research (CFIR) constructs will inform the evaluation of the multiple-level influential factors, including the inner setting (organization), the users (patients and providers), intervention characteristics (InCharge Health and HU Toolbox apps), and app implementation process. The Reach Effectiveness Adoption implementation and Maintenance (RE-AIM) framework domains will be used to evaluate the implementation process. Dashed arrows represent the possible influential effects across CFIR constructs and RE-AIM domains.

Activity 5: Evaluation plan

To visualize the potential interrelatedness of the multiple influential factors of the context, we created a matrix that grouped CFIR and TAM domains and mapped them to all 5 RE-AIM domains for both the patient-level (Supplementary Table S1) and provider-level (Supplementary Table S2) strategies. We designed specific plans to examine how influential factors of app utilization potentially moderate the relationship between (1) how the level of InCharge Health implementation might correlate with hydroxyurea adherence and (2) how the level of HU Toolbox implementation might correlate to increases in providers' knowledge and self-efficacy in prescribing hydroxyurea. To assess outcomes within each RE-AIM domain, quantitative measures were selected with possible moderation or mediation by CFIR constructs (Supplementary Tables S1, S2). In mixed methods evaluation, qualitative data complements and expands, within key themes, the “why” and “how” barriers and facilitators to recruitment, implementation, and sustainability affect implementation. The description of all measures utilized in this study and their frequency have been previously described (17). In brief, quantitative surveys are given at baseline and study exit, while qualitative data (semi-structured interviews) are conducted at the end of study participants at each site.

Discussion

Dissemination and Implementation research has a growing number of models and frameworks, and parsimoniously using them is essential, but this is not always possible when there is high complexity in contextual factors and intervention components. While there might exist a need to combine aspects of various theories, models, and frameworks in designing interventions and complex evaluations, the methods for approaching this blending process are not widely available. Our study serves as a case study demonstrating the process of co-developing a multi-level/multi-component intervention and designing its evaluation. In our study, we combined HBM, SCT, RE-AIM, CFIR, and TAM while incorporating the broad perspectives of the diverse team members to plan and operationalize the frameworks, measurements, and evaluation. This manuscript outlines a rationale, process, and application that may be useful as an example for others to consider in designing an implementation evaluation, particularly around electronic health interventions to improve medication adherence in chronic diseases.

When clinical guidelines are implemented, theories, models, and frameworks are not always used to guide intervention and strategy development or their evaluation (50, 51). When theories are not used to plan, undertake, or evaluate implementation, the correct diagnosis of the underlying reasons for the success or failure of implementation (i.e., the determinants or the barriers and facilitators of low guideline adoption) may not be identified, therefore reducing the likelihood of effective interventions. Our study exemplifies how theories, models, and frameworks can be used to guide the entire process of planning, execution, and evaluation of guideline implementation, in our case, the utilization of hydroxyurea in SCD.

The process evaluation of a multi-level implementation study is complex and includes considerations of potential interactions between the intervention elements and the targeted levels of the intervention. Additionally, in chronic diseases, where there is substantial variability among the clinical characteristics of the patient, patient and organizational care settings, stakeholder perspectives, and the health care provider's expertise level, attention to contextual factors is particularly relevant when interpreting the effects of the intervention on outcomes. The careful development of an evaluation process that accounts for the different intervention and contextual components is thus paramount but potentially very complex. The description of the development of the process evaluation of a multi-faceted intervention can advance our understanding of their effects by illustrating the evaluation process design.

Lessons learned

Over the course of 19 meetings, a 16-member multidisciplinary implementation team within the SCDIC conceived and implemented a project that utilized Intervention Mapping to guide the process, which leveraged and combined different models and frameworks to plan the evaluation of the implementation of a multi-level intervention to increase adherence to the evidence-based therapy for individuals with SCD, hydroxyurea. Our team tackled the difficult task of addressing a vital problem in the SCD field (the low hydroxyurea uptake) by overcoming the challenges of forming a functional and integrated diverse team that had first to learn how to work together, share knowledge, and develop the conceptual models, to create the interventions, select, and combine implementation frameworks, and design the evaluation plans that are appropriate for use in complex multilevel interventions. Our diverse community of scientists, government representatives, clinicians, and patient partners recognized that a proper understanding of the implementation process would require careful consideration of the multi-level factors that can influence the implementation in the SCD population. Embedded in this concept was the notion that for the group to properly function and advance the research question, developing trust, respect, and having a shared vision was essential. The research team integration followed collaboration and team science principles, which set clear expectations for sharing credit, authorship, and maintaining self-awareness (52). Strong communication was at the core of our team's functioning. The group consensus was initially slow to develop in trying to choose the design of the intervention (i.e., targets, function) and which model or framework for evaluation. Implementation researchers came with knowledge but often loyalty to a particular model, while clinical investigators were new to implementation methods and evaluation. The process creating a functional partnership between implementation science researchers and clinical investigators underscored the need to develop a “common language” between both teams while investing time to build knowledge of the respective fields across all team members. Organizing our thinking around a logic model and focusing on the “diagnosis of the implementation gap” (i.e., the determinants of the hydroxyurea utilization gap (illustrated in Figure 1) and choosing the frameworks second was instrumental in creating efficiency. Fortunately, there was sufficient lead time and support from the funding agency to examine the strengths and drawbacks associated with various models and frameworks and the possibility of combining aspects of each to reach an agreement. Accounting for the lead time (and possible delays) in creating multidisciplinary research teams and reaching team integration is important and should not be overlooked.

Our selection of theories, models, and frameworks followed our needs assessment phase. For this project, no models were adapted to fit the study intervention or evaluations. However, for some frameworks, not all constructs were applicable. For instance, the “Mobile health care systems self-efficacy” driver within TAM was less pertinent to developing the provider-level intervention. That's because, among SCD providers, the perception that mHealth could help with the task of prescribing hydroxyurea was less of a factor when the dominant perceived ease of use and usefulness of the intervention were used (i.e., judging if mHealth was helpful was less important than how much this tool could increase their productivity). Additionally, the outer setting construct within CFIR was not examined, as implementing mHealth in clinics is mainly controlled by the local leadership and policies and generally not regulated by the existing health systems policies. We did not encounter problems aligning our outcomes and influential contextual factors to the existing domains and constructs of the frameworks used. However, the issue of problems aligning with existing domains may occur depending on the research question, and population studied. Therefore, the careful selection of models and frameworks deserves extra attention and effort from the investigators, as adaptations to existing models and frameworks may be required.

Our work describing the process development of how to combine implementation models and frameworks to study how multi-level contextual factors optimally and comprehensively affect implementation outcomes adds to this growing literature. When models and frameworks are combined to evaluate the implementation of an intervention or practice, essential factors that caused the organization to reject or accept the intervention can be uncovered. For instance, formally appointing key stakeholders to ensure the fidelity of the intervention or gaining visible support from the system and local leaders are contextual factors that are not necessarily known as pre-conditions for optimal implementation (53, 54). While CFIR has been used to evaluate the implementation of the transition to adult care activities in SCD (55), rare examples of the combination of different models and frameworks in SCD exist (56), and none focused on medication utilization. Whereas RE-AIM provides a practical framework for planning and evaluating mHealth interventions, other models such as TAM and CFIR could explain why implementation might succeed or fail if used proactively and help to identify relevant modifiable factors affecting adoption, implementation, and maintenance. By understanding the why, we hope to identify mediators and moderators of the intervention and narrow down the components of the intervention that should be modified, removed, or new ones that need to be created in further adaptation while shedding light on the mechanism of interventions and identification of new implementation strategies. Finally, adding qualitative data to CFIR will allow us to map the level of influence on the CFIR constructs and domains. A qualitative design can expand quantitative data and provide new hypotheses to explain why implementation succeeds. Therefore, adding a qualitative design when using CFIR can be useful.

Limitations

Our process to select the theories, models, and frameworks was iterative and followed matching theories and strategies to the type of intervention we sought to implement to best fit the quality gap we were trying to address, namely, low hydroxyurea utilization in SCD. Our process ensured diverse perspectives and consensus. It is possible, however, that this selection was not optimal. For example, our choice of models and frameworks preceded the publication of the RE-AIM expansion, which recognizably could represent an alternative to our approach. RE-AIM and PRISM can also be combined to investigate contextual predictors of the implementation outcomes, and an expanded version of RE-AIM has recently been published (57). Although the process development of the intervention and its evaluation were carefully developed and reported, not all implementation strategies might have been pre-identified for tracking. This may limit our ability to evaluate our implementation in the future fully. A pre-specification and comprehensive identification of all implementation strategies need to be done, and appropriate time should be allotted to this activity during study planning.

Conclusions

In conclusion, we report the processes of developing a multi-level, multi-component intervention to foster greater use of hydroxyurea therapy among patients with SCD and detail this multi-level intervention and our comprehensive plan for its evaluation. Careful consideration of how the multiple components of the intervention can interact with the various targets and contextual factors will facilitate the description of the implementation's how, when, what, where, and who of the implementation and the why. The results of our mHealth adherence-enhancing study and future research will refine the evaluation approach to create new knowledge in developing an evaluation model for multi-level interventions to increase hydroxyurea uptake among adolescents and adults with SCD.

Data availability statement

The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author/s.

Author contributions

JSH and LMK: study concept, wrote the manuscript. MBP, MEF, LD, AAB, HBB, VRG, NS: study concept, edited manuscript. CM, SRJ, AAK, JS, JAG: revised manuscript. All authors contributed to the article and approved the submitted version.

Funding

The Sickle Cell Disease Implementation Consortium has been supported by US Federal Government cooperative agreements U24HL133948, U01HL133964, U01HL133990, U01HL133996, U01HL133994, U01HL133997, U01HL134004, U01HL134007, and U01HL134042 from the National Heart Lung and Blood Institute and the National Institute on Minority Health and Health Disparities (Bethesda, MD), and the American Society of Hematology Bridge Grant. P50 CA244431 partially supported Dr. Lisa Klesges time during the conduct of this work.

Conflict of interest

JSH receives consultancy fees from Forma Therapeutics, CVS Health, and Global Blood Therapeutics.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, the funding agencies or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/frhs.2022.1024541/full#supplementary-material.

References

1. Kato GJ, Piel FB, Reid CD, Gaston MH, Ohene-Frempong K, Krishnamurti L, et al. Sickle cell disease. Nat Rev Dis Primers. (2018) 4:18010. doi: 10.1038/nrdp.2018.10

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Desai P, Little J, Kanter J, Bridges K, Andemariam B, Lanzkron S. Kneeling was the first step for sickle cell disease. Ann Intern Med. (2021) 174(7):1004–5. doi: 10.7326/M20-7418

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Power-Hays A, McGann PT. When actions speak louder than words - racism and sickle cell disease. N Engl J Med. (2020) 383(20):1902–3. doi: 10.1056/NEJMp2022125

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Lee L, Smith-Whitley K, Banks S, Puckrein G. Reducing health care disparities in sickle cell disease: a review. Public Health Rep. (2019) 134(6):599–607. doi: 10.1177/0033354919881438

PubMed Abstract | CrossRef Full Text | Google Scholar

5. DiMartino LD, Baumann AA, Hsu LL, Kanter J, Gordeuk VR, Glassberg J, et al. The sickle cell disease implementation consortium: translating evidence-based guidelines into practice for sickle cell disease. Am J Hematol. (2018) 93(12):E391–E5. doi: 10.1002/ajh.25282

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Yawn BP, Buchanan GR, Afenyi-Annan AN, Ballas SK, Hassell KL, James AH, et al. Management of sickle cell disease: summary of the 2014 evidence-based report by expert panel members. JAMA. (2014) 312(10):1033–48. doi: 10.1001/jama.2014.10517

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Lobo CL, Pinto JF, Nascimento EM, Moura PG, Cardoso GP, Hankins JS. The effect of hydroxcarbamide therapy on survival of children with sickle cell disease. Br J Haematol. (2013) 161(6):852–60. doi: 10.1111/bjh.12323

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Le PQ, Gulbis B, Dedeken L, Dupont S, Vanderfaeillie A, Heijmans C, et al. Survival among children and adults with sickle cell disease in Belgium: benefit from hydroxyurea treatment. Pediatr Blood Cancer. (2015) 62(11):1956–61. doi: 10.1002/pbc.25608

PubMed Abstract | CrossRef Full Text | Google Scholar

9. de Araujo OM, Ivo ML, Ferreira Junior MA, Pontes ER, Bispo IM, de Oliveira EC. Survival and mortality among users and non-users of hydroxyurea with sickle cell disease. Rev Lat Am Enfermagem. (2015) 23(1):67–73. doi: 10.1590/0104-1169.3385.2526

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Voskaridou E, Christoulas D, Bilalis A, Plata E, Varvagiannis K, Stamatopoulos G, et al. The effect of prolonged administration of hydroxyurea on morbidity and mortality in adult patients with sickle cell syndromes: results of a 17-year, single-center trial (LaSHS). Blood. (2010) 115(12):2354–63. doi: 10.1182/blood-2009-05-221333

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Steinberg MH, Barton F, Castro O, Pegelow CH, Ballas SK, Kutlar A, et al. Effect of hydroxyurea on mortality and morbidity in adult sickle cell anemia: risks and benefits up to 9 years of treatment. JAMA. (2003) 289(13):1645–51. doi: 10.1001/jama.289.13.1645

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Kang HA, Barner JC. The association between hydroxyurea adherence and opioid utilization among Texas medicaid enrollees with sickle cell disease. J Manag Care Spec Pharm. (2020) 26(11):1412–22. doi: 10.18553/jmcp.2020.26.11.1412

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Mathias JG, Nolan VG, Klesges LM, Badawy SM, Cooper WO, Hankins JS, et al. Hydroxyurea use after transitions of care among young adults with sickle cell disease and Tennessee medicaid insurance. JAMA Netw Open. (2021) 4(10):e2128971. doi: 10.1001/jamanetworkopen.2021.28971

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Ritho J, Liu H, Hartzema AG, Lottenberg R. Hydroxyurea use in patients with sickle cell disease in a medicaid population. Am J Hematol. (2011) 86(10):888–90. doi: 10.1002/ajh.22134

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Kayle M, Valle J, Paulukonis S, Holl JL, Tanabe P, French DD, et al. Impact of medicaid expansion on access and healthcare among individuals with sickle cell disease. Pediatr Blood Cancer. (2020) 67(5):e28152. doi: 10.1002/pbc.28152

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Alberts NM, Badawy SM, Hodges J, Estepp JH, Nwosu C, Khan H, et al. Development of the InCharge health Mobile app to improve adherence to hydroxyurea in patients with sickle cell disease: user-centered design approach. JMIR Mhealth Uhealth. (2020) 8(5):e14884. doi: 10.2196/14884

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Hankins JS, Shah N, DiMartino L, Brambilla D, Fernandez ME, Gibson RW, et al. Integration of mobile health into sickle cell disease care to increase hydroxyurea utilization: protocol for an efficacy and implementation study. JMIR Res Protoc. (2020) 9(7):e16319. doi: 10.2196/16319

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Fernandez ME, Ruiter RAC, Markham CM, Kok G. Intervention mapping: theory- and evidence-based health promotion program planning: perspective and examples. Front Public Health. (2019) 7:209. doi: 10.3389/fpubh.2019.00209

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Loiselle K, Lee JL, Szulczewski L, Drake S, Crosby LE, Pai AL. Systematic and meta-analytic review: medication adherence among pediatric patients with sickle cell disease. J Pediatr Psychol. (2016) 41(4):406–18. doi: 10.1093/jpepsy/jsv084

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Badawy SM, Thompson AA, Penedo FJ, Lai JS, Rychlik K, Liem RI. Barriers to hydroxyurea adherence and health-related quality of life in adolescents and young adults with sickle cell disease. Eur J Haematol. (2017) 98(6):608–14. doi: 10.1111/ejh.12878

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Walsh KE, Cutrona SL, Kavanagh PL, Crosby LE, Malone C, Lobner K, et al. Medication adherence among pediatric patients with sickle cell disease: a systematic review. Pediatrics. (2014) 134(6):1175–83. doi: 10.1542/peds.2014-0177

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Lanzkron S, Strouse JJ, Wilson R, Beach MC, Haywood C, Park H, et al. Systematic review: hydroxyurea for the treatment of adults with sickle cell disease. Ann Intern Med. (2008) 148(12):939–55. doi: 10.7326/0003-4819-148-12-200806170-00221

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Brousseau DC, Richardson T, Hall M, Ellison AM, Shah SS, Raphael JL, et al. Hydroxyurea use for sickle cell disease among medicaid-enrolled children. Pediatrics. (2019) 144(1):e20183285. doi: 10.1542/peds.2018-3285

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Smeltzer MP, Howell KE, Treadwell M, Preiss L, King AA, Glassberg JA, et al. Identifying barriers to evidence-based care for sickle cell disease: results from the sickle cell disease implementation consortium cross-sectional survey of healthcare providers in the USA. BMJ Open. (2021) 11(11):e050880. doi: 10.1136/bmjopen-2021-050880

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Kanter J, Gibson R, Lawrence RH, Smeltzer MP, Pugh NL, Glassberg J, et al. Perceptions of US adolescents and adults with sickle cell disease on their quality of care. JAMA Netw Open. (2020) 3(5):e206016. doi: 10.1001/jamanetworkopen.2020.6016

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Hodges JR, Phillips SM, Norell S, Nwosu C, Khan H, Luo L, et al. Intentional and unintentional nonadherence to hydroxyurea among people with sickle cell disease: a qualitative study. Blood Adv. (2020) 4(18):4463–73. doi: 10.1182/bloodadvances.2020001701

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Stretcher V, Rosenstock IM. The health belief model. In: Glanz K, Rimer BK, Viswanath K, editors. Health Behavior and Health Education: Theory, Research, and Practice. San Francisco: Jossey-Bass; 1997.

28. Bandura A. Social cognitive theory. Handbook of theories of social psychology, vol 1. Thousand Oaks, CA: Sage Publications Ltd (2012). 349–73.

29. Cronin RM, Hankins JS, Byrd J, Pernell BM, Kassim A, Adams-Graves P, et al. Modifying factors of the health belief model associated with missed clinic appointments among individuals with sickle cell disease. Hematology. (2018) 23(9):683–91. doi: 10.1080/10245332.2018.1457200

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Schultz CL, Tchume-Johnson T, Schapira MM, Bellamy S, Smith-Whitley K, Ellison A. Adherence to prompt fever evaluation in children with sickle cell disease and the health belief model. Pediatr Blood Cancer. (2015) 62(11):1968–73. doi: 10.1002/pbc.25634

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Bollinger LM, Nire KG, Rhodes MM, Chisolm DJ, O'Brien SH. Caregivers' perspectives on barriers to transcranial doppler screening in children with sickle-cell disease. Pediatr Blood Cancer. (2011) 56:99–102. doi: 10.1002/pbc.22780

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Okocha EC, Gyamfi J, Ryan N, Babalola O, Etuk EA, Chianumba R, et al. Barriers to therapeutic use of hydroxyurea for sickle cell disease in Nigeria: a cross-sectional survey. Front Genet. (2021) 12:765958. doi: 10.3389/fgene.2021.765958

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Nilsen P. Making sense of implementation theories, models and frameworks. Implement Sci. (2015) 10:53. doi: 10.1186/s13012-015-0242-0

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Jonassaint CR, Shah N, Jonassaint J, De Castro L. Usability and feasibility of an mHealth intervention for monitoring and managing pain symptoms in sickle cell disease: the sickle cell disease Mobile application to record symptoms via technology (SMART). Hemoglobin. (2015) 39(3):162–8. doi: 10.3109/03630269.2015.1025141

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Shah N, Jonassaint J, De Castro L. Patients welcome the sickle cell disease Mobile application to record symptoms via technology (SMART). Hemoglobin. (2014) 38(2):99–103. doi: 10.3109/03630269.2014.880716

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Lunyera J, Jonassaint C, Jonassaint J, Shah N. Attitudes of primary care physicians toward sickle cell disease care, guidelines, and comanaging hydroxyurea with a specialist. J Prim Care Community Health. (2017) 8(1):37–40. doi: 10.1177/2150131916662969

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Palmer M, Barnard S, Perel P, Free C. Mobile phone-based interventions for improving adherence to medication prescribed for the primary prevention of cardiovascular disease in adults. Cochrane Database Syst Rev. (2018) 6:CD012675.

38. Schnall R, Rojas M, Bakken S, Brown W, Carballo-Dieguez A, Carry M, et al. A user-centered model for designing consumer mobile health (mHealth) applications (apps). J Biomed Inform. (2016) 60:243–51. doi: 10.1016/j.jbi.2016.02.002

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Lyon AR, Brewer SK, Arean PA. Leveraging human-centered design to implement modern psychological science: return on an early investment. Am Psychol. (2020) 75(8):1067–79. doi: 10.1037/amp0000652

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Badawy SM, DiMartino L, Brambilla D, Klesges L, Baumann A, Burns E, et al. Impact of the COVID-19 pandemic on the implementation of Mobile health to improve the uptake of hydroxyurea in patients with sickle cell disease: mixed methods study. JMIR Form Res. (2022) 6(10):e41415. doi: 10.2196/41415

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Waltz TJ, Powell BJ, Matthieu MM, Damschroder LJ, Chinman MJ, Smith JL, et al. Use of concept mapping to characterize relationships among implementation strategies and assess their feasibility and importance: results from the expert recommendations for implementing change (ERIC) study. Implement Sci. (2015) 10:109. doi: 10.1186/s13012-015-0295-0

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, et al. A refined compilation of implementation strategies: results from the expert recommendations for implementing change (ERIC) project. Implement Sci. (2015) 10:21. doi: 10.1186/s13012-015-0209-1

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Cullen MR, Lemeshow AR, Russo LJ, Barnes DM, Ababio Y, Habtezion A. Disease-specific health disparities: a targeted review focusing on race and ethnicity. Healthcare. (2022) 10(4):603. doi: 10.3390/healthcare10040603

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. (2009) 4:50. doi: 10.1186/1748-5908-4-50

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. (1999) 89(9):1322–7. doi: 10.2105/AJPH.89.9.1322

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Shelton RC, Chambers DA, Glasgow RE. An extension of RE-AIM to enhance sustainability: addressing dynamic context and promoting health equity over time. Front Public Health. (2020) 8:134. doi: 10.3389/fpubh.2020.00134

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Forman J, Heisler M, Damschroder LJ, Kaselitz E, Kerr EA. Development and application of the RE-AIM QuEST mixed methods framework for program evaluation. Prev Med Rep. (2017) 6:322–8. doi: 10.1016/j.pmedr.2017.04.002

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Davis F. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q. (1989) 13(3):319–40. doi: 10.2307/249008

CrossRef Full Text | Google Scholar

49. Venkatesh V, Davis FD. A model of the antecedents of perceived ease of use: development and test*. Decision Sciences. (1996) 27(3):451–81. doi: 10.1111/j.1540-5915.1996.tb01822.x

CrossRef Full Text | Google Scholar

50. Liang L, Bernhardsson S, Vernooij RW, Armstrong MJ, Bussieres A, Brouwers MC, et al. Use of theory to plan or evaluate guideline implementation among physicians: a scoping review. Implement Sci. (2017) 12(1):26. doi: 10.1186/s13012-017-0557-0

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Davies P, Walker AE, Grimshaw JM. A systematic review of the use of theory in the design of guideline dissemination and implementation strategies and interpretation of the results of rigorous evaluations. Implement Sci. (2010) 5:14. doi: 10.1186/1748-5908-5-14

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Bennett LM, Gadlin H. Collaboration and team science: from theory to practice. J Investig Med. (2012) 60(5):768–75. doi: 10.2310/JIM.0b013e318250871d

PubMed Abstract | CrossRef Full Text | Google Scholar

53. King DK, Shoup JA, Raebel MA, Anderson CB, Wagner NM, Ritzwoller DP, et al. Planning for implementation success using RE-AIM and CFIR frameworks: a qualitative study. Front Public Health. (2020) 8:59. doi: 10.3389/fpubh.2020.00059

PubMed Abstract | CrossRef Full Text | Google Scholar

54. Damschroder LJ, Reardon CM, AuYoung M, Moin T, Datta SK, Sparks JB, et al. Implementation findings from a hybrid III implementation-effectiveness trial of the diabetes prevention program (DPP) in the veterans health administration (VHA). Implement Sci. (2017) 12(1):94. doi: 10.1186/s13012-017-0619-3

PubMed Abstract | CrossRef Full Text | Google Scholar

55. Calhoun C, Luo L, Baumann AA, Bauer A, Shen E, McKay V, et al. Transition for adolescents and young adults with sickle cell disease in a US midwest urban center: a multilevel perspective on barriers, facilitators, and future directions. J Pediatr Hematol Oncol. (2022) 44(5):e872–e80. doi: 10.1097/MPH.0000000000002322

PubMed Abstract | CrossRef Full Text | Google Scholar

56. Bello-Manga H, Haliru L, Ahmed KA, Tabari AM, Farouk BU, Bahago GY, et al. Primary prevention of stroke in children with sickle cell Anemia in Nigeria: protocol for a mixed methods implementation study in a community hospital. JMIR Res Protoc. (2022) 11(6):e37927. doi: 10.2196/37927

CrossRef Full Text | Google Scholar

57. Holtrop JS, Estabrooks PA, Gaglio B, Harden SM, Kessler RS, King DK, et al. Understanding and applying the RE-AIM framework: clarifications and resources. J Clin Transl Sci. (2021) 5(1):e126. doi: 10.1017/cts.2021.789

PubMed Abstract | CrossRef Full Text | Google Scholar