Implementation of Electronic Adherence Monitors and Associated Interventions for Routine HIV Antiretroviral Therapy in Uganda: Promising Findings

Background: High, sustained adherence is critical for achieving the individual and public health benefits of HIV antiretroviral therapy (ART). Electronic monitors provide detailed adherence information and can enable real-time interventions; however, their use to date has largely been confined to research. This pilot study (NCT03825952) sought to understand feasibility and acceptability a relatively low-cost version of this technology and associated interventions for routine ART delivery in sub-Saharan Africa.

Methods: We provided two ART clinics in rural, southwestern Uganda with electronic adherence monitors for data-informed counseling as well as optional SMS messages to clients and/or social supporters (daily or triggered by missed or delayed doses) and/or an alarm. Clinic and ART client experiences were observed for 3 months per client, including time and motion studies. Qualitative interviews among clients, clinicians, and healthcare administrators were informed by the Consolidated Framework for Implementation Research.

Results: Fifty-one ART clients were enrolled; 57% were male and the median age was 34 years. Choice of associated intervention varied among participants. The median number of visits during follow-up was two per client. Counselors reviewed the adherence data with 90% of clients at least once; 67% reviewed data at all visits. Average adherence was 94%; four clients had adherence gaps >1 week. Acceptability was high; all but one client found the monitor "very useful” and all found SMS “very useful.” Clinic visits among clients with the intervention lasted 4 min longer on average than those in standard care. The monitors and daily SMS generally functioned well, although excess SMS were triggered, primarily due to cellular network delays. Overall, participants felt the technology improved adherence, clinic experiences, and clinician-client relationships. Few worried about stigma and privacy. Cost was a concern for implementation, particularly at scale.

Conclusion: We successfully implemented a relatively low-cost electronic ART adherence monitor and associated interventions for routine care in rural Uganda. Feasibility and acceptability were generally high, and individuals were identified who could benefit from adherence support. Future work should involve longitudinal follow-up of diverse populations, clinical outcomes, and detailed cost-effectiveness analysis to help drive policy decisions around the uptake of this technology for routine clinical care.

Clinical Trial Registration: identifier: NCT03825952.

Introduction

High and sustained adherence to HIV antiretroviral therapy (ART) is well-known to play a central role in achieving viral suppression, which is critical for individual health benefits as well as prevention of secondary viral transmission (1, 2). Adherence monitoring provides valuable information on the effectiveness of ART as well as enables targeted use of limited resources for adherence interventions. In routine clinical care, adherence is most commonly measured through self-report, if assessed at all. While simple, inexpensive, and quick, it often underestimates non-adherence due to social desirability and recall bias (3). Other alternatives include pill counts, pharmacy refill tracking, pharmacokinetic measures, and electronic monitors, each with their own strengths and weakness (4–6).

Electronic adherence monitoring involves the use of a “smart” pill box that records a time-and-date stamp each time it is opened as a proxy for medication ingestion. This data can either be stored on the monitor for later downloading when a patient comes to clinic, or it can be transmitted in real-time through cellular networks, depending on the technology used. While limited by potential inaccuracies (i.e., opening the monitor without dosing or removal of multiple pills at a time for later dosing), these monitors uniquely provide a day-to-day record of adherence behavior. This information can be used to identify specific periods of non-adherence, which can be useful for tailoring adherence counseling and/or delivery of “just-in-time” adherence intervention. For example, real-time monitors can automatically trigger text message reminders if a dose is not taken within a defined time period and/or connections can be made to known social support systems (7). Alternatively, incentives can be delivered for demonstrated adherence (8, 9). Electronic adherence monitors have been shown to be acceptable and improve ART adherence in multiple, although not all, contexts (7–12); however, their use has largely been limited to research studies because of expense and concerns about impact on clinical operations. Because the cost of these devices has been reduced dramatically in recent years, they now have potential for use in routine clinical care, and study of their acceptability and impact on routine clinical experiences is warranted.

Implementation science aims to improve the uptake of evidence-based interventions through systematic assessment of potential barriers and facilitators (13). The Consolidated Framework for Implementation Research (CFIR) is helpful to understand the multiple constructs that can influence intervention adoption, namely the intervention itself (e.g., features, relative advantage, cost), the individual (e.g., knowledge, preferences), the inner setting (e.g., clinical structures, staffing, culture), the outer setting (e.g., healthcare policies and expectations), and the implementation process (e.g., rollout plans, leadership models) (14). We previously conducted a qualitative study with healthcare administrators and clinicians and ART clients that explored relevant factors for uptake of real-time electronic adherence monitors and associated interventions for routine HIV care in Uganda according to the CFIR (15). In brief, participants viewed the monitors and associated interventions favorably, reporting that they would be beneficial for supporting adherence and improving clinical outcomes. At the individual level, participants felt that a desire for good health and a welcomed pressure to adhere favored implementation of the technology, although some participants were worried that clients would not use the monitors as directed and that poverty, stigma, and privacy concerns might inhibit their use. Within the clinic setting, participants felt that the adherence data would likely improve the quality of counseling and thereby clinical staff morale, as well as increase the efficiency of care delivery (e.g., targeting counseling only for those who demonstrated adherence challenges). They emphasized the need for proper training on use of the technology. They also stated that community influences, international norms around the importance of supporting adherence, and availability of funding would be important outer setting considerations. Other relevant factors included existing infrastructure and care expenditures. Additionally, participants felt the implementation process would need to be guided by the Ministry of Health (MoH) and other funders with attention to sustainability, the appropriate target populations for its use, and coordination across the health care system.

Here, we present a pragmatic, follow-up study in which we used mixed methods to understand the experience of implementing relatively low-cost, real-time electronic adherence monitors and associated interventions in two clinics providing ART in rural southwestern Uganda. The first clinic has experience with research, whereas the other is prototypical of HIV care in the region. The objective was to understand the feasibility, acceptability, and practical implications of this technology with minimal support from our research team.

Methods

Intervention

We studied use of the evriMED electronic adherence monitor (Wisepill Technologies, South Africa), which transmits device opening data to a central server via cellular networks; data can be viewed as a digital display (e.g., adherence graphs) on a web browser or smart phone app (Figure 1). Multiple interventions can be associated with the monitors, such as data to inform counseling sessions in clinic. Short message service (SMS) reminders can also be sent to clients that may be one-way (i.e., an outgoing message) and/or two-way (i.e., invite a response with any questions). One- or two-way SMS notifications can also be sent to individuals identified as supporting the clients (e.g., family or friends; henceforth called social supporters). These SMS may be sent at a scheduled time or in response to a delayed or missed dose (e.g., 30 min after the anticipated dosing time). SMS were routed through a local technology solutions provider (Yo! Uganda Limited). Additionally, the device has an optional alarm that may be triggered at or before the anticipated dosing time.

FIGURE 1

Figure 1. The evriMED monitor and associated interventions. *SMS may be scheduled or triggered by a delayed or missed dose; they may also involve 1- or 2-way communication.

Study Design and Setting

In the pilot study presented here, we first implemented the evriMED monitor and associated interventions at the Kabwohe Clinical Research Centre (KCRC) in rural, southwestern Uganda. This clinic provides comprehensive primary healthcare services to >5,000 patients, including HIV care. It also has experience participating in research studies. We then assessed use of the monitors and associated interventions at the Kabwohe Health Centre IV Clinic (KHC-IV). This clinic is a more prototypic clinic for the region, focused primarily on routine care delivery for >2,000 patients.

Participants

ART clients were eligible for participation if they were 18 years or older, received HIV care at one of the two above-noted facilities, and owned a cell phone; those who had participated in the earlier formative research were eligible. Enrollment was stratified with a goal of equal representation of early (<6 months) vs. established (>6 months) treatment experience and rural vs. peri-urban residence. Potential participants were approached as a convenience sample within these strata. Enrolled ART clients were then asked to invite a social supporter who knew their HIV status and owned a cell phone.

Healthcare administrators and clinicians who had participated in the formative research (15) were eligible for the current study and represented the following cadres: MoH officials, regional referral hospital administrators, district health officers, healthcare administrators, physicians, nurses, and ART adherence counselors. If the original participants were not available, others in the same role were invited to participate. The only exclusion criterion for any type of participant was the inability to provide informed consent.

Study Procedures

Clinic Training

Consistent with the implementation goals of this study, Ugandan-based study staff (authors JT and ETu) led a training on use of the technology for each clinic in the manner of typical in-service trainings conducted by the Ugandan MoH. Research staff were then available to assist the clinical staff with operational issues (RB and JTu) and technical advice (JT and ETu).

Participant Data Collection

RB and JTu administered baseline questionnaires to enrolled ART clients and social supporters, using the local language (Runyankole) or English, as preferred. The questionnaires assessed the demographics and HIV-related clinical history of the ART clients and social supporters, as well as the ART client's structural barriers to clinic access (16), food insecurity (17), depression (18), alcohol use (19), beliefs about HIV, satisfaction with clinical care (20), and perceived HIV stigma (21). RB and JTu also administered an exit survey to ART clients to assess acceptability of the monitor and associated interventions. Questions focused on perceived ease of use, usefulness, problems experienced, and privacy and confidentiality concerns.

Monitor and Intervention Implementation

Clinic staff independently trained the ART clients and social supporters on the use of the monitor and asked them to select any (or none) of the available interventions. Participants were observed for a three-month period that included at least one routinely scheduled clinic visit according to Ugandan ART guidelines (22). Research staff only engaged with ART clients or their social supporters in the event of a technical problem with the monitors or associated interventions.

Implementation Metric Collection

RB, JTu, and JT collected implementation data, including the number of (1) devices used, (2) counseling sessions that occurred, and (3) functionality/technical problems experienced as feasibility metrics. They additionally conducted time and motion studies to assess the inner setting, noting which staff engaged with the technology and the time involved, comparing participants using vs. not using the monitoring and associated interventions.

Qualitative Interviews

After completion of the above-noted procedures at the first clinic (KCRC), JH, LG, and SA prepared a summary of the participant and clinic experiences that RB and JTu presented to a subset of the ART clients (flip chart format) and healthcare administrators and clinicians (text format; see Appendix 1a,b). ART clients were purposively selected to reflect roughly equal balance by gender and duration of experience with ART. Interviews were conducted in English or Runyankole in a private, quiet space. RB and JTu are both bilingual in English and Runyankole and trained in qualitative methods. They asked participants about views on the technology, perceived utility for routine care, concerns about implementation, recommendations for improvement, and perceptions of the intervention relative to other clinical programs or interventions (see Appendix 2). Interviews were conducted until thematic saturation was achieved.

Analysis

We summarized quantitative data descriptively. Adherence reflects doses taken over the time monitored and was censored at death or study exit. Costs reflect salaries according to the Ugandan MoH and time recorded for effort spent on training and implementation of the monitors, associated counseling, and SMS. Costs solely related to study-related activities are not presented.

We analyzed qualitative data using an inductive, content analysis approach (23). LG, MS, and JH read the first 20% of transcripts to identify relevant content. LG and MS then formulated codes based on this content and assembled a codebook that was piloted and ultimately applied to the complete dataset. LG and MS performed the coding independently and disagreements were resolved through discussion. They entered data into Dedoose (version 4.12) to support the category development process, which consisted of characterizing core concepts, developing labels to represent the concepts, writing operational definitions, and selecting illustrative quotes as evidence from the interviews.

Ethics

All participants provided written informed consent. This study was approved by the institutional review boards at the Mbarara University of Science and Technology, Ugandan National Council for Science and Technology, and Mass General Brigham. The study was registered with ClinicalTrials.gov (NCT03825952).

Results

Initial Training

The training at KCRC consisted of all 21 clinical staff who represented all cadres (i.e., nurses, counselors, doctors, pharmacists, and administrators). It was interactive and lasted approximately 3 h. The primary challenge involved accessing the internet on some of the clinical staff's phones, many of which were several years old and had damaged screens and/or keypads. During the session, it became apparent that focused training was needed for the two triage nurses identified to be responsible for use of the devices in routine clinical care. At KHC-IV, training involved an overview for all 11 clinical staff who represented all the above-noted cadres and lasted approximately 1 h. Two triage nurses underwent more intensive training for 3 h. Three of the four total triage nurses reported no difficulty with learning the procedures for registering ART clients and/or setting up the SMS interventions. One triage nurse required repeat training on proper phone number formatting and other data entry at registration.

Participant Characteristics

A total of 51 ART clients were enrolled, 25 at KCRC and 26 at KHC-IV. All individuals approached for enrollment agreed to participate. Retention was high with 94% (48/51) completing the three-month follow-up period. One ART client at the KCRC site died from HIV complications shortly after enrollment; in KHC-IV, one participant moved away shortly after enrollment and was replaced, and one ART client at KHC-IV was lost to follow-up midway through the study.

As presented in Table 1, most clients (29/51; 57%) were male with a mean age of 34 years. Sixty percent of participants had a primary education or less, but 98% (50/51) were literate in Runyankole. Over half (30/51; 59%) were married and most were employed (46/51; 90%). Duration of ART use was equally split (25 were <6 months; 26 were >6 months). Roughly one-third (16/51) were taking efavirenz-based ART; most others took dolutegravir-based ART. Most clients had good immune function (average recent CD4 count 381 cells/mm³). Most (88%; 45/51) had disclosed their HIV status, and stigma in the form of negative perceived attitudes and disclosure concerns was moderate. Few structural barriers to care were reported. Food insecurity and depression affected one-fifth (10/51) and one-third (16/51) of clients, respectively. Nearly half (45%; 23/51) had problematic alcohol use. Clinic satisfaction was generally high, although more so in KCRC.

TABLE 1

Table 1. ART client and social supporter characteristics.

All ART clients enrolled in the study with a social supporter. Notably, four participants at KHC-IV selected the same clinic volunteer as their social supporter. Sixty percent (29/48) of social supporters were women with a mean age of 39 years. Roughly one-quarter each were spouses/partners (13/48) and other family (12/48). Three-quarters (36/48) of the social supporters were also living with HIV, including all in KHC-IV. All social supporters living with HIV were taking ART.

Monitors and Associated Intervention Functionality

We attempted to import 35 devices; however, only 25 were obtained due to loss during the shipping process. Eight technical errors occurred. Three devices stopped working for unclear reasons, of which two resumed function after restarting and one had to be replaced. Two SIM cards had to be re-positioned, two devices were stolen, and one participant's alarm was activated in error. The three devices that were non-functional or stolen were replaced. Most batteries lasted the full 3-month study period, although two required additional charging. Choice of interventions associated with the adherence monitors is shown in Table 2. All clients chose to receive SMS reminder (daily and/or triggered), and most also chose to send an SMS notification to the social supporter and an alarm. All clients were able to contact the triage nurses or counselors as desired, although the frequency of these calls was not tracked.

TABLE 2

Table 2. Intervention selection, functionality, and adherence outcomes.

SMS functionality is also shown in Table 2. Most SMS were sent as expected, although many triggered SMS were sent unnecessarily, primarily due to poor cellular network (i.e., the device was opened on time but the server received that data >30 min late). The most common technical problem arose from the one triage nurse who had challenges with training; she made errors in entering the desired SMS schedule at enrollment, resulting in both missed days of monitoring and overlapping dosing schedules. Error rates were similar by site and type of recipient. Most participants (45/51; 88%) experienced some SMS errors. Two social supporters received daily SMS as an error for 33 days. An additional challenge occurred with “anti-SPAM” legislation that initially blocked all SMS. Clients, however, were able to voluntarily unblock the SMS.

Clinic Experience

In both clinics, the triage nurses and ART adherence counselors used their personal smart phones to access the adherence data (reports could not be printed). Additional targeted training was provided as needed over the course of participant follow-up for approximately 5.5 h total. Forty-three time and motion studies were conducted among study participants and other ART clients (17 in KCRC and 26 in KHC-IV) and indicated that triage nurses played a key role in using the devices in clinic; their review of the data determined who needed in depth counseling and who could skip it entirely. ART adherence counselors also used the data to inform counseling sessions; nurses and doctors did not. An additional 4 min on average were spent per client using the adherence monitoring intervention compared to routine care (7 and 1 min in each site, respectively).

ART Client Experience and Acceptability

Clients were seen according to their routine clinic schedule, which varied depending on their prior duration of treatment. The median number of visits was two per client, ranging from one to three. At study exit, most (90%; 43/48) clients reported that their counselors reviewed the monitor data with them during one or more routine visits; 32/48 (67%) reviewed the data at all visits. The most common reason for not reviewing the data was poor internet connectivity, followed by time. As shown in Table 2, overall average adherence during the 3 months of follow-up was 94%; however, four clients had gaps in adherence of >7 days (three in KCRC and one in KHC-IV).

As indicated in Table 3, acceptability was high with all but one client reporting that the monitor and data used for counseling were very useful; 92% (44/48) also found the SMS very useful. Nearly all participants reported no problems with the monitor or the SMS. Thirteen percent (6/48) of clients were worried about others seeing the monitor, while 4% (2/48) were worried about someone seeing the SMS. One participant reported a problem with storing the monitor.

TABLE 3

Table 3. Acceptability of the monitors and associated interventions.

Cost

One-time costs included the 28 monitors used in the study, ($25 each) plus license fees, SIM cards, battery chargers, and shipping, totaling $2,170. On-going costs included SMS and data hosting fees of $1,020 per year. An additional $438 was spent on the initial training, accounting for staff time and refreshments, plus $3,190 for technical support (160 h at approximately $20 per hour) over the course of the study. If costs were extended for an anticipated 3-year lifetime for the device and assuming technical support needs would decrease by 75% in subsequent years, they would average a total of $139 per client per year.

Perceptions of Implementation

After reviewing the summary of participant and clinic experiences from KCRC (Appendix 1a,b), we interviewed 10 ART clients, of whom six were men and four women; six had taken ART for >6 months, while four had taken ART for <6 months. We also interviewed 19 healthcare administrators and clinicians, consisting of the following cadres: four MoH officials, two district health officers, two healthcare administrators, three physicians, two clinic officers, three nurses, and three adherence counselors. Reflections are presented here by CFIR domain (see Table 4).

TABLE 4

Table 4. Participant quotations from qualitative interviews according to the CFIR.

Intervention

Participants generally liked the monitors and SMS and felt they improved adherence through desired accountability. Although opinions differed on the accuracy of the data and potential for using the devices as expected, participants generally agreed that the objective information added value for clinical care.

Some participants had concerns about the technical function of the monitors and SMS, including faulty batteries. They were worried that poor cellular network would prevent reminders from coming as expected. Participants suggested modifications, including voice recordings and/or images for those with low literacy and modifying the monitor to track the number of pills taken out.

Clinic Setting

Participants reported that the adherence data qualitatively enhanced the conversations between adherence counselors and clients. For example, data demonstrating patterns of adherence behaviors allowed for more focused counseling conversations. Clinic staff also noted that knowing the data ahead of time allowed them to provide better care through longer counseling when needed.

Healthcare administrators and clinicians felt the intervention had other logistical benefits as well. The automated reminder messages reduced the time they spent calling or visiting ART clients, lightening their burden. They stated that the monitors additionally improved documentation and record-keeping, thus providing logistical benefits to the clinic and reduced costs.

Participants did not express concerns about the monitor data delaying them in clinic; however, opinions differed on the need for additional staffing and resources, including phones for professional use, technical support, and counselors.

Individual

Nearly all participants emphasized that the monitors and system of reminders built better adherence habits, while simultaneously reducing worries about defaulting on treatment. They also felt the intervention reduced the burden on both ART clients and their social supporters. Many saw the data as building trust within the client-counselor relationship, placing them on the same team and improving counselor morale. ART clients liked the technology, because it enabled them to demonstrate high adherence and thereby please their clinicians.

Despite anticipated concerns about the influence of social determinants of health in the formative interviews, relatively few participants mentioned poverty, stigma, and privacy. Most felt these issues could be managed through careful program implementation and individualized strategies.

Outer Setting

The most prevalent concerns from the outer setting were related to the cost implications of the technology, which could be positive or negative. Healthcare administrators and clinicians wanted to know the cost-benefit ratio, including impact on resources and staffing. These factors would guide public policy decisions regarding uptake of the monitors and associated interventions.

In deciding whether or not to implement this technology, some participants wanted a large, rigorous, study in multiple parts of the country with clinical outcomes, objective measures, and a comparison to standard counseling. Others wanted more qualitative evidence from communities of people living with HIV, and still others felt this pilot was adequate.

Process

Nearly all participants stated that the MoH would need to support and fund the implementation process. One healthcare administrator emphasized the need for a platform to translate research findings into practice. Healthcare administrators and clinicians stressed the importance of initial and ongoing in-service training to support proper use of the intervention, although some felt staff would adopt it readily.

Beliefs on the appropriate target populations for the intervention varied widely, ranging from people living with HIV who failed first-line ART to members of key populations to anyone with chronic disease. Some ART clients also recommended using it for short time periods to establish the habit of adherence, while other ART clients and clinicians recommended people might benefit from them over their whole lives.

Discussion

This pilot mixed methods study presents the pragmatic experience of implementing relatively low cost, real-time electronic adherence monitors and associated interventions for routine ART delivery in rural southwestern Uganda. Implementation was accomplished by clinic staff with minimal support from the study team. Feasibility was generally high with few, mostly temporary problems with the adherence monitor functionality. SMS reminders to ART clients and social supporters were transmitted reliably when sent daily; however, limitations in cellular network and some system errors resulted in SMS being triggered unnecessarily. Acceptability for both the monitors and SMS was very high with a minority expressing concerns about privacy. Perceptions of healthcare administrators, clinicians, and ART clients about implementation of the technology were generally favorable. They felt it had a positive influence on adherence, optimal clinic flow, and relationships in the clinical setting. Participants emphasized the importance of demonstrated benefit relative to cost and the need for buy-in from the MoH for actual implementation. Opinions differed about on-going support needs and target populations.

Although this study was small, the setting, participants, and types of ART were largely typical for HIV care in rural sub-Saharan Africa; our findings thus provide important insights into the use of this type of intervention in routine care. Moreover, experiences were generally similar in the two clinics regardless of prior experience with research. Well-known adherence challenges, including food insecurity, depression, alcohol use, and stigma, were present in the study population, indicating their potential need for adherence monitoring and support. Importantly, the relatively low levels of education did not seem to present a barrier to acceptability and use of the technology, and social determinants of health (e.g., poverty) were not prominent concerns in our qualitative interviews. That said, reported barriers to care were relatively low, and experiences for people with HIV who are less engaged in care may differ. Notably, the majority of participants in our study were men, while women comprise the majority of people living with HIV in sub-Saharan Africa. Additional studies in diverse settings and populations will therefore be important for understanding the potential of this technology broadly.

Both clinics chose to implement the technology among a small number of clinical staff (i.e., triage nurses and ART adherence counselors) who have more time and availability relative to physicians and nurses. Given the healthcare worker shortages across sub-Saharan Africa (24), the central role played by the triage nurses and adherence counselors bodes well for scalability of this technology. That said, the challenges in training one triage nurse and the subsequent impact on SMS errors suggest the need for careful skill assessment and/or a simpler user interface with the technology. Additionally, the impact on length of visits was minimal, and participants commented favorably on the potential of data-informed counseling to improve the time spent in clinic, including type of counseling and relationships between ART clients and clinic staff. These features are consistent with differentiated care models, which have been shown to improve clinical care (25). Notably, none of the participants described the monitors or associated interventions as disruptive or a distraction, as has been seen with other forms of technology used for adherence monitoring, such as cell phone-based self-report for the treatment of tuberculosis (26). This combination of simple, “smart” pill boxes for data-informed counseling and options of SMS and/or alarm support appears to have potential for improved overall patient-oriented clinical experiences.

Our study focused on assessment of implementing real-time adherence monitoring and associated interventions, rather than the effects on ART adherence itself which have been published in the literature (7–12). Nonetheless, we did identify potentially clinically meaningful episodes (i.e., >7 days) of non-adherence among four of the 51 study participants (8%) over just 3 months of follow-up per participant. Although these participants had generally high overall adherence, the risk for loss of viral suppression has been shown to increase with each consecutive day of non-adherence (27). The ability to detect these sustained gaps, trigger interventions, and tailor adherence counseling suggests potential for impact on long-term adherence, sustained viral suppression, and thereby use of low-cost, first-line ART.

Cost is a major consideration when assessing interventions for routine clinical care, particularly in light of limited resources. Indeed, participants indicated the cost-benefit ratio would be a main driver of the decision to take up this technology. The costing in this study was limited and could not evaluate cost-effectiveness; however, we tracked the primary costs incurred with clinical use of the technology. Although the monitors were only $25 USD each, additional one-time and on-going fees resulted in an estimated cost of $139 per client per year. An analysis of potential ART adherence monitoring interventions for sub-Saharan Africa found that up to $50 per person-year could be cost-effective, primarily based on differentiation of care and varying based on the availability of viral load testing (28). Although our estimates exceed this threshold, many costs would reduce considerably with economies of scale and cost-effectiveness should be explored in future studies involving high volume clinics.

This study has important limitations. Principally, as noted above, the study was small in scope and focused on short-term feasibility, acceptability, and perceptions of implementation. That said, the engagement of two clinics reflective of ART care in rural, sub-Saharan Africa with minimal support from research staff is a major strength. Additionally, our use of the CFIR provides broad opinions across the healthcare system, which are important for considering potential effects.

In sum, this study shows promise for the implementation of relatively low cost, real-time electronic adherence monitors and associated interventions for ART delivery in sub-Saharan Africa. Feasibility and acceptability were high with favorable impressions of impact on adherence and clinic experiences. Future work should involve longitudinal follow-up of diverse populations, clinical outcomes, and detailed cost-effectiveness analysis to help drive policy decisions around the uptake of this technology for routine clinical care.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved by the institutional review boards at the Mbarara University of Science and Technology, Ugandan National Council for Science and Technology, and Mass General Brigham. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

JH and SA conceived of the study. RB and JTu collected data for the study. JTi and ETu provided technical oversight. LG and ETi provided program management. LM supported study implementation. JH, LG, and MD conducted the qualitative analysis, while JH and NM conducted the quantitative analysis. JH wrote the first draft of the manuscript. All authors edited and approved the final manuscript.

Funding

This study was funded by the US National Institute of Mental Health (K24MH114732). Neither the funder, nor the manufacturer of the technology studied (Wisepill Technologies), had any influence on the design, implementation, or data interpretation in this study.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

We would like to thank the study participants and clinical staff at the Kabwohe Clinical Research Centre and Kabwohe Health Centre IV Clinic.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fdgth.2022.899643/full#supplementary-material

References

1. Conway B. The role of adherence to antiretroviral therapy in the management of HIV infection. J Acquir Immune Defic Syndr. (2007) 45(Suppl. 1):S14–8. doi: 10.1097/QAI.0b013e3180600766

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Cohen MS, Chen YQ, McCauley M, Gamble T, Hosseinipour MC, Kumarasamy N, et al. Prevention of HIV-1 infection with early antiretroviral therapy. N Engl J Med. (2011) 365:493–505. doi: 10.1056/NEJMoa1105243

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Kagee A, Nel A. Assessing the association between self-report items for HIV pill adherence and biological measures. AIDS Care. (2012) 24:1448–52. doi: 10.1080/09540121.2012.687816

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Okatch H, Beiter K, Eby J, Chapman J, Marukutira T, Tshume O, et al. Brief report: apparent antiretroviral overadherence by pill count is associated with HIV treatment failure in adolescents. J Acquir Immune Defic Syndr. (2016) 72:542–5. doi: 10.1097/QAI.0000000000000994

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Sangeda RZ, Mosha F, Prosperi M, Aboud S, Vercauteren J, Camacho RJ, et al. Pharmacy refill adherence outperforms self-reported methods in predicting HIV therapy outcome in resource-limited settings. BMC Public Health. (2014) 14:1035. doi: 10.1186/1471-2458-14-1035

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Spinelli MA, Haberer JE, Chai PR, Castillo-Mancilla J, Anderson PL, Gandhi M. Approaches to objectively measure antiretroviral medication adherence and drive adherence interventions. Curr HIV/AIDS Rep. (2020) 17:301–14. doi: 10.1007/s11904-020-00502-5

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Haberer JE, Musiimenta A, Atukunda EC, Musinguzi N, Wyatt MA, Ware NC, et al. Short message service (SMS) reminders and real-time adherence monitoring improve antiretroviral therapy adherence in rural Uganda. Aids. (2016) 30:1295–300. doi: 10.1097/QAD.0000000000001021

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Linnemayr S, Stecher C, Mukasa B. Behavioral economic incentives to improve adherence to antiretroviral medication. Aids. (2017) 31:719–26. doi: 10.1097/QAD.0000000000001387

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Bien-Gund CH, Ho JI, Bair EF, Marcus N, Choi RJ, Szep Z, et al. Brief report: financial incentives and real-time adherence monitoring to promote daily adherence to HIV treatment and viral suppression among people living with HIV: a pilot study. J Acquir Immune Defic Syndr. (2021) 87:688–92. doi: 10.1097/QAI.0000000000002628

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Orrell C, Cohen K, Mauff K, Bangsberg DR, Maartens G, Wood R, et al. Randomized controlled trial of real-time electronic adherence monitoring with text message dosing reminders in people starting first-line antiretroviral therapy. J Acquir Immune Defic Syndr. (2015) 70:495–502. doi: 10.1097/QAI.0000000000000770

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Sabin L, DeSilva MB, Gill CJ, Zhong L, Vian T, Xie W, et al. Improving adherence to antiretroviral therapy with triggered real-time text message reminders: the china adherence through technology study. J Acquir Immune Defic Syndr. (2015) 69:551–9. doi: 10.1097/QAI.0000000000000651

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Stringer KL, Azuero A, Ott C, Psaros C, Jagielski CH, Safren SA, et al. Feasibility and acceptability of real-time antiretroviral adherence monitoring among depressed women living with hiv in the deep South of the US. AIDS Behav. (2019) 23:1306–14. doi: 10.1007/s10461-018-2322-z

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Bauer MS, Damschroder L, Hagedorn H, Smith J, Kilbourne AM. An introduction to implementation science for the non-specialist. BMC Psychology. (2015) 3:32. doi: 10.1186/s40359-015-0089-9

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. (2009) 4:50. doi: 10.1186/1748-5908-4-50

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Haberer JE, Garrison L, Tumuhairwe JB, Baijuka R, Tindimwebwa E, Tinkamanyire J, et al. Factors affecting the implementation of electronic antiretroviral therapy adherence monitoring and associated interventions for routine HIV care in Uganda: qualitative study. J Med Internet Res. (2020) 22:e18038. doi: 10.2196/18038

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Coetzee B, Kagee A. The development of an inventory to assess the structural barriers to clinic attendance and pill-taking amongst users of antiretroviral therapy. AIDS Behav. (2013) 17:319–28. doi: 10.1007/s10461-012-0374-z

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Tsai AC, Bangsberg DR, Emenyonu N, Senkungu JK, Martin JN, Weiser SD. The social context of food insecurity among persons living with HIV/AIDS in rural Uganda. Soc Sci Med. (2011) 73:1717–24. doi: 10.1016/j.socscimed.2011.09.026

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Monahan PO Shacham E Reece M Kroenke K Ong'or WO Omollo O . Validity/reliability of PHQ-9 and PHQ-2 depression scales among adults living with HIV/AIDS in Western Kenya. J Gen Intern Med. (2009) 24:189–97. doi: 10.1007/s11606-008-0846-z

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Cherpitel CJ, Ye Y, Bond J, Borges G, Cremonte M, Marais S, et al. Cross-national performance of the RAPS4/RAPS4-QF for tolerance and heavy drinking: data from 13 countries. J Stud Alcohol. (2005) 66:428–32. doi: 10.15288/jsa.2005.66.428

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Babikako HM, Neuhauser D, Katamba A, Mupere E. Patient satisfaction, feasibility and reliability of satisfaction questionnaire among patients with pulmonary tuberculosis in urban Uganda: a cross-sectional study. Health Res Policy Syst. (2011) 9:6. doi: 10.1186/1478-4505-9-6

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Jeyaseelan L, Kumar S, Mohanraj R, Rebekah G, Rao D, Manhart LE. Assessing HIV/AIDS stigma in south India: validation and abridgement of the Berger HIV Stigma scale. AIDS Behav. (2013) 17:434–43. doi: 10.1007/s10461-011-0128-3

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Ministry of Health. Consolidated Guidelines for Prevention and Treatment of HIV in Uganda. Kampala (2020). Available online at: http://library.health.go.ug/publications/hivaids/consolidated-guidelines-prevention-and-treatment-hiv-uganda. (accessed July 6, 2022).

Google Scholar

23. Patton MQ. Qualitative Research and Evaluation Methods. 4th ed. Los Angeles: SAGE Publications, Inc. (2015).

Google Scholar

24. Oleribe OO, Momoh J, Uzochukwu BS, Mbofana F, Adebiyi A, Barbera T, et al. Identifying key challenges facing healthcare systems in Africa and potential solutions. Int J Gen Med. (2019) 12:395–403. doi: 10.2147/IJGM.S223882

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Barker C, Dutta A, Klein K. Can differentiated care models solve the crisis in HIV treatment financing? Analysis of prospects for 38 countries in sub-Saharan Africa. J Int AIDS Soc. (2017) 20:21648. doi: 10.7448/IAS.20.5.21648

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Thomas BE, Kumar JV, Onongaya C, Bhatt SN, Galivanche A, Periyasamy M, et al. Explaining differences in the acceptability of 99DOTS, a cell phone-based strategy for monitoring adherence to tuberculosis medications: qualitative study of patients and health care providers. JMIR Mhealth Uhealth. (2020) 8:e16634. doi: 10.2196/16634

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Haberer JE, Musinguzi N, Boum Y 2nd, Siedner MJ, Mocello AR, Hunt PW, et al. Duration of antiretroviral therapy adherence interruption is associated with risk of virologic rebound as determined by real-time adherence monitoring in rural Uganda. J Acquir Immune Defic Syndr. (2015) 70:386–92. doi: 10.1097/QAI.0000000000000737

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Phillips AN, Cambiano V, Nakagawa F, Bansi-Matharu L, Sow PS, Ehrenkranz P, et al. Cost effectiveness of potential ART adherence monitoring interventions in sub-Saharan Africa. PLoS ONE. (2016) 11:e0167654. doi: 10.1371/journal.pone.0167654

PubMed Abstract | CrossRef Full Text | Google Scholar