Virginia T. LeBaron1*
Bethany J. Horton2
Abish Adhikari3
Sandhya Chapagain4Manita Dhakal5Rajesh Gongal6
Regina Kattel7Ganesh Koirala6Anna Kutcher1Ben Hass8Martha Maurer9Daniel Munday10Bijay Neupane5
Krishna Sharma5Ramila Shilpakar4Amuna Shrestha7Sudip Shrestha7Usha Thapa5Rebecca Dillingham2,11Bishnu D. Paudel4- 1University of Virginia School of Nursing, Charlottesville, VA, United States
- 2University of Virginia School of Medicine, Charlottesville, VA, United States
- 3Kathmandu Cancer Center, Tathali, Nepal
- 4National Academy of Medical Sciences, Bir Hospital, Kathmandu, Nepal
- 5B.P. Koirala Memorial Cancer Hospital, Bharatpur, Nepal
- 6Hospice Nepal, Kathmandu, Nepal
- 7Nepal Cancer Hospital & Research Center, Lalitpur, Nepal
- 8Hass Software Consulting, Brooklyn, NY, United States
- 9Sonderegger Research Center, University of Wisconsin School of Pharmacy, Madison, WI, United States
- 10Usher Institute, University of Edinburgh, Edinburgh, United Kingdom
- 11University of Virginia Center for Global Health Equity, Charlottesville, VA, United States
Introduction: Quality palliative care, which prioritizes comfort and symptom control, can reduce global suffering from non-communicable diseases, such as cancer. To address this need, the Nepalese Association of Palliative Care (NAPCare) created pain management guidelines (PMG) to support healthcare providers in assessing and treating serious pain. The NAPCare PMG are grounded in World Health Organization best practices but adapted for the cultural and resource context of Nepal. Wider adoption of the NAPCare PMG has been limited due to distribution of the guidelines as paper booklets.
Methods: Building on a long-standing partnership between clinicians and researchers in the US and Nepal, the NAPCare PMG mobile application (“app”) was collaboratively designed. Healthcare providers in Nepal were recruited to pilot test the app using patient case studies. Then, participants completed a Qualtrics survey to evaluate the app which included the System Usability Scale (SUS) and selected items from the Mobile App Rating Scale (MARS). Descriptive and summary statistics were calculated and compared across institutions and roles. Regression analyses to explore relationships (α = 0.05) between selected demographic variables and SUS and MARS scores were also conducted.
Results: Ninety eight healthcare providers (n = 98) pilot tested the NAPCare PMG app. Overall, across institutions and roles, the app received an SUS score of 76.0 (a score > 68 is considered above average) and a MARS score of 4.10 (on a scale of 1 = poor, 5 = excellent). 89.8% (n = 88) “agreed” or “strongly agreed” that the app will help them better manage cancer pain. Age, years of experience, and training in palliative care were significant in predicting SUS scores (p-values, 0.0124, 0.0371, and 0.0189, respectively); institution was significant in predicting MARS scores (p = 0.0030).
Conclusion: The NAPCare PMG mobile app was well-received, and participants rated it highly on both the SUS and MARS. Regression analyses suggest end-user variables important to consider in designing and evaluating mobile apps in lower resourced settings. Our app design and pilot testing process illustrate the benefits of cross global collaborations to build research capacity and generate knowledge within the local context.
Introduction
An estimated 60 million people in the world have unmet palliative care needs, the majority of whom live in low and middle-income countries (LMICs) (1). Delivering quality palliative care, which prioritizes comfort and symptom control—especially optimal pain management—can reduce global suffering from non-communicable diseases (NCDs), such as cancer (2–5). For the purposes of this paper palliative care is broadly defined as holistic care provided to patients (and their family caregivers) with serious illness that has comfort and symptom control as the primary goal. Palliative care is particularly important in LMICs, where the cancer burden is rapidly growing, and where the majority of patients are first diagnosed with late stage cancer, which has a poor prognosis and is commonly accompanied by serious pain (6–8). Effectively managing physical pain is a foundational principle of palliative care (9), yet pain remains one of the most common, and most feared, symptoms of cancer (10–13). The World Health Organization (WHO) estimates that over 5.5 million people worldwide receive no, or minimal, treatment for their cancer pain (5, 14). In 2014 the World Health Assembly passed the landmark Palliative Care Resolution, which officially urged Member States to integrate palliative care in to national healthcare systems (15). Basic palliative care and pain relief has also been declared a universal human right (16–19). Despite these imperatives, a 2015 survey of 21 healthcare institutions in Nepal revealed that 83% of Nepal's population does not receive adequate palliative care (20).
Multiple professional organizations have created pain management guidelines (PMG) to address knowledge gaps and help healthcare providers (HCPs) effectively manage cancer pain (21–25). Unfortunately, despite evidence suggesting clear benefits, adherence to PMG remains low (26–29), and understanding contextual barriers to low adherence of PMG is limited, especially in LMICs (11, 27). One challenge is that cancer care guidelines are often “imported” from higher-resource, Western-oriented settings and may not translate well to LMIC settings. To address these challenges, it is critical that guidelines are designed and created that are relevant for the LMIC context. Once culturally and contextually relevant guidelines are created, leveraging Mobile Health (“mHealth”)—the use of mobile/wireless technology to improve healthcare—may enhance PMG implementation and adherence (30, 31). One viable approach to improve adherence to PMG and cancer pain care is via Mobile Health (“mHealth”) (32). mHealth (a subfield of electronic or “e-health”) involves leveraging mobile/wireless technology to improve healthcare (33). mHealth interventions range from 1-way text message alerts to mobile applications or “apps”—software programs downloaded to a mobile device (e.g., smart phone). A benefit of apps is that they are dynamic and portable, and can be accessed wherever, and whenever, the user desires (34). mHealth is increasingly viewed as a viable strategy in LMICs due to exploding telecommunication network capacity and decreasing cost of mobile devices (33, 35–41). For example, 73% of the 5.3 billion global mobile phone users are located in LMICs, with increasing penetration in remote rural areas (37). There is growing evidence that mHealth in LMICs can increase access to care (30, 33), support frontline healthcare workers (37, 39, 42, 43), enhance data collection (31, 44), and improve patient outcomes (45–47), but finding ways to evaluate effectiveness can be challenging (48, 49).
Nepal is a LMIC with a growing cancer burden and is well-poised to engage in collaborative palliative care and mHealth research. NCDs, including cancer, account for almost 60% of deaths in Nepal and are considered a major public health issue (50–55). In 2012, an estimated 58,000 adults needed palliative care in Nepal; approximately 20% were adults with cancer. These are likely underestimates, however, as Nepal only recently developed a population-based national cancer registry program (56–58), and does not yet have a national cancer control plan (56). Nepal does, however, have a national multisectoral NCD plan that recognizes the need for improved cancer services, including palliative care (59). The Nepal NCD plan also specifically calls to increase research capacity by strengthening infrastructure and training investigators (56, 59). A challenge in palliative care, particularly in LMICs, is generating and implementing evidence based research (60–63). Tremendous progress has been made over the past 5–10 years in Nepal to increase societal and healthcare provider (HCP) awareness regarding palliative care, to increase opioid availability for pain relief, and to understand the scope of palliative care needs (20, 56, 64, 65). However, gaps remain related to the infrastructure needed to conduct rigorous research. Research has been proposed as the fifth pillar of the WHO Public Health Strategy for Palliative Care (66), yet little palliative care research is conducted in LMICs (60, 62, 63, 66, 67).
This research represents a history of collaborative partnerships and successful initiatives undertaken in Nepal to improve palliative care and pain relief. Importantly, it also leverages foundational in-country work of The Nepalese Association of Palliative Care (NAPCare). NAPCare, a non-political, non-government organization, was founded in 2009 by community advocates and clinicians to improve access to palliative care (68). NAPCare has worked tirelessly to advance palliative care services and improve availability of essential pain medicines for patients with cancer (20, 64, 69), including the creation of Palliative Care Pain Management Guidelines (PMG) (70). First drafted in 2011, and updated in 2017, the NAPCare PMG are based on the WHO analgesic ladder (71) and adapted to the Nepal context. The NAPCare PMG have been distributed as paper booklets and designed to guide clinicians in appropriate cancer pain management. To date, implementation of the PMG has been limited.
With funding support from the National Institutes of Health, Fogarty International Center, US researchers partnered with investigators in Nepal to co-create and pilot test the “NAPCare PMG mobile health app,” which transformed the original paper-based PMG into a mobile app for use by healthcare providers (HCPs) (72). Our interdisciplinary team representing nursing, medicine, and social work consisted of clinicians and researchers from Nepal, United Kingdom, and the United States with expertise in oncology/palliative care, pain management, global health, and development and testing of mobile applications to improve health outcomes. Importantly, this research leveraged the enthusiasm, commitment, existing palliative care clinical services, and outstanding human capital within Nepal to conduct quality research that can inform and advance the evidence base regarding mHealth. The broad, overarching goal of this work is to support healthcare providers in delivering quality cancer pain care in LMICs, build research capacity within Nepal, and, ultimately, decrease patient suffering related to under or untreated cancer pain. A key objective of this paper is to present our collaborative process of mobile app development in sufficient detail so that it may be helpful to other researchers engaging in similar global work.
Materials and Methods
Overall Design
This was a feasibility and acceptability study grounded in a Community Based Participatory Research (CBPR) philosophy to develop and pilot test a mHealth decision support application “app” to promote PMG implementation within Nepal. CBPR is a collaborative research approach that involves engaging community stakeholders as members of the research team to implement relevant, sustainable change (73–76) and is a constructive approach in LMICs and can bridge knowledge and action, give voice to marginalized communities, and reduce health inequities (73–76). Core CBPR principles relevant to this research include engaging local stakeholders in establishing research priorities and the design of the project, recognizing and capitalizing on local expertise, sharing collective knowledge to work toward positive change, and promoting leadership growth and autonomy within the community (77). CBPR is a well-established research method and advocated as a critical approach to successful mHealth initiatives (33, 78–80).
Design of the NAPCare PMG Mobile Application
The design and pilot testing of the NAPCare PMG mobile app was a highly iterative, collaborative process between the Nepal and UVA research team facilitated by in-country fieldwork and remote virtual meetings and focus groups that occurred between November 2019—February 2021. Our initial fieldwork prompted important questions related to the scope, intent, and distribution of the mobile app, which were critical to discuss at the onset to clarify expectations. For example, there was significant interest by Nepal team members for the app to facilitate communication across healthcare providers within an institution, track patients over time, and link app results with existing clinical information documented in the medical record. However, the institutions selected for pilot testing generally lacked an existing electronic health record and building out such an infrastructure was well beyond the scope of this feasibility study. Key considerations, questions and challenges related to the NAPCare mobile app design and development are summarized in Table 1.
Table 1. Key considerations related to the NAPCare mobile app design and development.
The app design process was grounded in an Information Systems Research (ISR) framework, which advocates an iterative and collaborative process, focused on user needs (82). Our first step was to share the paper version of the NAPCare PMG with the engineering team to familiarize them with the document and discuss the goals and intent of the app. Then, a first draft of wireframes and high-fidelity app screen mock-ups were created and shared with study team members over Zoom and via Google docs to gather feedback and make iterative changes. In January 2020, an interactive, in-person workshop was held in Nepal to discuss app development. The focus of these discussions was to get the “foundation of the house” right (i.e., ensure the underlying architecture of the app is accurate) versus worrying too much about “what color we are painting the walls” (i.e., aesthetic details that are relatively easy to change). We discussed and clarified important topics, such as how to balance a thorough pain assessment with an easy, quick mobile app user interface recognizing that “we could build a ‘great' app but if no one uses it, it's useless.” We also clarified the intended audience for the app (nurses and physicians) and patient population (adult patients with cancer pain). Team members also wrote down on sticky notes “need,” “nice” and “next” preferences for the app. An example of the “need, nice, next” exercise was given to start: “I need a new motorcycle. A “need” is that it runs and can get me from point A to point B. A “nice” is that is has good gas mileage or that the color is red. A “next” is that it has Bluetooth capability so I can listen to my music while I ride.” Team members then put their sticky notes up on the wall in the “need” “nice” and “next” columns. The lists were read and discussed to reach group consensus regarding which preferences belonged in which columns and to inform app design (Table 2).
Table 2. Desired features and functionality of the NAPCare mobile app as identified by the Nepal team.
A next step was to convert the NAPCare PMG into pre-defined processes that could inform the design and programming of the mobile app (Figure 1). Working collaboratively, the Nepal and UVA team iterated various flowcharts of pre-defined processes related to the NAPCare PMG. During the in-person field visit to Nepal, members of the UVA team printed out large, 3′ × 6′ colored, vinyl posters of the most updated pre-defined process flowcharts. The posters were displayed, and team members circulated around the posters and used different colored sticky notes to make suggestions and comments directly on the decision flowchart posters. We validated and finalized these processes by testing various simulated patient case scenarios to ensure the pre-defined process algorithms produced the correct results. Specifically, Nepal team members brainstormed 5 common pain scenarios they see in clinical practice and worked through these cases on the decision flowchart posters. Each clinical decision point in the pain assessment and treatment process was marked with a sticky note. This was a very helpful exercise and allowed the team to identify problems with flow in the app, or decisions that needed to come earlier, later or were unnecessary (Figure 2).
Figure 1. An example of a pre-defined process flowchart used to inform the NAPCare PMG app design.
Figure 2. Discussing pre-defined process flowcharts to inform app design with the Nepal team during a fieldsite visit.
After the in-person workshop in Nepal, additional rounds of iterative feedback were conducted remotely using GoogleDocs to finalize the app decision flowcharts and create high-fidelity mock-ups of the various app screens. The mock-ups were reviewed, discussed, and iteratively revised during regular, collaborative meetings between the Nepal and UVA teams held over Zoom. After reaching consensus, the wireframes were then used to program the actual mobile app on the Android OS platform using the Kotlin programming language. This step included determining the core underlying algorithm to classify the type of pain and recommend treatment options, based on the NAPCare PMG. Developing the algorithm was complex and done in consultation with pain experts within our team. Finally, we conducted multiple rounds of internal testing within our study team, as well as external pre-testing of the app with a small number of nurses/physicians in Nepal (n = 8) not directly involved with app design and development. A selection of screenshots of the current beta NAPCare PMG app 1.0 (with sample data) are displayed below (Figure 3) to provide a sense for the design and user interface of the app. The NAPCare PMG app is currently available on the Google Play Store as a privately available (not public) app to download for beta testing within our team. The key steps and timeline of our design process are summarized in Supplementary Table 1.
Figure 3. Selected screenshots of the beta version of the NAPCare PMG mobile health app 1.0.
Importantly, design of the beta version of the app was informed by a comprehensive survey of oncology healthcare providers from four diverse cancer care institutions within Nepal (the same four sites where we later pilot tested the app). The goal of this survey was to better understand barriers and facilitators to cancer pain management and to gather end-user feedback about design of the mobile app (81). For example, a finding from our survey was that a key barrier to pain management was a high volume of patients. Therefore, we prioritized an app design that optimized drop-down menu choices that are quick and fast to use. Primary results from this survey that informed the design of the NAPCare PMG mobile app are presented in Table 3.
Table 3. Selected survey results that informed design of the mobile app.
Pilot Testing of the Mobile Application
Setting and Sample
After design of the mobile app, we conducted feasibility and acceptability testing of the NAPCare PMG mobile app with selected oncology healthcare providers using simulated patient case scenarios developed by the Nepal team. The final beta version of the app was deployed on Android study phones and a purposive quota sample of healthcare providers were recruited from four diverse study sites within Nepal (a public cancer hospital; a private cancer hospital; a public general hospital; and a hospice) to pilot test the app. These were the same 4 sites that also provided survey data that helped inform design of the app (81). The 4 study sites were carefully selected as each institution cares for adult patients with cancer and employed a key stakeholder of our study team affiliated with NAPCare, and also represented a diverse patient population and context. For example, private sector hospitals within Nepal generally care for a higher-income patient population and have enhanced resource availability, whereas public sector hospitals often care for patients of lower socioeconomic status with more constrained resources. Additionally, 3 of the sites were located within an urban setting, whereas the public cancer hospital was more rural.
Study participants were recruited primarily through word-of-mouth and screened for eligibility by a trained site study leader following a standardized protocol to conduct pilot testing in the field. We pre-loaded the app on Android study smartphones to ensure limited and controlled app distribution for beta testing. Study site leaders recruited healthcare providers aged 18 or over working in cancer or palliative care units who provide direct care to patients with cancer. The target number of participants for each study site was determined by the size of the institution (for example, the hospice facility operates with a small staff of approximately 10 healthcare providers and thus had a target of 7 providers), the scope of this pilot study, and what the Nepal research team members felt were realistic recruitment goals. The study was approved by respective ethical committees (UVA Social and Behavioral Sciences Institutional Review Board and the Nepal Health Research Council), prior to any data collection.
After providing informed consent, basic demographic data were collected and participants were provided a study smartphone preloaded with the NAPCare PMG app. Participants were then asked to use the app to work through two (out of four) randomly selected representative cancer pain patient case studies, written by members of the Nepal research team (Supplementary Table 2). These case studies represented various types of common pain scenarios seen by clinicians in Nepal. Data had the ability to be entered off-line (i.e., without internet connectivity), stored, and then later pushed to a secure automation tool (Integromat) when the user reconnected to the internet. Integromat then created a row in a Google spreadsheet to store app response data.
After participants finished using the app, they completed a brief Qualtrics survey about their experience on a study tablet. This survey included the industry-standard 10-item System Usability Scale (83, 84) (SUS) and selected items (n = 12) from the Mobile App Rating Scale (85–87) (MARS) to assess functionality, aesthetics, information, subjective quality, and perceived impact. An open text field for additional comments or suggestions was also included. The survey was designed in consultation with the UVA Center for Survey Research and reviewed by a native Nepali speaker not directly involved in the study; some survey items were modified slightly for clarity. Study tablets and phones had data plans added to reduce the risk of problems with potentially unstable institution WiFi.
Data Analysis
Qualtrics survey data were cleaned and exported for analysis. Descriptive and summary statistics were completed and compared across institutions and roles, and open-text responses compiled and summarized. The SUS was calculated as recommended, with a range of scores from 0 to 100, with a score of 68 considered above average (88). The MARS overall score was calculated as a simple average of the 12 items from the survey, and we also calculated subgroup MARS category scores (function; aesthetics; information; subjective quality, perceived impact). Additionally, we conducted regression analyses to explore relationships between selected demographic variables and SUS and MARS scores. Specifically, overall SUS and MARS scores were modeled using the Generalized Linear Model (GLM) procedure and variables considered included age, clinical role, institution, primary practice area, total years as a nurse or physician, formal training in palliative care, and pre-existing familiarity with the NAPCare PMG. These variables were selected as we hypothesized they may be most influential in predicting how a respondent experienced the NAPCare PMG app. To identify covariates with potential significance in modeling overall SUS and MARS, univariate models of SUS and MARS were initially considered. Only variables with a p-value <0.10 were considered for the final model of SUS and MARS. In order to establish the final model for these outcomes, all variables significant in univariate models were combined. Covariates were then sequentially removed by highest p-value if no longer significant in the presence of other covariates at α = 0.05 level. All analyses were performed using SPSS 27.0 (IBM SPSS, Chicago, IL) and SAS 9.4 (SAS Institute, Cary, NC). Due to small group sizes, several variables were regrouped, combining similar subgroups for use in the models. These variables included: clinical role, combining “MD” and “MD student”; combining providers aged 41 and older; condensing primary practice area groups to “Medical oncology,” “Palliative care” and “Other”; and combining total years as a nurse or physician to responses of 5 years or less in a single group.
Results
Participant Demographics
A total of 98 healthcare providers (n = 98) across four cancer care centers within Nepal pilot tested the NAPCare PMG app (Table 4). The majority of participants were nurses (n = 70; 71.4%) with 22 physicians (n = 22; 22.4%) and 6 medical students (n = 6; 6.1%). Most participants were female (n = 79; 80.6%), had completed a Bachelor's degree (n = 50; 51%) and almost half of respondents were between ages 18–30 (n = 47; 48%). An equal number of respondents were from the public cancer hospital, private cancer hospital and public general hospital (n = 30; 30.6%) and a smaller number from the hospice, which generally only has a total staff of about 10 people (n = 8; 8.2%). An almost equal number of participants reported completing formal training in palliative care/cancer pain management (n = 48; 49.0%) versus not (n = 50; 51.0%). Almost half of participants (n = 47; 48%) reported being “very familiar” with the NAPCare PMG (score of 8 or higher on a 0, no familiarity to 10, very familiar, scale); 28.6% (n = 28) participants reported they “never” use mobile apps to provide clinical care to patients, whereas 21.4% (n = 21) reported using mobile apps for clinical care “very often” (score of 8 or higher on 0, never to 10, very often). Many more participants (n = 64; 65.3%) reported using mobile apps for personal reasons “very often” (score of 8 or higher on 0, never to 10, very often); only 5 participants (n = 5; 5.1%) said they “never” use mobile apps for personal reasons.
Table 4. Demographics of participants who tested the NAPCare PMG mobile application.
Descriptive Survey Results
Overall, across institutions and roles, the app received an SUS score of 76.0 (a score above 68 is considered above average). Table 5 displays individual SUS survey items and responses across institutions, with preferred responses in italics. The overall MARS average, across institutions and roles, was 4.10 (on a scale of 1 = poor, 5 = excellent). Over 75% of the app testers (n = 75; 76.1%) rated the app as 4 or 5 stars (out of 5); 85% (n = 83) “agreed” or “strongly agreed” that the app increased their awareness of the importance of managing cancer pain; 86% (n = 84) “agreed” or “strongly agreed” that the app increased their knowledge of managing cancer pain; and 89.8% (n = 88) “agreed” or “strongly agreed” that the app will help them better manage cancer pain. Over 60% (n = 61; 62.2%) reported they would recommend the NAPCare PMG to all nurses and physicians. Table 6 displays MARS survey items and responses across institutions; Table 7 displays MARS responses, grouped by subcategories, across institutions. Summary SUS and MARS results are displayed in Figures 4, 5 and Table 8. A total of 44 free-text responses were recorded (32 from nurses; 12 from physicians/medical students); the majority provided feedback regarding specific medications and dosing recommendations.
Table 5. SUS survey responses, by institution.
Table 6. MARS survey responses, by institution.
Table 7. Summary of MARS scores, by subcategories.
Figure 4. Overall SUS scores by clinical role (left) and institution (right).
Figure 5. Overall MARS scores by clinical role (L) and institution (R).
Table 8. Overall SUS and MARS scores, by institution and role.
Modeling Results
Nine variables were considered in modeling overall SUS scores (Supplementary Table 3). Seven factors were significant in univariate models: age, clinical role, institution, total years as a nurse or physician, formal training in palliative care, use of mobile apps for clinical care, and use of mobile apps for personal use. Once combined in a multivariate model, only age, years of experience, and training in palliative care were significant in modeling SUS (p-values, 0.0124, 0.0371, and 0.0189, respectively). Supplementary Table 4 indicates that participants under 41 years old have higher SUS than those who are 41 and older, having 6 to 10 years of experience is associated with the highest estimates of SUS, and having no formal training in palliative care is associated with higher SUS. Boxplots (Figure 6) display the relationships between SUS and variables significant in the SUS model.
Figure 6. Boxplots of statistically significant variables in final multivariate SUS models.
Among 9 demographic variables considered in the initial MARS modeling, 2 were significant in univariate models (Supplementary Table 5): clinical role and institution (p-values 0.0834 and 0.0030, respectively). Once combined in a multivariate model (Supplementary Table 6), only institution remained significant in modeling MARS (p-value = 0.0030). Hospice and the private cancer hospital were associated with the highest MARS scores; the general public hospital was associated with the lowest MARS scores. Boxplots of overall MARS by institution display the differences in MARS across institution (Figure 5).
Discussion
Overall Results and the App Design Process
Overall, users reported very positive feedback and receptivity regarding the NAPCare PMG mobile app. Our results demonstrate strong feasibility and acceptability of oncology care providers in Nepal using the NAPCare PMG mobile app to help guide cancer pain care. Across diverse care contexts, and care provider roles, the overall SUS (72/100) and MARS scores (4.1/5) indicate users find the app useful, easy to navigate, and helpful. Potential uptake of the app is further supported by our findings that smartphone use is ubiquitious and that healthcare providers commonly use mobile apps both for clinical and personal reasons. It is also encouraging that the majority of open-text feedback and suggestions related to the app focused on very specific—and highly fixable—medication and dosing changes, vs. larger structural or conceptual concerns. Our research adds to the existing and rapidly expanding body of literature related to the use of mobile apps in LMICs to support healthcare providers and patient care (41, 89), the benefits of global partnerships to strengthen research capacity (90–92), and the value of using a community based participatory approach in designing mobile apps (73, 79). Importantly, our findings contribute knowledge related to the importance of developing health-related mobile apps that are congruent with the local cultural and resource context. We see a key strength of this work as leveraging the pre-existing NAPCare PMG which were created by community and clinical leaders within Nepal and already had a high degree of buy-in from healthcare providers prior to app development. We also find it noteworthy that we were able to accomplish the aims of this grant despite the severe disruption of the COVID-19 global pandemic, which disportionately impacted our Nepal team members. That we were able to successfully complete development and pilot testing of the app despite the myriad and unpredictable challenges wrought by COVID-19 is a testament to the dedication, commitment and tenacity of our partners in Nepal.
In terms of our app development design process, our in-person workshop in January 2020 was extremely productive; this proved very fortitious as future planned in-person workshops were impossible due to COVID-19 travel restrictions. The remaining app design work subsequently occurred over Zoom, which had its challenges, but was made significantly easier by the foundational work we had successfully completed during the prior in-person workshop. The January workshop was particularly effective due to the hands-on and interactive aspect of the work, which was assisted by printing out the large posters which visually represented the app flow. Working through potential case studies using the poster, and identifying problems, was one of the most effective activities during our workshop and proved invaluable in determining the finalized app algorithm. Our project also benefitted from the engagement of a highly collaborative software engineer who deftly navigated cross-cultural communications and interactions. It is important to note that while mobile app development and pilot testing was a key deliverable of this project, the overarching and larger goal was building research capacity and solidifying our collaboration.
MARS and SUS Results
We used a combination of descriptive and inferential statistical analysis to more fully understand the user experience related to pilot testing the NAPCare PMG and to help inform future work. Importantly, our regression analyses suggests end-user variables important to consider (age, years of experience, training in palliative care, and institutional setting) in designing and evaluating mobile apps to improve cancer pain management in lower resourced settings.
Descriptive
By institution, both the overall average SUS and MARS scores were highest for the private cancer hospital (80.1; 4.31, respectively). By role, the SUS scores were highest for medical students (80.4) and MARS scores were highest for nurses (4.15). One possible explanation for the higher SUS and MARS scores in the private cancer hospital is that this group of providers, given their work in a generally higher-resourced context, may have had more exposure to in-services and palliative care training, and perhaps more interest and experience in using apps to guide patient care. Interpreting SUS and MARS results by role is perhaps less straightforward and should be done cautiously as we had a small group of medical students (n = 6) and, with MARS, smaller differences between provider roles.
An additional challenge is knowing how to best interpret divergent results between overall SUS with MARS scores. For example, hospice had the lowest overall SUS scores (71.3), but the second highest MARS score (4.30, just behind the private cancer hospital at 4.31). Whether this is more reflective of the SUS and MARS capturing different facets of the user experience, or truly represents a difference between institutional mobile app needs, requires further exploration—as was done with our modeling analysis (see below). One noteworthy finding from our prior survey (81) related to differences in permissibility of the use of mobile phones in the clinical work setting between physicians and nurses (nurses reported being less likely to be allowed to use mobile phones in the work setting). As our hospice participants were predominantly nurses (n = 7, 88%), perhaps this impacted their experience testing the app and SUS scores.
MARS average scores by subcategories (function, aesthetics, information, subjective quality, and perceived impact) allows for a more granular exploration of why results may differ across institutions. Subjective quality and perceived impact were rated most highly by hospice (4.63, 4.42, respectively), whereas function, aesthetics and information were rated most highly by the private cancer hospital (4.24, 4.25, 4.47, respectively). Higher scores in these areas by hospice may be because nurses in this practice environment frequently need to verify information using Google or reference books in the hospice library. The NAPCare PMG app may be seen as particularly impactful in this type of care setting as it reduces the need for additional verification and provides reassurance related to the accuracy and quality of the information. Additionally, hospice had the highest percentage of participants younger than 30 years of age—individuals who are potentially more comfortable and receptive to mobile apps—which may have contributed to higher MARS scores. Higher scores by the private cancer hospital may be related to the generally higher resource context of private sector hospitals, and also that the private cancer hospital included in our study has a separate palliative care ward and specific dedicated palliative care nurses. Conversely, the lower MARS scores reported by the public general hospital may be due to the lack of participants who worked specifically in palliative care.
Modeling
Our final multivariate models revealed that age (<41 years), years in practice (6–10 years), and previous training in palliative care/cancer pain management (no) were all significant in predicting higher SUS scores. While younger age and more years in practice seem logical in predicting higher SUS scores (e.g., younger participants may have more familiarity/ease in using mobile apps and healthcare providers with a fair amount of clinical experience recognize the value of the app to help manage cancer pain), the finding related to previous training is interesting. One possible explanation is that healthcare providers with previous formal training in palliative care or cancer pain management, and a greater fund of knowledge in these areas, expected more from the app, and were more attuned to areas for improvement, or corrections needed with specific pharmacological recommendations. Therefore, providers without palliative care or cancer pain management training may have been less critical of the app, resulting in higher SUS scores for this participant group.
The only significant variable in predicting MARS scores was institution, with hospice and the private cancer hospital associated with higher MARS scores, and the public cancer hospital associated with lower MARS scores (In our descriptive analysis, the private cancer hospital also had the highest overall average SUS and MARS scores.). This finding supports our initial hypothesis that mobile app needs and preferences will vary across different clinical contexts. Possible reasons for these differences across institutions may relate to the volume of cancer patients seen, presence of dedicated palliative care beds/wards and specialized palliative care personnel, and staffing turnover, which may affect knowledge and attitudes regarding pain management and palliative care.
With our modeling analysis, differing variables of significance between the SUS and MARS are most likely attributed to the fact that these tools are evaluating different facets of the user experience, and thus different aspects are important in predicting scores. For example, the SUS focuses more on the user experience with technology systems in general, while the MARS is more specifically designed to assess the user experience with mobile apps. This finding also highlights the challenges of measurement (in general, but in this case, specifically in evaluating technology) across cultures and of ensuring a shared understanding of what different tools are actually measuring, especially in diverse populations and healthcare contexts. Interestingly, “clinical role” as a demographic variable was not found to be significant with either the SUS or MARS, although this warrants further exploration in future work as our descriptive findings and prior work suggests there may be important disciplinary considerations in the use of mobile apps to support patient care.
Limitations
The primary limitation of this study is the relatively small sample size of palliative care-sensitized healthcare providers. However, our sampling strategy and size is consistent with the scale and scope of a pilot feasibility and acceptability study (93–95), and must be considered in the context of a global project conducted during the COVID-19 pandemic. Additionally, most of our participants were nurses, as at all sites more nurses are employed than physicians. We beta tested the app using simulated case studies vs. clinical encounters with real patients; again, this approach is consistent with a pilot study designed to assess user experience of the app vs. clinical effectiveness. One particular challenge of this work related to finding the best tool(s) to evaluate the mobile app. After extensive searching in the literature, we ultimately decided on the SUS and selected items from the MARS. However, both tools required some adapation in terms of simplifying and clarifying language with input from our Nepal team members and an external native Nepali speaker/interpreter consultant. An important area of future work should be the development of concise, simple tools to meaningfully evaluate mobile apps in lower-resourced settings with diverse user groups.
Conclusion
Overall, and across institutions and roles, the NAPCare PMG mobile app was extremely well-received, and participants rated it highly on both the SUS and MARS. Regression analyses suggest end-user variables important to consider (e.g., age, years of experience, training in palliative care, and institutional setting) in designing and evaluating mobile apps in lower resourced settings. Healthcare providers within LMICs can utilize mobile apps to improve cancer pain care and support adherence to clinical practice guidelines, but it is critical these tools are culturally and contextually congruent. Our app design and pilot testing process illustrate the benefits of cross global collaborations to build research capacity and generate knowledge within the local context. Using locally developed PMG—vs. those imported from the West—is a key strength of this work and offers a scalable approach that can be reproduced in other LMICs. Future work will include enhancing features and functionality of the NAPCare PMG app and testing its efficacy in real-world clinical encounters on relevant healthcare provider, patient, and organizational outcomes with a larger and more diverse sample of participants.
Data Availability Statement
De-identified data, in compliance with institutional data sharing requirements, will be made available by the authors upon reasonable request.
Ethics Statement
This study was reviewed and approved by University of Virginia Social and Behavioral IRB and the Nepal Health Research Council. The participants provided their informed consent to participate in this study as per IRB guidelines. Written informed consent was obtained from the relevant individuals, for the publication of any potentially identifiable images or data included in this article.
Author Contributions
VL, AA, SC, RG, RK, GK, MM, DM, BN, KS, RS, SS, UT, RD, and BP conceptualized the study. BHa wrote the software code for the mobile app. VL, AA, SC, MD, RG, BHa, RK, GK, MM, DM, BN, KS, RS, AS, SS, UT, and BP conducted and/or supported data collection. VL, BHa, and AK led data analysis. VL, BHa, SC, GK, AK, MM, RS, UT, RD, and BP assisted with interpretation of findings. VL wrote first draft of manuscript and incorporated feedback from co-authors into the final paper. All co-authors reviewed and approved the final manuscript.
Funding
This research was supported by the Fogarty International Center, National Institutes of Health, R21TW011244.
Conflict of Interest
BHa is the founder of and employed by Hass Software Consulting. RD provides consultative services for Warm Health Technologies, Inc.
The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher's Note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Acknowledgments
The authors would like to thank Dael Cepeda, Kara Fitzgibbon, Lahar Shrestha, and Amber Steen for their support.
Supplementary Material
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpain.2022.910995/full#supplementary-material
References
1. Knaul FM, Farmer PE, Krakauer EL, De Lima L, Bhadelia A, Kwete J, et al. Alleviating the access abyss in palliative care and pain relief—an imperative of universal health coverage: the Lancet Commission Report. Lancet. (2018) 391:1391–454. doi: 10.1016/S0140-6736(17)32513-8
2. Daniels ME, Donilon TE, Bollyky TJ, Tuttle CM. The emerging global health crisis: noncommunicable diseases in low- and middle-income countries. Council on Foreign Relations Independent Task Force Report No. 72. Available online at: https://www.cfr.org/report/emerging-global-health-crisis doi: 10.2139/ssrn.2685111 (accessed December 05, 2014).
3. Partridge EE, Mayer-Davis EJ, Sacco RL, Balch AJ. Creating a 21st century global health agenda: the general assembly of the united nations high level meeting on non-communicable diseases. CA Cancer J Clin. (2011) 61:209–11. doi: 10.3322/caac.20120
4. World Health Organization (WHO). Palliative Care for Non-Communicable Diseases: A Global Snapshot. (2020). Available online at: https://www.who.int/publications/i/item/palliative-care-for-noncommunicable-diseases-a-global-snapshot-2020.
5. Knaul FM, Atun R, Farmer P, Frenk J. Seizing the opportunity to close the cancer divide. Lancet. (2013) 381:2238–9. doi: 10.1016/S0140-6736(13)60176-2 (accessed May 25, 2022).
6. Krakauer EL, Wenk R, Buitrago R, Jenkins P, Scholten W. Opioid inaccessibility and its human consequences: reports from the field. J Pain Palliat Care Pharmacother. (2010) 24:239–43. doi: 10.3109/15360288.2010.501852
7. Worldwide Hospice Palliative Care Alliance (WHPCA) World Health Organization (WHO). Global Atlas of Palliative Care. 2nd ed. (2020). Available online at: http://www.thewhpca.org/resources/global-atlas-on-end-of-life-care (accessed May 25, 2022).
8. Cleary JF, Husain A, Maurer M. Increasing worldwide access to medical opioids. Lancet. (2016) 387:1597–9. doi: 10.1016/S0140-6736(16)00234-8
9. Worldwide, Hospice Palliative Care Association (WPCA). About the WHPCA. Available online at: https://www.thewhpca.org/ (accessed May 25, 2022).
10. Mehta SJ. Patient satisfaction reporting and its implications for patient care. AMA J Ethics. (2015) 17:616–21. doi: 10.1001/journalofethics.2015.17.7.ecas3-1507
11. Deandrea S, Corli O, Consonni D, Villani W, Greco MT, Apolone G. Prevalence of breakthrough cancer pain: systematic review and a pooled analysis of published literature. J Pain Symptom Manage. (2014) 47:57–76. doi: 10.1016/j.jpainsymman.2013.02.015
12. Lemay K, Wilson KG, Buenger U, Jarvis V, Fitzgibbon E, Bhimji K, Dobkin P. Fear of pain in patients with advanced cancer or in patients with chronic noncancer pain. Clin J Pain. (2011) 27:116–24. doi: 10.1097/AJP.0b013e3181f3f667
13. Goodwin PJ, Bruera E, Stockler M. Pain in patients with cancer. J Clin Oncol. (2014) 32:1637–9. doi: 10.1200/JCO.2014.55.3818
14. Farmer P, Frenk J, Knaul FM, Shulman LN, Alleyne G, Armstrong L, et al. Expansion of cancer care and control in countries of low and middle income: a call to action. Lancet. (2010) 376:1186–93. doi: 10.1016/S0140-6736(10)61152-X
15. 67th World Health Assembly. Strengthening of palliative care as a component of comprehensive care throughout the life course. WHA67.10 (2014). Available online at: https://apps.who.int/gb/ebwha/pdf_files/WHA67/A67_R19-en.pdf?ua=1 (accessed May 25, 2022).
16. Brennan F, Lohman D, Gwyther L. Access to pain management as a human right. Am J Public Health. (2019) 109:61–5. doi: 10.2105/AJPH.2018.304743
17. Radbruch L, De Lima L, Lohmann D, Gwyther E, Payne S. The prague charter: urging governments to relieve suffering and ensure the right to palliative care. Palliat Med. (2013) 27:101–2. doi: 10.1177/0269216312473058
18. Lohman D, Schleifer R, Amon JJ. Access to pain treatment as a human right. BMC Med. (2010) 8:8. doi: 10.1186/1741-7015-8-8
19. International Pain Summit of the International Association for the Study of Pain. Declaration of Montréal: declaration that access to pain management is a fundamental human right. J Pain Palliat Care Pharmacother. (2011) 25:29–31. doi: 10.3109/15360288.2010.547560
20. Munday D, Basnyat R, Paudel BD, Russell R. IAPCON 2017 abstracts: review of palliative care services in Nepal. Indian J Palliat Care. (2017) 23:121–79.
21. Fallon M, Giusti R, Aielli F, Hoskin P, Rolke R, Sharma M, et al. Management of cancer pain in adult patients: ESMO clinical practice guidelines. Ann Oncol. (2018) 29(Suppl. 4):iv166–91. doi: 10.1093/annonc/mdy152
22. World Health Organization (WHO) guidelines for the pharmacological radiotherapeutic management of cancer pain in adults adolescents. (2019). Available online at https://www.who.int/publications/i/item/9789241550390 (accessed May 25, 2022).
23. Dy SM, Asch SM, Naeim A, Sanati H, Walling A, Lorenz KA. Evidence-based standards for cancer pain management. J Clin Oncol. (2008) 26:3879–85. doi: 10.1200/JCO.2007.15.9517
24. Swarm RA, Paice JA, Anghelescu DL, Are M, Bruc JY, Buga S, et al. Adult cancer pain, version 3.2019, NCCN clinical practice guidelines in oncology. J Natl Compr Canc Netw. (2019) 17:977–1007. doi: 10.6004/jnccn.2019.0038
25. Ahmedzai SH, Bautista MJ, Bouzid K, Gibson R, Gumara T, Hassan AAI et al. Optimizing cancer pain management in resource-limited settings. Support Care Cancer. (2019) 27:2113–24. doi: 10.1007/s00520-018-4471-z
26. Pen SLD, Pen ARD, Polissar N, Hansberry J, Kraybill BM, Stillman M, et al. Implementing guidelines for cancer pain management: results of a randomized controlled clinical trial. J Clin Oncol. (1999) 17:361–70. doi: 10.1200/JCO.1999.17.1.361
27. Mearis M, Shega JW, Knoebel RW. Does adherence to National Comprehensive Cancer Network guidelines improve pain-related outcomes? An evaluation of inpatient cancer pain management at an academic medical center. J Pain Symptom Manage. (2014) 48:451–8. doi: 10.1016/j.jpainsymman.2013.09.016
28. Foley KM. How well is cancer pain treated? Palliat Med. (2011) 25:398–401. doi: 10.1177/0269216311400480
29. Kaasa S, Haugen DF. Cancer pain research: time to reset the strategy and the agenda. Palliat Med. (2011) 25:392–93. doi: 10.1177/0269216311407403
30. Lewis J, Ray P, Liaw ST. Recent worldwide developments in eHealth and mHealth to more effectively manage cancer and other chronic diseases–a systematic review. Yearb Med Inform. (2016) 1:93–108. doi: 10.15265/IY-2016-020
31. Levine R, Corbacio A, Konopka S, Saya U, Gilmartin C, Paradis J. mhealth Compendium. Vol. 5. Arlington, VA: African Strategies for Health, Management Sciences for Health (2015). Available online at: http://www.africanstrategies4health.org/uploads/1/3/5/3/13538666/mhealthvol5_final_15jun15_webv.pdf (accessed May 25, 2022).
32. Odeh B, Kayyali R, Nabhani-Gebara S, Philip N. Optimizing cancer care through mobile health. Support Care Cancer. (2015) 23:2183–8. doi: 10.1007/s00520-015-2627-7
33. WHO Global Observatory for eHealth. mhealth: New Horizons for Health Through Mobile Technologies: Second Global Survey on eHealth. World Health Organization (2011). Available online at: https://apps.who.int/iris/handle/10665/44607 (accessed May 25, 2022).
34. Klasnja P, Pratt W. Healthcare in the pocket: mapping the space of mobile-phone health interventions. J Biomed Inform. (2012) 45:184–98. doi: 10.1016/j.jbi.2011.08.017
35. Poushter J, Bishop C, Chwe H Pew Research Center. Social media use continues to rise in developing countries but plateaus across developed ones. (2018). Available online at: https://www.pewresearch.org/global/2018/06/19/social-media-use-continues-to-rise-in-developing-countries-but-plateaus-across-developed-ones/ (accessed May 25, 2022).
36. Abaza H, Marschollek M. mHealth application areas and technology combinations: a comparison of literature from high and low/middle income countries. Methods Inf Med. (2017) 56:e105–22. doi: 10.3414/ME17-05-0003
37. Chib A, van Velthoven MH, Car J. mHealth adoption in low-resource environments: a review of the use of mobile healthcare in developing countries. J Health Commun. (2015) 20:4–34. doi: 10.1080/10810730.2013.864735
38. Haas S. mHealth Compendium, Special Edition 2016: Reaching Scale. Arlington, VA: African Strategies for Health, Management Sciences for Health. Available online at: http://www.africanstrategies4health.org/uploads/1/3/5/3/13538666/2016_mhealth_31may16_final.pdf (accessed May 25, 2022).
39. Agarwal S, Perry HB, Long LA, Labrique AB. Evidence on feasibility and effective use of mHealth strategies by frontline health workers in developing countries: systematic review. Trop Med Int Health. (2015) 20:1003–14. doi: 10.1111/tmi.12525
40. Labrique AB, Vasudevan L, Kochi E, Fabricant R, Mehl G. mHealth innovations as health system strengthening tools: 12 common applications and a visual framework. Glob Health Sci Pract. (2013) 1:160–71. doi: 10.9745/GHSP-D-13-00031
41. Labrique A, Agarwal S, Tamrat T, Mehl G. WHO digital health guidelines: a milestone for global health. NPJ Digit Med. (2020) 3:120. doi: 10.1038/s41746-020-00330-2
42. Love RR, Ferdousy T, Paudel BD, Nahar S, Dowla R, Adibuzzaman M, et al. Symptom levels in care-seeking Bangladeshi and Nepalese adults with advanced cancer. J Glob Oncol. (2016) 3:257–60. doi: 10.1200/JGO.2016.004119
43. Braun R, Catalani C, Wimbush J, Israelski D. Community health workers and mobile technology: a systematic review of the literature. PLoS ONE. (2013) 8:e65772. doi: 10.1371/journal.pone.0065772
44. Style S, Beard BJ, Harris-Fry H, Sengupta A, Jha S, Shrestha BP, et al. Experiences in running a complex electronic data capture system using mobile phones in a large-scale population trial in southern Nepal. Glob Health Action. (2017) 10:1330858. doi: 10.1080/16549716.2017.1330858
45. Jones CO, Wasunna B, Sudoi R, Githinji S, Snow RW, Zurovac D. “Even if you know everything you can forget”: Health worker perceptions of mobile phone text-messaging to improve malaria case-management in Kenya. PLoS ONE. (2012) 7:e38636. doi: 10.1371/journal.pone.0038636
46. Zurovac D, Sudoi RK, Akhwale WS, Ndiritu M, Hamer DH, Rowe AK, et al. The effect of mobile phone text-message reminders on Kenyan health workers' adherence to malaria treatment guidelines: a cluster randomised trial. Lancet. (2011) 378:795–803. doi: 10.1016/S0140-6736(11)60783-6
47. Adepoju IOO, Albersen BJA, De Brouwere V, van Roosmalen J, Zweekhorst M. mHealth for clinical decision-making in sub-Saharan Africa: a scoping review. JMIR Mhealth Uhealth. (2017) 5:e38. doi: 10.2196/mhealth.7185
48. Agarwal S, LeFevre AE, Lee J, L'Engle K, Mehl G, Sinha C, et al. Guidelines for reporting of health interventions using mobile phones: mobile health (mHealth) evidence reporting and assessment (mERA) checklist. BMJ. (2016) 352:i1174. doi: 10.1136/bmj.i1174
49. WHO Guideline. Recommendations on digital interventions for health system strengthening. (2019). Available online at: https://www.who.int/publications/i/item/9789241550505 (accessed May 25, 2022).
50. World Health Organization (WHO). Non-Communicable Disease Fact Sheet. (2021). Available online at: https://www.who.int/news-room/fact-sheets/detail/noncommunicable-diseases (accessed May 25, 2022).
51. World Health Organization (WHO). Non-communicable disease country profiles 2018, Nepal. Available online at: https://www.who.int/publications/i/item/9789241514620 (accessed May 25, 2022).
52. Poudel KK, Huang Z, Neupane PR, Steel R. Prediction of the cancer incidence in Nepal. Asian Pac J Cancer Prev. (2017) 18:165–8. doi: 10.22034/APJCP.2017.18.1.165
53. Swarbrick E, Munday D, Pietroni M. IAPCON abstracts. Based on the prevalence of non-communicable diseases, what is the projected need for palliative care in Nepal? Indian J Palliat Care. (2017) 23:121–79.
54. Sung H, Ferlay J, Siegel RL, Laversanne M, Soerjomataram I, Jemal A, et al. Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clinic. (2020) 71:209–49. doi: 10.3322/caac.21660
55. International International Agency for Research on Cancer World Health Organization. Cancer Tomorrow. (2020). Available online at: https://gco.iarc.fr/tomorrow/home (accessed May 25, 2022).
56. Gyawali B, Sharma S, Shilpakar R, Dulal S, Pariyar J, Booth CM, et al. Overview of delivery of cancer care in Nepal: current status and future priorities. JCO Glob Oncol. (2020) 6:1211–7. doi: 10.1200/GO.20.00287
57. Nepal Health Research Council (NHRC). Population Based Cancer Registry, Nepal. (2018). Available online at: http://nhrc.gov.np/wp-content/uploads/2019/04/Progress-Interim-_cancer.pdf (accessed May 25, 2022).
58. Chapagain S, Budukh A, Dangal G, Jha AK. Initiation of population-based cancer registry in Nepal. J Nepal Health Res Counc. (2019) 17:I–II. doi: 10.33314/jnhrc.v17i3.2359
59. Government of Nepal. Multisectoral Action Plan for the Prevention and Control of Non Communicable Diseases (2014–2020). Available online at: https://www.who.int/docs/default-source/searo/ncd-surveillance/pages-from-nep-ncd-action-plan-2014–2020-me.pdf?sfvrsn=9eaf4d33_2 (accessed May 25, 2022).
60. Pastrana T, Vallath N, Mastrojohn J, Namukwaya E, Kumar S, Radbruch L, et al. Disparities in the contribution of low-and middle-income countries to palliative care research. J Pain Symptom Manage. (2010) 39:54–68. doi: 10.1016/j.jpainsymman.2009.05.023
61. Harding R, Powell RA, Downing J, Connor SR, Mwangi-Powell F, Defilippi K, et al. Generating an African palliative care evidence base: the context, need, challenges, and strategies. J Pain Symptom Manage. (2008) 36:304–9. doi: 10.1016/j.jpainsymman.2008.04.008
62. Clark J, Gardiner C, Barnes A. International palliative care research in the context of global development: a systematic mapping review. BMJ Support Palliat Care. (2018) 8:7–18. doi: 10.1136/bmjspcare-2015-001008
63. Wenk R, De Lima L, Eisenchlas J. Palliative care research in Latin America: Results of a survey within the scope of the Declaration of Venice. J Palliat Med. (2008) 11:717–22. doi: 10.1089/jpm.2007.0212
64. Paudel BD, Ryan KM, Brown MS, Krakauer EL, Rajagopal MR, Maurer M, et al. Opioid availability and palliative care in Nepal: influence of an international pain policy fellowship. J Pain Symptom Manage. (2015) 49:110–16. doi: 10.1016/j.jpainsymman.2014.02.011
65. Swarbrick EM, Pietroni MA, Munday DM. the need for palliative care in adults in Nepal: Projections based on modeling mortality data. Indian J Palliat Care. (2019) 25:41–5. doi: 10.4103/IJPC.IJPC_177_18
66. Harding R, Selman L, Powell RA, Namisango E, Downing J, Merriman A, et al. Research into palliative care in sub-Saharan Africa. Lancet Oncol. (2013) 14:e183–8. doi: 10.1016/S1470-2045(12)70396-0
67. Demment MM, Peters K, Dykens JA, Dozier A, Hawaz H, McIntosh S, et al. Developing the evidence base to inform best practice: a scoping study of breast and cervical cancer reviews in low- and middle-income countries. PLoS ONE. (2015) 10:e0134618. doi: 10.1371/journal.pone.0134618
68. The Nepalese Association of Palliative Care (NAPCare). (2022). Available online at: http://napcare.org.np/ (accessed May 25, 2022).
69. Gongal R, Upadhyay SK, Baral KP, Watson M, Kernohan GW. Providing palliative care in rural Nepal: perceptions of mid-level health workers. Indian J Palliat Care. (2018) 24:150–5. doi: 10.4103/IJPC.IJPC_196_17
70. Nepalese Association of Palliative Care (NAPCare). NAPCare Guidelines for Pain Management in Palliative Care. (2017). Available online at: http://napcare.org.np/wp-content/uploads/2020/09/pain-protocol_final.A-1.pdf (accessed May 25, 2022).
71. Anekar AA, Cascella M. WHO analgesic ladder. In: StatPearls. Treasure Island, FL: StatPearls Publishing (2022). Available online at: https://www.ncbi.nlm.nih.gov/books/NBK554435/ (accessed May 25, 2022).
72. Scutti S. NIH Fogarty International Center Global Health Matters: Focus on pain management in low middle-income countries. Scientists use mobile health app to mitigate pain in Nepal. (2020) 19. Available at: https://www.fic.nih.gov/News/GlobalHealthMatters/march-april-2020/Pages/pain-research-mhealth-nepal.aspx?utm_medium=email&utm_campaign=ghm&utm_source=ghm2020april (accessed May 25, 2022).
73. Wallerstein NB, Duran B. Using community-based participatory research to address health disparities. Health Promot Pract. (2006) 7:312–23. doi: 10.1177/1524839906289376
74. Baum F, MacDougall C, Smith D. Participatory action research. J Epidemiol Community Health. (2006) 60:854. doi: 10.1136/jech.2004.028662
75. Campbell J. A critical appraisal of participatory methods in development research. Int J Soc Res Methodol. (2002) 5:19–29. doi: 10.1080/13645570110098046
76. Loewenson R, Flores W, Shukla A, Kagis M, Baba A, Ryklief A, et al. Raising the profile of participatory action research at the 2010 global symposium on health systems research. MEDICC Rev. (2011) 13:35–8. doi: 10.37757/MR2011V13.N3.8
77. Campbell BR, Ingersoll KS, Flickinger TE, Dillingham R. Bridging the digital health divide: toward equitable global access to mobile health interventions for people living with HIV. Expert Rev Anti Infect Ther. (2019) 17:141–4. doi: 10.1080/14787210.2019.1578649
78. Källander K, Tibenderana JK, Akpogheneta OJ, Strachan DL, Hill Z, Asbroek AHA, et al. Mobile health (mHealth) approaches and lessons for increased performance and retention of community health workers in low-and middle-income countries: a review. J Med Internet Res. (2013) 15:e17. doi: 10.2196/jmir.2130
79. Smith SA, Whitehead MS, Sheats J, Mastromonico J, Yoo W, Coughlin SS. A community-engaged approach to developing a mobile cancer prevention app: THE mCPA study protocol. JMIR Res Protoc. (2016) 5:e34. doi: 10.2196/resprot.5290
80. Meyers DJ, Filkins M, Bangura AH, Sharma R, Baruwal A, Pande S, et al. Management challenges in mHealth: failures of a mobile community health worker surveillance programme in rural Nepal. BMJ Innov. (2017) 3:19–25. doi: 10.1136/bmjinnov-2015-000102
81. LeBaron V, Adhikari A, Bennett R, Acharya SC, Dhakal M, Elmore CE, et al. A survey of cancer care institutions in Nepal to inform design of a pain management mobile application. BMC Palliative Care. (2021) 20:171. doi: 10.1186/s12904-021-00824-0
82. Schnall R, Rojas M, Bakken S, Brown W, Carballo-Dieguez A, Carry M, et al. A user-centered model for designing consumer mobile health (mHealth) applications (apps). J Biomed Inform. (2016) 60:243–51. doi: 10.1016/j.jbi.2016.02.002
83. Sauro J. A Practical Guide to the System Usability Scale: Background, Benchmarks & Best Practices. Denver, CO: Measuring Usability LLC (2011).
84. U.S. Department of Health and Human Services. The System Usability Scale. (2020). Available online at: https://www.usability.gov/how-to-and-tools/methods/system-usability-scale.html (accessed May 25, 2022).
85. Stoyanov SR, Hides L, Kavanagh DJ, Wilson H. Development and validation of the user version of the Mobile Application Rating Scale (uMARS). JMIR mHealth uHealth. (2016) 4:e72. doi: 10.2196/mhealth.5849
86. Salazar A, de Sola H, Failde I, Moral-Munoz JA. Measuring the quality of mobile apps for the management of pain: systematic search and evaluation using the Mobile App Rating Scale. JMIR mHealth uHealth. (2018) 6:e10718. doi: 10.2196/10718
87. Stoyanov SR, Hides L, Kavanagh DJ, Zelenko O, Tjondronegoro D, Mani M. Mobile App Rating Scale: a new tool for assessing the quality of health mobile apps. JMIR mHealth uHealth. (2015) 3:e27. doi: 10.2196/mhealth.3422
88. Sauro J. MeasuringU: Measuring Usability with the System Usability Scale (SUS). (2011). Available online at: https://measuringu.com/sus/ (accessed May 25, 2022).
89. World Health Organization. Global Strategy on Digital Health 2020–(2025). (2021). Available online at: https://apps.who.int/iris/handle/10665/344249 (accessed May 25, 2022).
90. Dean L, Gregorius S, Bates I, Pulford J. Advancing the science of health research capacity strengthening in low-income and middle-income countries: qa scoping review of the published literature, 2000–2016. BMJ Open. (2017) 7:e018718. doi: 10.1136/bmjopen-2017-018718
91. Franzen SRP, Chandler C, Lang T. Health research capacity development in low and middle income countries: reality or rhetoric? A systematic meta-narrative review of the qualitative literature. BMJ Open. (2017) 7:e012332. doi: 10.1136/bmjopen-2016-012332
92. Magrath I. Building capacity for cancer control in developing countries: the need for a paradigm shift. Lancet Oncol. (2007) 8:562–3. doi: 10.1016/S1470-2045(07)70179-1
93. Moore CG, Carter RE, Nietert PJ, Stewart PW. Recommendations for planning pilot studies in clinical and translational research. Clin Transl Sci. (2011) 4:332–7. doi: 10.1111/j.1752-8062.2011.00347.x
94. Julious SA. Sample size of 12 per group rule of thumb for a pilot study. Pharm Stat. (2005) 4:287–91.
Keywords: palliative care, cancer, mobile health, mobile applications, pain, pain management guidelines, Nepal, capacity building
Citation: LeBaron VT, Horton BJ, Adhikari A, Chapagain S, Dhakal M, Gongal R, Kattel R, Koirala G, Kutcher A, Hass B, Maurer M, Munday D, Neupane B, Sharma K, Shilpakar R, Shrestha A, Shrestha S, Thapa U, Dillingham R and Paudel BD (2022) A Global Collaboration to Develop and Pilot Test a Mobile Application to Improve Cancer Pain Management in Nepal. Front. Pain Res. 3:910995. doi: 10.3389/fpain.2022.910995
Received: 01 April 2022; Accepted: 21 June 2022;
Published: 28 July 2022.
Edited by:
Bassey Ebenso, University of Leeds, United KingdomCopyright © 2022 LeBaron, Horton, Adhikari, Chapagain, Dhakal, Gongal, Kattel, Koirala, Kutcher, Hass, Maurer, Munday, Neupane, Sharma, Shilpakar, Shrestha, Shrestha, Thapa, Dillingham and Paudel. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Virginia T. LeBaron, dnRsNmsmI3gwMDA0MDt2aXJnaW5pYS5lZHU=