Investigating the Usability, Efficacy and Accuracy of a Medication Entering Software System for a Healthcare Robot

Purpose: This research aimed to evaluate medication software for a healthcare robot. Study I compared two software versions (RoboGen and RoboGen2) for system usability, speed and accuracy of medication entry; Study II evaluated system usability and community pharmacists’ views of RoboGen2.

Methods: Study I had a within-subjects experimental design and recruited 40 Health Sciences students to enter different, comparable sets of prescriptions into the two systems, in randomized order, within a limit of 15 min. Screen activity was recorded to observe prescription errors. Study II had a cross-sectional observational design and recruited 20 community pharmacists using convenience sampling. Pharmacists entered three prescriptions using RoboGen2. Participants in both studies completed the System Usability Scale (SUS) following each task. Study I participants completed a questionnaire on system preference, and Study II participants a semi-structured interview.

Results: Study I participants preferred Robogen2 (p < 0.001) due to its sleek and modern layout, good flow, ease of use, and intuitive design. SUS scores [t (40) = −3.40, p = 0.002] and speed of medication entry favored Robogen2 (t = 3.65, p < 0.001). No significance was found in accuracy (t = 1.12, p = 0.27). In study 2, pharmacists rated the usability of RoboGen2 below average. Themes from interviews were navigation and streamlining the system, ease of use, and integration with pharmacy software systems.

Conclusion: Adding safety features and better aesthetics can improve the usability and safety of a medication prescription system. Streamlining workflow and pre-populating data can increase speed of prescription entry without compromising patient safety. However, a better approach is integration with pre-existing pharmacy systems to reduce workload while incorporating safety features built into existing dispensing systems.

1 Introduction

Chronic health conditions affect nearly one in four New Zealand (NZ) adults (Ministry of Health, 2020a) and contribute towards almost 90% of healthy life lost due to early illness, disability, or death (Ministry of Health, 2020b). The World Health Organization predicts that globally, chronic diseases account for 41 million deaths each year (World Health Organization, 2021). Whilst significant advances in healthcare have seen a substantial decline in mortality, this has also meant an increase in multiple long-term medications. With greater medication regimen complexity comes a greater risk for reduced medication adherence (Rantanen et al., 2017). Evidence suggests that adherence to long-term therapy is suboptimal with almost 40% of people stopping their medicines after the first year of therapy (Torres-Robles et al., 2021). Factors contributing to poor medication adherence include intentional factors, where patients deliberately forego treatment, or unintentional reasons such as forgetfulness (Sabaté and World Health Organization, 2003). As people age and/or take more medicines, forgetfulness becomes more common (Patton et al., 2017).

Medical device solutions designed to target forgetfulness include memory aids such as reminder packaging, text messaging, automatic prescription refills, and electronic reminders and monitoring devices (Rantanen et al., 2017). Alarm-based aids and pill-monitoring devices are active solutions that, when connected to the internet, can inform the patient if a dose is missed. Research has shown these aids act most effectively when combined with assistance from a care provider (Granger and Bosworth, 2011; Velligan et al., 2013). Other solutions include software-based systems, such as personal health portals and interactive/social robots, where users can share health-related information and receive personalized feedback (Datta et al., 2011). Social robots are emerging technologies that show promise for improving medication adherence by acting as medication reminders (Broadbent et al., 2018), as well as serving as social companions and coordinating patient medication information with healthcare professionals and/or carers (Rantanen et al., 2017). However, for health technologies to be usable, they need to be well designed, intuitive, easy to use, and meet the profession’s standards (Johnson et al., 2005). Failing to incorporate these features effectively may lead to inefficient care and prove time-consuming and/or labor-intensive (Mansoor et al., 2014; Hall et al., 2016; MacLure and Stewart, 2016; Pinto et al., 2018; Navti and Apampa, 2019). Furthermore, the aesthetics of an application can influence perceived usability even if there are no differences in functionality offered (Kurosu and Kashimura, 1995; Tractinsky, 1997).

In 2011, RoboGen, a web-based application, with enabled end-user programming, was designed by a team of health informatics and electrical and computer engineering researchers at the University of Auckland (Figure 1) (Datta et al., 2011). RoboGen was designed to be used by healthcare professionals as a medication management support system on a healthcare/social robot for residents in a retirement village (Datta et al., 2011, 2012; Broadbent et al., 2014). Field studies suggested that a robotic platform offered opportunities that were not possible with other medication management systems, and end-user programming allowed health professionals to manage medication instructions, dosing schedules, health education, appointment and refill reminders (Datta et al., 2011). Subsequently, RoboGen was expanded to specifically support patients with chronic obstructive pulmonary disease (COPD) by providing patient education, and in managing medication adherence, rehabilitation exercises, monitoring symptoms, and peak expiratory flow (Broadbent et al., 2018). During the COPD study, issues were reported by healthcare professionals entering patients’ prescriptions, in particular, that medication names and doses had to be typed in and all entries were saved in the system, resulting in a confusing collection of different spellings, generic names, and dosages.

FIGURE 1

FIGURE 1. RoboGen user interface: medication selection.

The overall research problem that this study aimed to address was how to make the medication management software more usable for healthcare professionals. In response, RoboGen2 was developed to include a different user interface, an imported list of medications consistent with the New Zealand Universal List of Medications (NZULM), and supplementary safety features (Figure 2 and Figure 3). An official drop-down list of medications and doses was used instead of typing names and doses.

FIGURE 2

FIGURE 2. RoboGen2 user interface: medication selection search function.

FIGURE 3

FIGURE 3. RoboGen2 test patient exercise information.

To determine whether these changes were an improvement on the original RoboGen software and were better able to meet the requirements of healthcare professionals entering medication on healthcare robots, the following research questions were proposed:

1) What is the difference in system usability, efficiency and accuracy of medication entry between RoboGen and RoboGen2? The authors hypothesized that RoboGen2 would be superior to RoboGen in system usability, efficiency and accuracy.

2) How usable do pharmacists find RoboGen2, and what are their views on using the software to support patient and caregivers to monitor medication adherence?

To answer the research questions and test the hypothesis, the research was designed as two studies. Study I aimed to compare the two versions of the medication entering software systems (RoboGen with RoboGen2) with participants who had some familiarity with medication entry to determine whether the changes to RoboGen system affected the usability, speed, and error rate of medication entry. Study II aimed to test system usability of RoboGen2 with pharmacists, who are licensed to dispense prescription medicines and use other medication management systems in the workplace, and to gather their views on using the software. Few studies on medication management software have involved pharmacists.

2 Materials and Methods

Studies I and II had approval from the University of Auckland Human Participants Ethics Committee (Reference numbers 020792 and 021323).

2.1 Study I

Study I was an experiment with a within-subjects design. Students enrolled in Health Sciences disciplines were recruited from the Faculty of Medical and Health Sciences at the University of Auckland via email, lecture announcements, and posters around campus. Inclusion criteria were students over the age of 16, familiarity with medication and/or prescriptions entry, and English language competence. After providing consent, participants completed a baseline demographic questionnaire, were given a demonstration of a prescription entered into each system and were asked to perform the same task with both RoboGen and RoboGen2. The task required participants to enter a set of fictional prescriptions, provided by the School of Pharmacy, into the system as accurately and efficiently as possible, with an imposed time limit of 15 min, to reflect a realistic pharmacy practice. Participants entered one set of prescriptions into the original RoboGen, and a different, comparable set into RoboGen2. The order of items in each of these sets of prescriptions was randomized as was the order of versions used by each participant. The study was conducted July–August 2018.

The independent variable was the version of RoboGen, and the dependent variables were the speed and accuracy with which the prescriptions were entered, self-reported usability, and preference. Following each task, participants were asked to complete the System Usability Scale (SUS) (Bangor, 2009) scored out of 50, with reliability analysis, Cronbach’s alpha of 0.91. The questionnaires included open-ended questions about likes and dislikes of each system, and which system participants preferred. Screen activity was recorded to observe prescription errors. Data were analyzed using a chi-square goodness-of-fit test for system preference, and paired sample t-tests to compare the two systems in terms of usability, speed (number of prescriptions entered in 15 min), and the difference in the proportion of correct entries (i.e., accuracy) between the two systems. Initial results from Study I on usability have been published in a brief report which did not include accuracy or speed results (Broadbent et al., 2020).

2.2 Study II

This was a cross-sectional observational study that included both a survey and interview. Pharmacies were recruited within central Auckland using convenience sampling, and data were collected from June–September 2018. Email addresses and phone numbers of pharmacies were obtained from the online portal Healthpoint. co.nz. Email invitations were sent to managers of the first 100 pharmacies together with a participant information sheet and consent form. Researchers followed up with a phone call 1 week after sending the email to enquire whether pharmacists wished to participate. A time and place for where the study would occur was negotiated with the 20 pharmacists expressing interest in participating. Recruitment of multiple pharmacists from the same pharmacy was allowed.

After consenting, pharmacists were asked to complete a short demographic questionnaire, and enter three fictional prescriptions using RoboGen2, each containing oral medications with various dosing instructions. Subsequently, pharmacists were invited to complete the SUS, and participate in an interview. Semi-structured interview questions were developed to enable open-ended opinions of the RoboGen2 software, as well as questions related to technology used in pharmacy, reported elsewhere (Law et al., 2021). The interview was recorded with a digital voice recorder and transcribed verbatim. Transcribed interviews were checked for accuracy by sending these to participants who wished to review these before analysis, and entered into NVivo 12 (QSR International Pty Ltd. Version 12, 2018). An inductive approach was used to thematically analyze and code the qualitative data (Thomas, 2006). Researchers individually read through the transcripts to familiarise themselves with the data and grouped them into common ideas or patterns (codes). Through a series of group meetings, codes were discussed and collectively developed as themes.

3 Results

3.1 Study I—Health Sciences Students

Participants’ age ranged from 18 to 48 years with the majority of the 40 participants recruited to the study identifying as female (n = 27), Asian (n = 20), having moderate (n = 18) data entry experience, and representing undergraduate pharmacy (n = 12) (Table 1).

TABLE 1

TABLE 1. Study I participant demographics (N = 40).

The mean usability scores for the two systems were significantly different {RoboGen = 33.27, SD 8.31, RoboGen2 = 39.12, SD 7.66), t (40) = −3.40, p = 0.002, mean difference −5.85 (95% CI [−9.12, −2.57]} (Figure 4). The speed of prescription entry within the allocated time was significantly faster for RoboGen2 compared with RoboGen [t (40) = 3.65, p < 0.001], with a mean difference of 1.87 min (95% CI [0.83, 2.91]). Accuracy of prescription entry error by system showed there was no significant difference in the proportion of correct prescription entries between the two systems (t = 1.12, p = 0.27), with the mean difference 0.04 (95% CI [-0.29,0.75]) (Figure 5).

FIGURE 4

FIGURE 4. System usability of RoboGen versus RoboGen2.

FIGURE 5

FIGURE 5. Errors by system. The red bars indicate errors and the blue bars indicate correct entries.

The main errors for RoboGen were incorrect medication name (including spelling mistakes) and incorrect number of medicines provided, e.g., converting 1/12 (1-month supply) to 1 instead of 30. Incorrect number of medicines provided was also common for RoboGen2, although this was more likely to be typed incorrectly instead of a conversion error. Another common error was selecting an incorrect generic name when converting from a trade (brand) name (e.g., “levonorgestrel 1.5 mg tablet” instead of “levonorgestrel 150 μg + ethinylestradiol 30 μg tablet” for Ava 30 ED tablets). Both systems had an equal number of errors with transposing first/last patient names or spelling errors.

Significantly more participants (n = 32) preferred RoboGen2 to Robogen 1 (n = 8) [Chi-Square (1,14) = 14.40, p < 0.001]. Of the 12 pharmacy students, seven preferred Robogen2 and five preferred Robogen. The majority of participants preferred RoboGen2 due to its sleek and modern layout, good flow, ease of use, and intuitive design. Participants perceived it to be more efficient (hence less time-consuming) and selecting medicines from a dropdown list was believed to allow for more accuracy, reducing risk of dispensing error. Some participants thought this system would better support medication entry for those with less medicine knowledge. Limitations of the system included the listing of medicines by generic name only. This made finding the trade name difficult, particularly for medicines with multiple active ingredients such as oral contraceptives. Some participants wanted more options to enter patient data and autofill options for medication dosing instructions. Other restrictions were using a mouse to navigate rather than arrow keys, the inability to delete a prescription, a small dropdown box for medicines, and need for multiple clicks to navigate between screens.

Those who preferred RoboGen thought it was a simpler interface allowing users to input additional patient information, and manual medicine entry was considered easier than sifting through a dropdown list. Previously entered medicines were saved, which meant they appeared in future searches and allowed for faster entry as more prescriptions were added, saving time with repeat prescriptions. Limitations were the user interface, which was seen to be “ugly,” outdated, and visually confusing. Participants thought it presented too many options, wasn’t intuitive, didn’t offer auto-fill options and the navigation was puzzling. Some participants felt it was a “real brain drain” staying focussed while using this system.

3.2 Study II—Community Pharmacists

Twenty pharmacists were recruited of which the majority were female (n = 11) and aged between 20 and 30 years (n = 13) (Table 2).

TABLE 2

TABLE 2. Study 2 participant demographics (N = 20).

The overall SUS score was 67.7, a marginal C-grade. According to the survey, most pharmacists found the task easy to do and were confident in performing the task, but were ambivalent as to whether they would like to use the system frequently.

Three main themes were developed from the interviews, namely navigation and streamlining the system, ease of use, and integration with pharmacy software systems (selected quotes in Table 3).

TABLE 3

TABLE 3. Selection of community pharmacists’ quotes.

3.2.1 Navigation and Streamlining the System

Although most pharmacists were able to navigate the system fairly easily, some did express that the software was “a bit clunky and difficult to navigate” initially. Several pharmacists thought that the process of entering prescription details could be designed to “flow better.”

3.2.2 Ease of Use

While most pharmacists found the software relatively straightforward to use, some believed that re-entering medicines with each new prescription was time-consuming and awkward. A few pharmacists suggested that if the software was connected to the medication database as their current dispensing software was, it would make the process much faster and hence easier to use. Others suggested that using a keyboard would be a faster way to enter information, which is what they were used to with their pharmacy dispensing databases.

3.2.3 Integration With Pharmacy Software Systems

The majority of pharmacists believed that integration with their current pharmacy dispensing software (Toniq™ or LOTS™) was not only a way to be more time-efficient, but also ensure patient medicines were up to date. Furthermore, their current dispensing programs had inbuilt drug interaction checking features and were connected to drug formularies where pharmacists could check for drug information. They believed that having another system would potentially increase the risk of dispensing errors.

4 Discussion

This study demonstrates that Health Sciences students preferred RoboGen2, which resulted in significantly higher usability scores and speed of medication entry when compared with an older version, but with no significant differences in the accuracy. Interviews with community pharmacists suggest that the usability of RoboGen2 could translate well into their practice, but substantial changes would need to be made to better integrate healthcare robot software into existing pharmacy dispensing systems.

Although health students were able to enter significantly more prescriptions within 15 min using RoboGen2, analysis of the errors made did not show a significant difference between the two versions. This could be due to the heterogeneity in participants’ education background and expertise levels with 52.5% of participants self-identifying as moderate to very experienced in data entry. Furthermore, 30% of the sample population were third-year pharmacy students who are trained from their second year in prescription regulation and data entry. It is not unreasonable to expect that those students with more familiarity with medication names (trade and generic), conversion rates, and prescription entry requirements would produce fewer errors regardless of the software system.

As suggested by Tractinsky (1997), altering the aesthetics of RoboGen2 increased both perceived usability and preference. In this study, while many participants found RoboGen2 to have superior usability, predominantly due to its modernized interface and autofill options, this only worked well for single active medications. Best practice guidelines in NZ promote generic prescribing, and RoboGen2 was designed with this in mind. In some instances, however, prescribers may prefer to prescribe by trade name, for example where different brands have specific formulation characteristics or products with multiple ingredients. This raises a software handling mapping issue between generic and trade names and is a major limitation of the RoboGen2 design.

Of the 20% of health sciences participants who preferred the old system, 62.5% were pharmacy students. Reasons for this preference were the ability to input more medicines and patient information giving users more autonomy in developing their own drug banks. Similarly, some participants preferred manual entry as a safety feature to prevent them from potentially selecting an incorrect formulation from the dropdown menu and jeopardizing patient safety. Medical error can occur at point-of-care where patients are given incorrect medication, wrong dose, or medication at the wrong time (Kushniruk and Borycki, 2008). Although health information technologies (HIT) have been developed to prevent medical error, and improve safety (Kushniruk et al., 2005), they can facilitate error if they fail to take human capability and cognitive limits into account (Kushniruk and Borycki, 2008). Poorly designed systems that contribute to technology-induced errors are strongly related to user interface design (Kushniruk et al., 2005).

A chief aim in HIT development is to match the user’s competencies (Johnson et al., 2005). If a system neglects to meet work demands and workflow practices and is slow and difficult to comprehend, this too can contribute to technology-induced error (Tractinsky, 1997). In this study, pharmacists were accustomed to using HIT in practice and found RoboGen2 easy to use, but the system usability scale was below 68, signifying more work needs to be done to improve usability. Amongst allied health professionals, pharmacists are the highest users of technology in their daily practice, which is thought to be associated with their aptitude for technology use (MacLure and Stewart, 2016). Nonetheless, for Robogen2 to be successfully implemented in pharmacy practice, it needs to be integrated with pharmacists’ current dispensing software systems to keep patient information up to date and accurate. Traditional dispensing systems include clinical decision support systems, such as drug interaction checking, which reduce the risk for medical error and improve quality of care and patient safety (Alotaibi and Federico, 2017), and must be integrated with medication entry software. Integration with dispensing systems also allows pharmacists to connect with medicines formularies and essential drug information resources, as well as medicine prices, subsidized medicines, refill authorizations, medicines reconciliation, inventory management, and more (Odukoya and Chui, 2012). Failure to integrate systems leads to an increase in prescription processing time and has major effects on workflow and load. However, integration only works if there is a standardized set of interfaces between systems, otherwise it requires re-implementation for each pharmacist system. An alternate approach would be to utilize data from a central database of prescriptions. While NZ does have a central database, this store is not available for use outside of general practice and pharmacist systems.

Pharmacists found RoboGen2 more time-consuming to use than their current dispensing software, and the number of tabs awkward. Pharmacists deemed it essential to have patient, prescriber and medication details visible to reduce the risk of duplication or omission of medicines. For medication entry software to be useful with processing repeat prescriptions, it must also be able to recall previous entries to reduce time spent on the activity. Mixed findings have been reported regarding the adoption of HIT in pharmacy to reduce time spent on medication entry. Where some technologies have reduced dispensing time (Motulsky et al., 2008; Rahimi and Timpka, 2011), others were found impractical to use and increased workload (Nanji et al., 2009). As pharmacists may use HIT more comprehensively than other healthcare professionals resulting in additional needs, failing to involve them in HIT testing may result in reduced efficacy and efficiency of the system that could impact patient safety and cause frustration for pharmacists (Darby et al., 2019).

The study had several limitations. Student participants varied widely in age, education, data entry experience, cultural background, and skill level. As participants were health sciences students, results may not generalize to other groups. Recruitment of pharmacists was limited to the central urban Auckland area, the majority of whom were under the age of 30. Whilst NZ community pharmacists use one of two medication dispensing software, pharmacists working in hospitals or general practices may have experiences with software not accounted for in this study. Selection bias could have recruited pharmacists who were more open-minded or interested in technologies, and several pharmacists were recruited from the same pharmacy, increasing the risk for groupthink and perhaps limiting variability of responses. Furthermore, the user will not always be a pharmacist. In the previous COPD study and retirement home study, health professionals who entered the patient medication and other information, were physiotherapists, doctors, or nurses. These healthcare professionals may have different levels of experience and different opinions, and this may mean the robot needs to be tailored to the users.

This paper presents ongoing developments and testing of medication management software for robotic systems. The overall aim of the research was to identify how to improve the usability of the Robogen software for health professionals. The results showed that Robogen2 improved system usability and efficiency but not accuracy, which partially supports the hypothesis. Further developments are necessary to make the software more compatible with existing pharmaceutical systems for pharmacists in particular. Key learnings are that adding safety features and better aesthetics can improve the overall usability and safety of a medication prescription system. By streamlining the workflow and pre-populating data, it is possible to increase the rate that prescriptions are entered, without compromising patient safety. However, the next step could be to integrate the pre-existing pharmacy dispensing systems with specialist medication management systems. This could reduce the overall workload for pharmacists, while incorporating many of the safety features that are built into dispensing systems. Future development of the system for pharmacists should link the software to existing pharmacy systems, and further improve the interface.

Data Availability Statement

The raw data supporting the conclusion of this article will be made available by the authors, without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved by the University of Auckland Human Participants Ethics Committee (Reference numbers 020792 and 021323). The participants provided their written informed consent to participate in this study.

Author Contributions

EB, CS and NM contributed to the conception and design of the work. RM-W, JK, SZ, LL, and DV acquired the data. All authors contributed to analysing and interpreting the data. NM wrote the manuscript. EB, CS and NM edited the manuscript and all authors approved it for publication.

Funding

This research was supported by the School of Pharmacy and Centre for Automation and Robotics Engineering Science, University of Auckland.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

We would like to extend our thanks to the participants who gave up their time to participate in this study.

References

Alotaibi, Y. K., and Federico, F. (2017). The Impact of Health Information Technology on Patient Safety. Smj 38, 1173–1180. doi:10.15537/smj.2017.12.20631

PubMed Abstract | CrossRef Full Text | Google Scholar

Bangor, A., Kortum, P., and Miller, J. (2009). Determining what Individual SUS Scores Mean: Adding an Adjective Rating Scale. J Usability Studies. 4 (3), 114–123. doi:10.5555/2835587.2835589

CrossRef Full Text | Google Scholar

Broadbent, E., Garrett, J., Jepsen, N., Li Ogilvie, V., Ahn, H. S., Robinson, H., et al. (2018). Using Robots at Home to Support Patients with Chronic Obstructive Pulmonary Disease: Pilot Randomized Controlled Trial. J. Med. Internet Res. 20, e45. doi:10.2196/jmir.8640

PubMed Abstract | CrossRef Full Text | Google Scholar

Broadbent, E., Montgomery Walsh, R., Martini, N., Loveys, K., and Sutherland, C. (2020). “Evaluating the Usability of New Software for Medication Management on a Social Robot,” in Companion of the 2020 ACM/IEEE International Conference on Human-Robot Interaction, Cambridge, United Kingdom, March 23–26, 2020, 151–153. doi:10.1145/3371382.3378320

CrossRef Full Text | Google Scholar

Broadbent, E., Peri, K., Kerse, N., Jayawardena, C., Kuo, I., Datta, C., et al. (2014). “Robots in Older People's Homes to Improve Medication Adherence and Quality of Life: A Randomised Cross-Over Trial,” in Social Robotics Lecture Notes in Computer Science. Editors M. Beetz, B. Johnston, and M.-A. Williams (Cham: Springer International Publishing), 64–73. doi:10.1007/978-3-319-11973-1_7

CrossRef Full Text | Google Scholar

Darby, A. B., Su, Y., Reynolds, R. B., and Madlock-Brown, C. (2019). A Survey-Based Study of Pharmacist Acceptance and Resistance to Health Information Technology. Perspect. Health Inf. Manag. 16, 1a. Available at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6462883/(Accessed January 17, 2021).

PubMed Abstract | Google Scholar

Datta, C., Tiwari, P., Yang, H. Y., Kuo, I., Broadbent, E., and MacDonald, B. (2012). “An Interactive Robot for Reminding Medication to Older People,” in 2012 9th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI), Daejeon, Korea (South), 26-28 Nov. 2012, 190. doi:10.1109/urai.2012.6462967

CrossRef Full Text | Google Scholar

Datta, C., Yang, H. Y., Tiwari, P., Kuo, I. H., and MacDonald, B. A. (2011). “End User Programming to Enable Closed-Loop Medication Management Using a Healthcare Robot,” in Australasian Conference on Robotics and Automation (Melbourne Australia: Monash University).

Google Scholar

World Health Organization (2003). in Adherence to Long-Term Therapies: Evidence for Action. Editor E. Sabaté (Geneva: World Health Organization).

Google Scholar

Granger, B. B., and Bosworth, H. B. (2011). Medication Adherence: Emerging Use of Technology. Curr. Opin. Cardiol. 26, 279–287. doi:10.1097/HCO.0b013e328347c150

PubMed Abstract | CrossRef Full Text | Google Scholar

Hall, J., Bond, C., Kinnear, M., and McKinstry, B. (2016). Views of Patients and Professionals about Electronic Multicompartment Medication Devices: a Qualitative Study. BMJ Open 6, e012915. doi:10.1136/bmjopen-2016-012915

CrossRef Full Text | Google Scholar

Johnson, C. M., Johnson, T. R., and Zhang, J. (2005). A User-Centered Framework for Redesigning Health Care Interfaces. J. Biomed. Inform. 38, 75–87. doi:10.1016/j.jbi.2004.11.005

CrossRef Full Text | Google Scholar

Kurosu, M., and Kashimura, K. (1995). “Apparent Usability vs. Inherent Usability: Experimental Analysis on the Determinants of the Apparent Usability,” in Conference Companion on Human Factors in Computing Systems CHI ’95 (New York, NY, USA: Association for Computing Machinery), 292–293. doi:10.1145/223355.223680

CrossRef Full Text | Google Scholar

Kushniruk, A. W., and Borycki, E. M. (2008). Human, Social, and Organizational Aspects of Health Information Systems. Hershey, United States: IGI Global. Available at: http://ebookcentral.proquest.com/lib/auckland/detail.action?docID=3309308 (Accessed December 18, 2020).

Google Scholar

Kushniruk, A. W., Triola, M. M., Borycki, E. M., Stein, B., and Kannry, J. L. (2005). Technology Induced Error and Usability: The Relationship between Usability Problems and Prescription Errors when Using a Handheld Application. Int. J. Med. Inform. 74, 519–526. doi:10.1016/j.ijmedinf.2005.01.003

CrossRef Full Text | Google Scholar

Law, M., Zeng, S., Koo, J., Verches, D., Lam, L., and Martini, N. (2021). Perceptions of Community Pharmacists to Implementing Technologies in the Workplace: an Exploratory Study. Int. J. Clin. Pharm. 43, 1227–1236. doi:10.1007/s11096-021-01238-x

CrossRef Full Text | Google Scholar

MacLure, K., and Stewart, D. (2016). Digital Literacy Knowledge and Needs of Pharmacy Staff: A Systematic Review. jhi 23, 560. doi:10.14236/jhi.v23i3.840

PubMed Abstract | CrossRef Full Text | Google Scholar

Mansoor, S. M., Aslani, P., and Krass, I. (2014). Pharmacists' Attitudes and Perceived Barriers to Provision of Adherence Support in Australia. Int. J. Clin. Pharm. 36, 136–144. doi:10.1007/s11096-013-9840-6

CrossRef Full Text | Google Scholar

Ministry of Health (2020a). Health and Independence Report 2017. Ministry of Health NZ. Available at: https://www.health.govt.nz/publication/health-and-independence-report-2017 [Accessed July 14, 2020a].

Google Scholar

Ministry of Health (2020b). Health Loss in New Zealand 1990–2013. Ministry of Health NZ. Available at: https://www.health.govt.nz/publication/health-loss-new-zealand-1990-2013 [Accessed July 14, 2020b].

Google Scholar

Motulsky, A., Winslade, N., Tamblyn, R., and Sicotte, C. (2008). The Impact of Electronic Prescribing on the Professionalization of Community Pharmacists: A Qualitative Study of Pharmacists' Perception. J. Pharm. Pharm. Sci. 11, 131–146. doi:10.18433/J3X59G

CrossRef Full Text | Google Scholar

Nanji, K. C., Cina, J., Patel, N., Churchill, W., Gandhi, T. K., and Poon, E. G. (2009). Overcoming Barriers to the Implementation of a Pharmacy Bar Code Scanning System for Medication Dispensing: A Case Study. J. Am. Med. Inform. Assoc. 16, 645–650. doi:10.1197/jamia.M3107

CrossRef Full Text | Google Scholar

Navti, B., and Apampa, B. (2019). Pharmaceutical Care Services to People Living with Dementia in Care Homes: A Qualitative Study of Community Pharmacists' Perceptions. Dementia 18, 2282–2302. doi:10.1177/1471301217743305

PubMed Abstract | CrossRef Full Text | Google Scholar

Odukoya, O., and Chui, M. A. (2012). Retail Pharmacy Staff Perceptions of Design Strengths and Weaknesses of Electronic Prescribing. J. Am. Med. Inform. Assoc. 19, 1059–1065. doi:10.1136/amiajnl-2011-000779

CrossRef Full Text | Google Scholar

Patton, D. E., Hughes, C. M., Cadogan, C. A., and Ryan, C. A. (2017). Theory-Based Interventions to Improve Medication Adherence in Older Adults Prescribed Polypharmacy: A Systematic Review. Drugs Aging 34, 97–113. doi:10.1007/s40266-016-0426-6

PubMed Abstract | CrossRef Full Text | Google Scholar

Pinto, S., Simon, A., Osundina, F., Jordan, M., and Ching, D. (2018). Study to Measure the Impact of Pharmacists and Pharmacy Services (STOMPP) on Medication Non-adherence: Medication Adherence and Clinical Outcomes. Innov. Pharm. 9, 11. doi:10.24926/iip.v9i1.1104

CrossRef Full Text | Google Scholar

Rahimi, B., and Timpka, T. (2011). Pharmacists' Views on Integrated Electronic Prescribing Systems: Associations between Usefulness, Pharmacological Safety, and Barriers to Technology Use. Eur. J. Clin. Pharmacol. 67, 179–184. doi:10.1007/s00228-010-0936-9

CrossRef Full Text | Google Scholar

Rantanen, P., Parkkari, T., Leikola, S., Airaksinen, M., and Lyles, A. (2017). An in-home Advanced Robotic System to Manage Elderly Home-care Patients' Medications: A Pilot Safety and Usability Study. Clin. Ther. 39, 1054–1061. doi:10.1016/j.clinthera.2017.03.020

PubMed Abstract | CrossRef Full Text | Google Scholar

Thomas, D. R. (2006). A General Inductive Approach for Analyzing Qualitative Evaluation Data. Am. J. Eval. 27, 237–246. doi:10.1177/1098214005283748

CrossRef Full Text | Google Scholar

Torres-Robles, A., Benrimoj, S. I., Gastelurrutia, M. A., Martinez-Martinez, F., Peiro, T., Perez-Escamilla, B., et al. (2021). Effectiveness of a Medication Adherence Management Intervention in a Community Pharmacy Setting: a Cluster Randomised Controlled Trial. BMJ Qual. Saf., bmjqs–2020. doi:10.1136/bmjqs-2020-011671

CrossRef Full Text | Google Scholar

Tractinsky, N. (1997). “Aesthetics and Apparent Usability: Empirically Assessing Cultural and Methodological Issues,” in Proceedings of the ACM SIGCHI Conference on Human factors in computing systems CHI ’97 (New York, NY, USA: Association for Computing Machinery), 115–122. doi:10.1145/258549.258626

CrossRef Full Text | Google Scholar

Velligan, D., Mintz, J., Maples, N., Xueying, L., Gajewski, S., Carr, H., et al. (2013). A Randomized Trial Comparing in Person and Electronic Interventions for Improving Adherence to Oral Medications in Schizophrenia. Schizophrenia Bull. 39, 999–1007. doi:10.1093/schbul/sbs116

CrossRef Full Text | Google Scholar

World Health Organization (2021). Noncommunicable Diseases. Available at: https://www.who.int/news-room/fact-sheets/detail/noncommunicable-diseases [Accessed January 1, 2022].

Google Scholar