Improving TB Surveillance and Patients' Quality of Care Through Improved Data Collection in Angola: Development of an Electronic Medical Record System in Two Health Facilities of Luanda

TB Programs should promote the use of digital health platforms, like Electronic Medical Records (EMR) to collect patients' information, thus reducing data incompleteness and low accuracy and eventually improving patients' care. Nevertheless, the potential of digital health systems remains largely unexploited in low-resource settings. Angola is one of the 14 countries with a triple burden of TB, TB/HIV and MDR-TB (multidrug-resistant TB) and it is among the three countries, together with Congo and Liberia that have never completed a drug-resistance survey so far. The Sanatorium Hospital of Luanda and the Tuberculosis Dispensary of Luanda are the two reference health facilities in Luanda dealing with most of the TB cases, and they both rely entirely on paper-based data collection. The aim of this paper is to describe a three-stage process for the development of a TB EMR system in these two health facilities of Luanda and to share the lessons learned. The description is focused on the activities that took place from March 2019 to January 2020. Main lessons learned were identified in the importance of engaging all the stakeholders in the development process, in the mainstream of the “think digital” transition, in the promotion of a monitoring and evaluation (M&E) culture and in the planning of the system's sustainability. This approach may be replicated in similar contexts where the development of a TB EMR system is sought, and the lessons learned could assist and facilitate the programming of the interventions.

Introduction

The Tuberculosis (TB) pandemic, including its multidrug-resistant (MDR-TB) form, is a global health threaten. TB is one of the top 10 causes of death worldwide and the leading cause of death from a single infectious agent. An estimated 10 million people fell ill with TB in 2019 and 1.2 million people died (1).

The WHO End TB Strategy aims to end the global TB epidemic, with targets to reduce TB deaths by 95% and incidence by 90% between 2015 and 2035, and to ensure that no family is burdened with catastrophic expenses due to TB (2). In the years 2015–2019, TB incidence rate dropped by 9%, however this was less than halfway to reach the End TB Strategy milestone of 20% incidence reduction between 2015 and 2020 (3).

To achieve the End TB strategy goals and improve patients' management, effective surveillance systems are needed to measure and monitor the burden and the determinants of the disease. Implementing a functional TB surveillance system often remains challenging in many low- and middle-income countries (4). TB programs should promote the use of digital health platforms to collect patients' information, to reduce time and efforts to fill paper forms and to avoid common mistakes like data incompleteness and low accuracy. Digital health interventions can strengthen surveillance and monitoring, program management and ultimately patients' care (5).

Introducing a new digital health system needs an enabling environment and to deploy a formative stage to document bad and good practices, pivotal for the uptake of the new system (6). Therefore, a comprehensive approach is required, that starts with gaps and needs identification and the description of the patient's pathway for TB care (7).

Angola is among the 14 high burden countries for TB, TB/HIV and MDR-TB prevalence and is among the three countries, together with Congo and Liberia, that have never completed a drug-resistance survey so far (1). Moreover, according to the WHO country profile, Angola had a treatment success rate of 25% in 2017, much lower as compared to a pooled rate of 76.2% in sub-Saharan Africa countries in the last decade, according to a recent systematic review (8).

The Angola TB network is made of Sanatoriums, TB Dispensaries (DAT), Diagnostic and Treatment Units (UDT with smear laboratory), and Treatment Units (UT without smear laboratory). The TB surveillance system in Angola is weak, with main gaps regarding cases notification, data reporting and elaboration, and quality of data collected (9). The Angolan TB National Program (PNCT) uses a digital database to collect aggregated data from the network, however countrywide TB facilities continue to rely on paper-based tools leading to incompleteness, poor accuracy of data and delays in reporting. Therefore, it is imperative to enhance data collection and management in order to strengthen TB surveillance, programmatic decisions and patient's management. To this purpose, the implementation of user-friendly Electronic Medical Record (EMR) systems could improve data quality and strengthen the continuum of care by ensuring adherence to clinical guidelines and reducing errors in data recording and reporting. A study based in Malawi documented the positive outcomes of a point-of-care EMR system for HIV-TB patients in a public clinic (10). In Rwanda the use of EMR data for evidence-based clinical decisions improved HIV patients monitoring (11). A study in Kenya reported significant improvements in data quality through the implementation of a cloud-based EMR system for maternal and child health (12).

A systematic review showed that the factors that challenged the widespread use of EMR systems in Sub-Saharan Africa were the high costs of procurement and maintenance of the EMR system, poor implementation planning, issues with electricity supply and internet connectivity, and user's limited computer skills (13). Strategies such as phased implementation planning, financial sustainability, appropriate EMR system and training of users, showed positive results in EMR systems implementation.

Context

Luanda is the capital city of Angola with one-third of the Angolan population living in conditions of overcrowding and poor sanitation. Luanda has the highest TB mortality rate in the country (13.9^*100.000 inhabitants) and 245 MDR-TB cases over 534 (46%) notified at country level in 2018 (14). A recent study also showed that in Luanda chronic diseases like diabetes are an important comorbidity for TB patients (15).

The Sanatorium Hospital of Luanda (HSL) and the Anti-Tuberculosis Dispensary (DAT) of Luanda are the two reference health facilities in Luanda dealing with most of the TB cases, and being also reference centers for the diagnosis and management of MDR-TB (HSL) and provincial reference center for the PNCT (DAT). These two health facilities collect clinical and surveillance data entirely on paper medical records and registries and this may result in poor quality of data and ultimately poor patients' care.

The project “Stop TB and TB/HIV in Angola: Improving Access to TB and HIV treatment by enhancing diagnostic quality and patient management in the Province of Luanda” funded by the Italian Cooperation Agency (AICS—Agenzia Italiana per la Cooperazione allo Sviluppo) and implemented by the Italian non-governmental organization “Doctors with Africa CUAMM” with the technical support of the “Italian National Institute of Health” (ISS) aimed at improving the quality of diagnosis and management of TB and TB/HIV patients at the HSL and the DAT of Luanda. The project, among other activities, included the development of a TB EMR system, based on an enhanced TB medical record and the setting up of software and hardware features, to be piloted in the two health facilities. The EMR system development was a three-stage process and included first a situation analysis, second the development of an enhanced medical record and finally the development of the EMR system. The situation analysis supported the identification of the patients' flow and actual gaps and needs in data recording and reporting at the HSL and the DAT of Luanda. The results of the situation analysis were used to develop an enhanced medical record that aimed at filling the gaps and needs identified, and finally in the development of the EMR system. The process was consolidated through a workshop for the staff of the two health facilities (around 40 people) involved in the data collection with the double aim of promoting awareness about the importance of data quality and getting acquainted with the TB EMR system.

The aim of this paper is to describe the three-stage process for the development of the EMR system and share the lessons learned. The description is focused on the activities that took place from March 2019 to January 2020.

The process could be replicated in other health facilities in Angola and other similar contexts to improve TB surveillance, control and quality of care, to boost the achievement of the WHO END TB strategy.

Stage 1: Lay the Foundation for the EMR System Development

Stakeholders' Engagement

A series of meetings, interviews, site visits and observations were arranged involving local stakeholders in order to engage them in the design of the EMR system and to promote ownership and sustainability of the system. Representatives of the Ministry of Health and local authorities, key staff of the two health facilities, health care workers (HCWs), international organizations and other local NGOs with similar experiences were involved in this step, to gain their insights and recommendations, discuss perceived barriers and come to a consensus on a plan of actions for the implementation of the EMR system and ensure its adherence to the national guidelines. The process involved around 20 stakeholders that were constantly consulted along the duration of the project.

Situation Analysis

From March to September 2019 the EMR expert team, including two epidemiologists from the Italian National Institute of Health, CUAMM project manager, a TB specialist consultant and a software developer consultant, conducted specific site visits at the two health facilities to gain full knowledge and understanding of the patients' flow and data collection system. The final aim was to design a user-friendly EMR system that could meet the stakeholders needs, while improving data collection and reporting. During the visits, the TB patients' flow was analyzed from the entrance to the discharge and the data collection points, established by the health facilities in accordance with the patient's assistance and care pathway, were identified and described, including the data collection tools in use.

Interviews with key staff and HCWs at the two health facilities were also carried out to feed their suggestions into the development of the EMR system. A purposive sample of data was collected and analyzed with the aim of assessing the quality of the information collected.

The situation analysis allowed to identify strengths and weaknesses of the current data collection system in the two health facilities and to define areas of improvement. Moreover, this stage identified opportunities and threats to the implementation of the EMR system.

Strengths, weaknesses, opportunities, and threats (SWOT) that resulted from the situation analysis are reported in Table 1.

TABLE 1

Table 1. SWOT analysis.

Areas of Improvement (Weaknesses)

Data collection and reporting presented some gaps and inaccuracies, mostly linked to the great flow of data to be collected on paper-based tools, which could also undermine data privacy and security. Main areas of improvement were identified in the use of multiple forms and codes for the same patient that could lead to misinterpretation of data, including patients' identification, or loss of information. In addition, the need to improve the timeliness and accuracy of internal and external reports that rely only on paper-based registers emerged as an important issue, to support programmatic decision-making and TB surveillance. Finally, some cruxes along the pathway of patient's care do not provide for the collection of data, thus some essential information for patients' management is lost.

An adequate M&E system should be implemented, and data/information quality should be enhanced, particularly regarding the quality of patient's information recorded (eg., completeness, readability, numeration etc.,); quality of clinical management of patients' information (eg., bacilloscopy results/timing, HIV test result, body weight etc.,); quality of data flow information (eg., timing of internal reports, coherence between registries and reports, reports availability etc.,). As an example, patients that do not perform bacilloscopy (BK) are often reported as BK negative, while they should be reported as not assessed. Reported cases of relapses are very few, raising the question that relapses might be often treated as new cases without ascertaining the possible development of a resistant form of TB. The final treatment outcome is most of the time absent in the registries, possibly linked to the poor patient's follow-up.

At the time of visiting the facilities there was no updated TB manual providing consolidated guidelines about recording and reporting procedures, which might at least partially explain the occurrence of the shortcomings noticed above. The same can be said for the management of MDR-TB.

The areas of improvement identified suggested the need to enhance the medical record currently in use and refine the current data collection and reporting system, to improve patients' management and quality of care (Table 2).

TABLE 2

Table 2. Examples of indicators of weaknesses in the current data collection system and the correspondent EMR system response.

Stage 2: Enhancing the TB Medical Record

After discussing the gaps and needs of the current data collection system emerged during the situation analysis, the EMR expert team fed the results in the elaboration of an enhanced TB medical record. To this purpose, the two medical records and patients' forms in use at the two health facilities were used as a starting point. The DAT used the standard clinical record adopted by the National TB Program with essential information about diagnosis and treatment. The HSL medical record was based on the National TB Program standard record, but included more information, reflecting its role in the management of complex cases and MDR-TB cases.

A preliminary version of the enhanced TB medical record was discussed with key stakeholders to assess information clarity, content and sequencing, and compliance to national guidelines, and adjusted accordingly.

A formative workshop was arranged in January 2020 in the two health facilities to introduce the HCWs (around 40) to the enhanced medical record and to instruct them about their use for the digitalised data collection. The formative workshop allowed the HCWs to increase their awareness about the importance of data collection for TB surveillance, for the quality of the patient management and to enhance the accuracy of data reported to the National TB Programme for programmatic decisions.

The enhanced medical record is divided into several sections that follow the patient's clinical pathway (Table 3).

TABLE 3

Table 3. Enhanced TB medical record sections description.

Stage 3: Setting up the EMR System

The EMR expert team fed the results of the situation analysis and the contents of the enhanced medical record in the design of the EMR system. Software and hardware components were defined, according to the stakeholders' needs and recommendations. The EMR system has been designed as a real-time, point-of-care, internet-based system based on open-source software that employs C# and Java language and the MSSQL server 2017. The system is available in Portuguese and English. Although the system is web based, and it allows to work online and offline, so far it has been configured to be used on a LAN (Intranet) at the health facility level in offline modality only.

The system is accessible only to authorized staff that has been provided with username and password and keeps an audit trail by tracking users, locations, and time for all the accesses. Data is encrypted during offsite data back up using secure encryption protocols. The system owns the technical features needed to communicate with the District Health Information Software 2 (DHIS2) (16), the Health Management Information System that is increasingly being used by the TB National Program for aggregate data reporting. The system can produce internal and external reports according to formats and indicators in use in the two health facilities and within the TB National Program. Multiple testing sessions assessed the usability of the EMR system.

A print screen of the EMR system is shown in Figure 1.

FIGURE 1

Figure 1. EMR system print screen.

Lessons Learned

The following lessons learned were collected during the project, through a continuous consultation process with the stakeholders involved.

Engage the Stakeholders

Local stakeholders were identified as the representatives of local health authorities, key staff and HCWs of the two health facilities and other TB health facilities, representatives from international organizations and NGOs working in the same area. Engaging all the stakeholders in an inclusive process allowed to promote the ownership and buy-in of the final product and its sustainability, to encourage trust among all the actors, to share a common vision about how to overcome the challenges and to identify capacity building needs (10).

Introducing innovation in information systems needs to be supported by an enabling environment, since it could be perceived by the final users more as a challenge than an advantage, unless tangible benefits are clearly communicated (17). As an example, all the stakeholders agreed that the reporting process was time-consuming, resulting in delays and inaccuracies, therefore the new EMR system aimed at simplifying the process while improving the quality of the data reported. In addition, the stakeholders agreed to the fact that the time spent by the HCWs recording data on paper-based forms and registries, could have been better employed for the management of the patients and for improving the quality of care.

The stakeholders were engaged during the whole process and were consulted at each step, to consult with them about the best strategy to meet their needs. This allowed to enhance the medical record with the information they needed to improve the management of the patients. The patient socio-economic and clinical data were enriched, and even if this may increase the duration of the first visit, the information collected will enable a better care of the patient. The pilot of the system should take this aspect into consideration to find an operational balance.

Mainstream the “Think Digital” Transition

Digital health is a cultural transformation, mostly in low- and middle-income countries, where technology is not yet fully exploited. Local stakeholders endorsed the digital transition and “think digital” promoters in the two health facilities were asked to endorse the digital health cause and to provide evidence of the benefits to the other HCWs (4). These promoters started to support HCWs of the two health facilities few months before the EMR system piloting, in order to guide them to a smooth transition to the digital system and highlight the gaps that the EMR system could fill.

Challenges to the “think digital” transition were mostly identified in the poor digital skills of the HCWs and resistance to the new system due to previous negative experiences. To address these challenges, the EMR system was developed as an easy-to-use system, with a simple interface to reduce complexity. Standard desktop computers with large screens were bought and a basic IT training has been planned at the beginning of the piloting phase to uniform digital skills of the HCWs. Continuous training on the job activities and a stepwise rollout of the EMR system during the piloting stage was planned to allow the HCWs to adapt to the transition from a paper-based system, without causing any disruption to the daily activities.

The development of the EMR system had to be gradual, to have all the stakeholders on board and feed all the recommendation in the process. The organizational leadership of the two health facilities supported the development of the system and addressed critical issues that arose during the process.

Promotion of an M&E Culture

The situation analysis showed the importance of strengthening the monitoring and evaluation system and culture, particularly the quality and the regular evaluation of the data collected. Moreover, it highlighted the necessity to define roles and responsibilities for M&E activities. From the meetings with local stakeholders and HCWs of the two health facilities it was evident that awareness-raising activities about the importance of data quality for TB surveillance and patients' care, should be regularly performed.

Highlighting the practical advantages of the EMR system for data accuracy and timeliness during the recording and reporting to the National TB Program, was a strong argument that allowed to sow the seeds for an M&E and accountability culture to grow among the stakeholders. Another opportunity to promote M&E culture was to identify M&E champions among local stakeholders and in the two health facilities, to encourage their peers to effective data collection, monitoring and evaluation. The M&E champions were seen by the staff of the two health facilities as a valuable support to improve their M&E and data quality skills. Linking the concept of data quality to a better management of patients, to a reduced time in reporting and to an improved functionality of the facility allowed the HCWs to consider data quality as a priority. The M&E culture promotion will continue to be strengthen during the planned training on the job activities during the the piloting stage of the EMR system.

During the formative workshop in January 2020 in the two health facilities, HCWs were introduced to the importance of accuracy, transparency and accountability of data collected and reported to improve TB surveillance, programmatic decisions and clinical management of patients.

Plan for Sustainability

Stakeholders' engagement was the first step toward the EMR system sustainability, in order for them to endorse the ownership and the buy in of the system (18). After the piloting and evaluation phases of the EMR system, the local stakeholders would need to endorse the system running cost that consist in the hardware and software maintenance and possibly in capacity building activities, like training on the job to increase uptake and utilization of the system by the HCWs.

The involvement of local software programmers in the development of the EMR system will allow system improvement and maintenance beyond the project, without the need to involve foreign consultants, and therefore decreasing the costs. As a matter of fact, looking for in-country expertise can promote initiatives that are more likely to be sustainable than those depending heavily on external support (4).

The possibility for the EMR system to be integrated with the DHIS-2 software, conferred to the EMR system the possibility to be extended to other health units in the country. Also, the flexible features of the system, the user-friendly interface and the easy programming language, will allow it to adapt to different needs and skills, therefore increasing its chance of scalability and sustainability.

Limitations

The situation analysis was done only in two health facilities. Other relevant needs of other health facilities of the Angolan TB Network might not have been considered.

However, the enhanced medical record and the EMR system were developed considering comprehensive needs of all the stakeholders involved, related to the enhancement of data collection and management, in order to allow replicability in other TB services in Angola.

Conclusion

This study was a first attempt to promote a digital health intervention in the context of the TB National Program in Angola. The study was a three-stage process. The first stage allowed the identification of gaps and needs in the data collection systems of the two health facilities. The second stage included the development of the enhanced medical record, based on the findings of the situation analysis. The third stage supported the development of the EMR system. The EMR system developed in the two health facilities in Luanda owns all the features to improve data collection, data quality and finally patients' management and could be exported to the other health facilities across the country.

The main lessons learned in the process were to engage all the involved stakeholders since the beginning and during all the stages; to promote the “think digital” cultural revolution; to support the development of an M&E system through all the steps of the data collection flow; to include sustainability of the system among the priorities areas to be discussed before starting the process. The lessons learned were similar to other studies in similar context (10), thus reinforcing their relevance.

Covid-19 pandemic caused severe disruption in the planned activities for the piloting of the EMR system in the two health facilities. Nevertheless, once the health emergency situation will decrease, the system will be piloted, evaluated and refined. According to other similar studies, during the piloting stage of the EMR system, particular care needs to be focused on enhancing the accuracy of the data recorded (19, 20). The evaluation of the EMR system will target the actual impact of the system on data quality and quality of care. Weaknesses related to data collection that were highlighted during the situation analysis, and the compliance to the EMR system requirements, will be assessed. This will also include an analysis on how well the EMR system matches the actual workflow of patients, and if this could be improved to enhance patient's care.

The TB EMR system would be an important tool to be scaled up to other TB services of the Angolan network, allowing to improve data collection and management and finally to strengthen national TB surveillance, programmatic decision-making and patients' quality of care.

Data Availability Statement

The data were retrieved from patients' registries. Requests to access these datasets should be directed to c.robbiati@cuamm.org.

Author Contributions

CR, MD, MT, and GM contributed to conception and study design, data acquisition, and interpretation of results and drafting the manuscript. GP and FD contributed to study conception and interpretation of results and revised the manuscript critically for important intellectual content. PS, OL, and JN contributed to data acquisition and revised the manuscript critically for important intellectual content. MS contributed to study conception and revised the manuscript critically for important intellectual content. All authors read and approved the final manuscript.

Funding

This study was performed with the support of the Italian Cooperation Agency (AICS—Agenzia Italiana per la Cooperazione allo Sviluppo).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

The authors would like to dedicate this article to the late JN, whose incessant efforts largely contributed to the progress in the fight of TB in Angola. We would like to deeply thank the Angolan Institutions, the directors and staff of the HSL and DAT of Luanda, for their willingness to take part and support the project. Our acknowledgement also goes to all the people who worked within the project and to the Angolan population.

References

1. WHO. Global Tuberculosis Report 2020. (2021). Available online at: https://www.who.int/teams/global-tuberculosis-programme/tb-reports/global-tuberculosis-report-2020 (accessed January 21, 2021).

Google Scholar

2. Uplekar M, Weil D, Lonnroth K, Jaramillo E, Lienhardt C, et al. WHO's new end TB Strategy. Lancet. (2015) 385:1799–801. doi: 10.1016/S0140-6736(15)60570-0

PubMed Abstract | CrossRef Full Text | Google Scholar

3. WHO. Tuberculosis (TB). (2021). Available online at: https://www.who.int/en/news-room/fact-sheets/detail/tuberculosis (accessed January 21, 2021).

Google Scholar

4. Falzon D, Timimi H, Kurosinski P, Migliori G, Van Gemert W, Denkinger C, et al. Digital health for the End TB Strategy: developing priority products and making them work. Eur Respir J. (2016) 48:29–45. doi: 10.1183/13993003.00424-2016

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Reid M, Arinaminpathy N, Bloom A, Bloom BR, Boehme C, Chaisson R, et al. Building a tuberculosis-free world: the lancet commission on tuberculosis. Lancet. (2019) 393:1331–84. doi: 10.1016/S0140-6736(19)30024-8

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Squire S, Ramsay A, van den Hof S, Millington K, Langley I, Bello G, et al. Making innovations accessible to the poor through implementation research [State of the art series. Operational research Number 7 in the series]. Int J Tuberc Lung Dis. (2011) 15:862–70. doi: 10.5588/ijtld.11.0161

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Sismanidis C, Shete P, Lienhardt C, Floyd K, Raviglione M. Harnessing the power of data to guide local action and end tuberculosis. J Infect Dis. (2017) 216:S669–72. doi: 10.1093/infdis/jix374

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Izudi J, Semakula D, Sennono R, Tamwesigire I, Bajunirwe F. Treatment success rate among adult pulmonary tuberculosis patients in sub-Saharan Africa: a systematic review and meta-analysis. BMJ Open. (2019) 9:e029400. doi: 10.1136/bmjopen-2019-029400

PubMed Abstract | CrossRef Full Text | Google Scholar

9. República de Angola Ministério da Saúde. Relatório Anual 2017. Luanda: Programa Nacional De Controlo Da Tuberculose (2017).

Google Scholar

10. Tweya H, Feldacker C, Gadabu OJ, Ng'ambi W, Mumba SL, Phiri D, et al. Developing a point-of-care electronic medical record system for TB/HIV co-infected patients: experiences from Lighthouse Trust, Lilongwe, Malawi. BMC Res Notes. (2016) 9:146. doi: 10.1186/s13104-016-1943-4

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Karema N, Manzi A, Gaju E, Kayiganwa M, Remera E, Nshimyiryo A, et al. Using electronic medical record data to improve HIV patient monitoring, clinical decision-making, and quality improvement: lessons from Rwanda. Stud Health Technol Inform. (2015) 216:880.

PubMed Abstract | Google Scholar

12. Haskew J, Rø G, Saito K, Turner K, Odhiambo G, Wamae A, et al. Implementation of a cloud-based electronic medical record for maternal and child health in rural Kenya. Int J Med Inform. (2015) 84:349–54. doi: 10.1016/j.ijmedinf.2015.01.005

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Odekunle FF, Odekunle RO, Shankar S. Why sub-Saharan Africa lags in electronic health record adoption and possible strategies to increase its adoption in this region. Int J Health Sci. (2017) 11(4):59-64.

PubMed Abstract | Google Scholar

14. República de Angola Ministério da Saúde. Relatório Anual 2017. Luanda: Programa Nacional De Controlo Da Tuberculose (2017).

Google Scholar

15. Segafredo G, Kapur A, Robbiati C, Joseph N, de Sousa J, Putoto G, et al. Integrating TB and non-communicable diseases services: pilot experience of screening for diabetes and hypertension in patients with tuberculosis in Luanda, Angola. PLoS ONE. (2019) 14:e0218052. doi: 10.1371/journal.pone.0218052

PubMed Abstract | CrossRef Full Text | Google Scholar

16. DHIS2. Collect, Manage, Visualize and Explore your Data. (2021). Available online at: https://www.dhis2.org/ (accessed January 21, 2021).

17. Cresswell K, Morrison Z, Crowe S, Robertson A, Sheikh A. Anything but engaged: user involvement in the context of a national electronic health record implementation. J Innov Health Inform. (2011) 19:191–206. doi: 10.14236/jhi.v19i4.814

PubMed Abstract | CrossRef Full Text | Google Scholar

18. DeRiel E, Puttkammer N, Hyppolite N, Diallo J, Wagner S, Honoré J, et al. Success factors for implementing and sustaining a mature electronic medical record in a low-resource setting: a case study of iSanté in Haiti. Health Policy Plan. (2017) 33:237–46. doi: 10.1093/heapol/czx171

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Moomba K, Williams A, Savory T, Lumpa M, Chilembo P, Tweya H, et al. Effects of real-time electronic data entry on HIV programme data quality in Lusaka, Zambia. Public Health Action. (2020) 10:47–56. doi: 10.5588/pha.19.0068

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Sung S, Hørthe H, Svendsen ØV, van Duinen AJ, Salvesen Ø, Vandi A, et al. Early evaluation of the transition from an analog to an electronic surgical logbook system in Sierra Leone. BMC Med Educ. (2021) 21:578. doi: 10.1186/s12909-021-03012-z

PubMed Abstract | CrossRef Full Text | Google Scholar