Yuewen Sun
Yuan Li1,2
Hueiming Liu
Puhong Zhang- 1Nutrition and Lifestyle Department, The George Institute for Global Health at Peking University Health Science Center, Beijing, China
- 2Faculty of Medicine, University of New South Wales, Sydney, NSW, Australia
- 3Barts and The London School of Medicine and Dentistry, Institute of Preventive Medicine, Queen Mary University of London, London, United Kingdom
- 4Health Systems Science Department, The George Institute for Global Health, Sydney, NSW, Australia
Background: Salt reduction is a cost-effective, and rather challenging public health strategy for controlling chronic diseases. The AppSalt program is a school-based multi-component mobile health (mhealth) salt reduction program designed to tackle the high salt intake in China. This mixed-methods process evaluation was conducted to investigate the implementation of this program across sites, identify factors associated with the implementation, and collect evidence to optimize the intervention design for future scale-up.
Methods: Mixed methods were used sequentially to collect data regarding five process evaluation dimensions: fidelity, dose delivered, dose received, reach, and context. Quantitative data were collected during the intervention process. Participation rate of intervention activities was calculated and compared across cities. The quantitative data was used for the selection of representative intervention participants for the qualitative interviews. Qualitative data were collected in face-to-face semi-structured interviews with purposively selected students (n = 33), adult family members (n = 33), teachers (n = 9), heads of schools (n = 9), key informants from local health, and education departments (n = 8). Thematic analysis technique was applied to analyze the interview transcripts using NVivo. The qualitative data were triangulated with the quantitative data during the interpretation phase.
Results: The total number of families recruited for the intervention was 1,124. The overall retention rate of the AppSalt program was 97%. The intervention was implemented to a high level of fidelity against the protocol. About 80% of intervention participants completed all the app-based salt reduction courses, with a significant difference across the three cities (Shijiazhuang: 95%; Luzhou: 73%; Yueyang: 64%). The smartphone app in this program was perceived as a feasible and engaging health education tool by most intervention participants and key stakeholders. Through the interviews with participants and key stakeholders, we identified some barriers to implementing this program at primary schools, including the left-behind children who usually live with their grandparents and have limited access of smartphones; perceived adverse effects of smartphones on children (e.g., eyesight damage); and overlooked health education curriculum at Chinese primary schools.
Conclusion: This process evaluation demonstrated the feasibility and acceptability of using smartphone applications delivered through the education system to engage families in China to reduce excessive salt intake.
Clinical Trial Registration: The AppSalt study was registered at www.chictr.org.cn, identifier: ChiCTR1800017553. The date of registration is August 3, 2018.
Introduction
Salt reduction is one of the best-buy strategies for controlling the burden of non-communicable diseases (NCDs) (1). Many high-income countries have implemented their national salt reduction strategies (2) and achieved a significant reduction of average population salt intake, such as Finland and the United Kingdom (3, 4). However, their experience might not be suitable for the Chinese context because of the significant differences in dietary habits. For instance, most of the salt in the Chinese diet is discretionarily added by the consumers to dishes or soups during cooking, such as salt, soy sauces, and other sauces (5).
Unhealthy dietary habits typically emerge from early childhood and result in long-term health damage in adulthood (6, 7). Therefore, cultivating healthy lifestyles to prevent related diseases is essential during childhood and early adolescence (8). A randomized controlled trial conducted in Northern China demonstrated that a school-based education program (School-EduSalt) could effectively reduce the salt intake of primary school students and their family members (9). This program exemplifies a potential cost-effective salt reduction intervention model through the education system for the Chinese context (10). However, the generalizability of this intervention model is limited because of its reliance on teachers for intervention delivery and the extra workload to schools. Built on the School-EduSalt program, a further study, named AppSalt, was designed to incorporate a smartphone application to deliver salt awareness education in order to alleviate the workload of health education on school teachers (11).
To test the effectiveness of the AppSalt program, the researchers conducted a cluster randomized controlled trial (cRCT) at 3 cities (Shijiazhuang, Luzhou, Yueyang) and recruited 54 primary schools to participate in the trial (11). These 3 cities were located in the north, southwest, and central China, which were selected for their geographical locations and relevant dietary habits differences. Half of the recruited schools at each site were randomized to the intervention group and participated in the AppSalt intervention activities for 1 year. The control group continued with their usual health education curriculum. The usual health education courses for third graded students include the following topics: infectious diseases prevention, healthy lifestyles, sight protection, smoking cessation, etc. The knowledge on salt is not covered in the usual curriculum. The primary outcome for assessing the effectiveness of the trial is measured by the difference between the intervention and control groups in the change of salt intake measured by 24-h urinary sodium from baseline to the end of the trial (11). The effectiveness evaluation found that the salt intake in intervened adults decreased by 0.82 g/day (12).
Process evaluation is recommended as an integral part of complex interventions by the UK Medical Research Council (13). With the development of modern technology, mobile phones and apps have become a popular platform for public health interventions (14). However, there is a paucity of studies investigating the implementation process of these mobile health(mHealth) interventions, participants' satisfaction with these apps, or factors affecting the implementation. Additionally, most of these mhealth programs at schools were conducted among middle/high school students (15). Fewer programs have used apps for health education among primary school students, who require parents' assistance of using phones. Therefore, the intervention mechanism of the AppSalt program is different from other mHealth interventions.
We designed and conducted a mixed-methods process evaluation alongside the cRCT to investigate the intervention process of the AppSalt program in a real-world setting. Besides, key stakeholders' opinions on scaling up the program were collected through semi-structured interviews.
Methods
Intervention Design and Implementation
AppSalt program is a multi-component salt reduction platform used to deliver salt reduction knowledge and skills to primary school students and their families (11). The intervention strategies were developed following the principles of Health Belief model and Socioecological model (16, 17). In this program, a smartphone app named “AppSalt” was designed for delivering standardized health education courses and tasks to third-grade students (8–9 years old) and their parents. These courses and tasks are the core intervention strategies at the individual level. Besides, offline intervention strategies were developed to enable interpersonal and community impacts for salt reduction, including parent group meetings, class competitions, and school environment cultivation. All students in the selected intervention school, who have access to smartphones with their (grand)parents' assistance and are willing to take part, are considered eligible to participate in the AppSalt program. The details of the AppSalt program intervention components are provided in Table 1. These intervention activities should be completed in two terms (~10 months). A more detailed description of the AppSalt program and its logic model are reported in previous publications (11, 18).
Table 1. AppSalt program intervention components.
The implementation of the AppSalt program in the trial was facilitated by a joint partnership of multiple stakeholders, including the intervention schools, the local CDC office, local education authority, and the central program office. In this AppSalt program, the parents and students were instructed by teachers to install the App and use it for salt reduction courses and salt intake monitoring. During the implementation, the staff of the intervention schools, especially the teachers, were responsible for checking children's homework, communicating with students and parents, organizing offline activities, and making school-level motivation schemes for students. Local CDC office and education departments were responsible for providing professional support to schools and facilitating the implementation of offline intervention activities. The central program office provided technical assistance and support to partners, managed the delivery of online health education sessions, and monitored the implementation progress.
When the intervention group received AppSalt program, the control group continued their usual health education curriculum. The usual health education courses for third graded students include the following topics: infectious diseases prevention, healthy lifestyles, sight protection, smoking cessation, etc. The knowledge on salt was not covered in the usual curriculum.
Study Design and Participants
This study was informed by the process evaluation framework proposed by Steckler et al. (19). A mixed-methods approach was used to systematically evaluate the implementation process of the AppSalt program, including fidelity, reach, dose delivered, dose received, and context. We only evaluated the intervention group in the trial. The control group was not evaluated as they continued with their usual health education and did not receive any extra intervention activities or materials from the program. The detailed demograhic characteristics of the participants in the AppSalt program could be found in our previous publication (12).
Data Collection
Quantitative Data Collection
The quantitative data, including app logs, activity logs, and routine monitoring data, were collected alongside the intervention process. App usage log was generated when the users logged onto the App and used it for health education and other functions. Activity logs were kept by the teachers, which recorded the details of offline intervention activities, including the date, number of participants, and description of the activities. Routine monitoring data was collected by a program coordinator every week in the intervention process to evaluate the implementation of all intervention strategies. Preliminary analysis of routine monitoring data was conducted to identify implementation deviations and optimize the intervention process.
Qualitative Data Collection
The qualitative data was collected using one-on-one semi-structured interviews from October 2019 to December 2019, i.e., within 1 month after completing the 1-year intervention of the AppSalt program. In total, 32 students, 32 family members, 9 teachers, 9 heads of schools, and 8 representatives from local health and education departments were interviewed by trained researchers.
Maximum variation sampling was applied for selecting participants for the interviews (20). At each study site, we purposively selected three intervention schools to represent different implementation performance levels. From each selected school, the trained interviewers conveniently selected 3 or 4 students and their adult family members to participate in the interviews regarding their experience of participating in the AppSalt program. In addition, the interviewers also invited the teachers, and school head-teachers for the interviews to ask their experience of organizing and implementing this program. At each study site, we also invited representatives from local health and education authorities for interviews. More details are published in the protocol of this study (21).
The interviews with selected students, family members, teachers, and school principals were performed one-on-one in the classroom or office at intervention schools; interviews with local health and education authorities were performed at their office. Each interviewee was offered a gift as a token of appreciation for participating in the interviews.
Data Analysis and Interpretation
Quantitative Data Analysis
The participation rate of online health education courses and offline activities were analyzed to represent participants' adherence to the intervention program. The participation rate was compared across sites. Descriptive statistics (comparisons of means, medians, or percentage as appropriate) was used to assess the implementation rates of the five intervention components of the AppSalt program.
Qualitative Data Analysis
Deductive analysis of semi-structured interviews was performed using thematic analysis and managed using QSR Nvivo 12 (22). The lead author (YS), a public health researcher responsible for routine monitoring of the AppSalt program, developed the preliminary coding scheme based on the semi-structured interview guides and the process evaluation framework adopted in this study. Another public health researcher (JS) independently used the preliminary coding scheme and coded 5 interview transcripts to examine the feasibility and accuracy of this coding scheme. Discrepancies emerged during this phase were discussed. We refined the coding scheme through consensus-building. Once the coding scheme was agreed by both authors, YS continued coding all the interview transcripts. The coding scheme could be found in Additional File 1. After completing the coding, researchers summarized the patterns of the codes and identified recurrent themes from these interviews. The transcripts are in Chinese, and the selected quotations were translated into English for use in this manuscript.
Integration
The qualitative data is triangulated with the quantitative data during the interpretation phase to explain the 5 process evaluation dimensions. The quantitative data evaluate how the implementation of the AppSalt program is achieved in different settings. The qualitative data provide complementary information to the 5 evaluation components, which allow researchers to investigate the in-depth reasons behind the varied implementation performance (23). As the fidelity and dose delivered are closely related, we combined these two components when interpretating and presenting the results of this study (24).
Results
Reach
The total number of families recruited for the intervention was 1,124 (Shijiazhuang: 467; Luzhou: 324; Yueyang: 333). The number of students recruited from each intervention class varied from 11 to 57, with a median of 46. This program's reach was highest in Shijiazhuang, where 96% of eligible students participated in this program. The other two sites both recruited about 72% of eligible students for participation.
In the interviews with participants, we found different initial motivations for joining this program. 19 out of 32 parents said they volunteered to participate in the program as they wanted to learn some knowledge on salt and its health impacts. About 40% of the parents said they were selected by the teachers and agreed to participate in helping the teachers complete their tasks.
Although some participants were probably not self-motivated when first joining the program, most of them completed their 1-year participation in the AppSalt program, which helped achieve a high retention rate. In total, 34 students dropped out of the program during the intervention (Shijiazhuang 7; Luzhou 13; Yueyang 14), resulting in a retention rate of 97%. These dropping-outs were mostly due to inevitable reasons, including transferring to another school or inaccessibility of smartphones happened during the intervention process.
Fidelity and Dose Delivered
Salt reduction knowledge and skills were delivered to the participants through multiple channels in the AppSalt program, including online health education courses and tasks, 7-day salt intake monitoring, offline salt reduction activities, communication leaflets, and brochures.
App-Based Intervention Strategies
During the 1-year intervention period, online salt reduction courses and tasks were posted on the App following the intervention schedule. Nine online salt reduction courses, six online practical sessions, and four 7-day salt intake monitoring tasks were delivered through the App as scheduled. Each online session was open for the users for 2 weeks, from the day it was posted.
Adaptation Made During the Intervention
The central program office performed weekly supervision of app usage and implementation status to ensure timely monitoring of program implementation status and identifying potential implementation deviation. After about 4-months of implementation, the researchers noticed that many users answered the after-class quizzes without watching the health education videos. This issue would reduce the amount of knowledge received by the participants. To improve the dose of intervention received, the researchers adapted the App, which made watching 80% of online videos a mandate pre-requisite for answering the quizzes.
Offline Intervention Strategies
The fidelity of implementing offline intervention activities varied across schools in the three study sites, especially for the parent group meetings. Four class-level activities were designed in the protocol and implemented in all intervention schools by teachers during the intervention. The fidelity of implementing offline activities was higher in Shijiazhuang than that at Luzhou and Yueyang.
Four parent group meetings were required in the protocol and should be delivered at the beginning and the end of both terms. Of the 27 intervention schools, 12 schools completed four or more group meetings, eight schools did three group meetings, and the rest seven schools did only two group meetings (Figure 1). All intervention schools in Yueyang did not manage to deliver the required number of parent group meetings.
Figure 1. Number of parent group meetings held at intervention schools.
Factors Affecting the Implementation of Offline Intervention Activities
Teachers and school staff were the focal implementers of this program and should implement the offline activities with support from local CDC staff. Their experiences and perceptions of the salt reduction intervention would cause a remarkable difference in the intervention dose delivered to the participants, especially for those offline intervention activities. Almost all teachers recognized the importance of reducing salt intake and said they learned some salt reduction knowledge from this program.
One teacher said
“In the beginning, I considered this program as a task from the city education department. Then I gradually realized the importance and benefits of salt reduction. Now, I would not eat any high-salt food, and even do not want to look at these foods.” (Teacher A, School A, Luzhou)
However, many teachers said that this program created extra time demands on their already-tight schedule, which made them feel burned out. Therefore, they could not fully implement this program as required, especially the parent group meetings. Some teachers mentioned that the biggest challenge for implementing this program was communicating with students' parents and asking them to complete the tasks on time, because most parents thought this program was extra work on their schedule.
“ I have to use my spare time to work for this project, including communicating with students' parents and making plans for motivating students, which was challenging for me.” (Teacher B, School B, Luzhou)
“ When parents approached me with a question on this program, I had to ask the CDC staff because I was not trained in that field. Luckily, the CDC staff provided much support. Then I added him to our class WeChat group so that the parents could ask him questions directly.” (Teacher A, School A, Yueyang)
“Communicating with students' parents was the greatest difficulty for me when delivering the program. First, this is a long-term program. Second, this program does not have much to do with my teaching. I am more used to teaching students rather than pushing parents.” (Teacher C, School C, Luzhou)
Dose Received
The Dose Received of Online Intervention Activities
Participation in online intervention strategies was high at three study sites. About 80% of participants completed all nine sessions of online salt reduction courses; about 75% of participants completed four or more times of 7-day salt reduction monitoring during the intervention process. As shown in Table 2, the online intervention strategies achieved a significantly higher participation rate in Shijiazhuang than in Luzhou or Yueyang.
Table 2. Participation in intervention activities at three study sites.
The Dose Received of Offline Intervention Activities
The participation rate of offline activities varied across three sites and was highest in Shijiazhuang. The knowledge competitions were the most popular activities, while the art competition was the least popular among the participants (Table 2).
Satisfying Experiences of Using the App Among Participants
The participant's experience of using the App was generally satisfying, especially for those younger parents. Most parents said that the App was easy and convenient to use. In the interviews, several parents mentioned the difficulties encountered in using the App. For example, the volume of videos was low for the first several sessions; the App's installation was not easy compared with installing other apps. Most of these problems were resolved quickly by the information technology (IT) team.
Suggestions for Improving the Contents and Design of Intervention Materials
The videos, posters, leaflets, brochures are the main intervention materials used in the AppSalt program. Suggestions for improving the course videos consistently emerged in the interviews. Three students and 3 teachers said that the videos were a bit dull for watching and could be improved and made more interestingby bringing in cartoon characters.
Regarding the article-based intervention materials (leaflets and brochures), 22% (7/32) of the interviewed adult participants said they did not read these things because they did not have time. In addition, 50% (16/32) of the interviewed adults thought these materials were not helpful for reducing salt intake. As for the posters, 4 out of 9 interviewed teachers said that the posters could not attract much attention from the students and should be designed according to schools' context and needs.
Polarized Feedbacks on the Effectiveness of 7-Day Salt Intake Monitoring
Participants' feedback on 7-day salt intake monitoring was polarized. Most adult participants thought it was a useful tool to monitor the family's salt reduction progress and make them aware of the gap between their current salt intake level and the recommended amount of salt intake.
“This (7-day salt intake monitoring) is a handy tool, which could give you a subjective amount of your salt intake. When I saw the result of the first 7-day salt intake monitoring, I realized that my family members and I ate too much salt, and we have to reduce it.” (Parent A, School A, Shijiazhuang)
Some participants said that 7-day monitoring took too much time and effort to complete. And some participants were concerned about the accuracy of the monitoring results.
“This task cost too much time and energy. First, we needed to weigh the take-out food and record its weight whenever we eat it. Second, it took seven days to complete. Sometimes, I might forget to fill it out and need to restart it from the beginning.” (Parent A, School B, Luzhou)
“I doubt the accuracy of its results. Because in the process, I was not sure if I put the right information in the right place. If the information was not accurate, I could not be sure of the accuracy of its results.” (Parent C, School C, Yueyang)
Knowledge Sharing Within the Intervention Families
Knowledge sharing on salt reduction within the intervention families was bidirectional, either from children to parents/grandparents or from parents to children. Most parents said that they would encourage the students to share the salt reduction knowledge with their grandparents. Some parents said that they would personally watch the online courses first and then tell their children about salt knowledge in the online courses.
“My mother (student's grandmother) prepares meals for the whole family, and the meals were salty before we took part in this program. After we joined this program, I started to tell them the salt reduction knowledge. And we are gradually reducing salt intake this year.” (Parent B, School B Luzhou)
Context
For identifying enabling and barriers factors for implementing this program, we qualitatively interviewed selected participants and key informants. From these interviews, many key stakeholders mentioned that some public health programs relevant to salt reduction were being implemented regionally, which could raise public health awareness and therefore promote the implementation of the AppSalt program at schools.
Although the raised awareness of salt intake might be beneficial for implementing the AppSalt program in primary schools, we also identified some barriers for implementation and scale-up in the interviews with selected participants and key stakeholders. The most noted barriers are as follows.
Perceived Adverse Effects of Using Mobile Phones on Students
Many interviewees, including 4 parents, 5 teachers, 3 school principals, and 2 representatives from local authorities, mentioned their concerns about letting children use smartphones because of the potential damage to their eyesight. Moreover, several interviewed parents said that they would limit their child's screen time per day and would not give them smartphones if it is not necessary. In addition, some interviewees were concerned that using mobile phones would make the students exposed to the temptations of games and other things. One school principal said in the interview that:
“App has its advantages for adult users, such as convenience and flexibility. Every adult has his/her own phone and are willing to use it for self learning. However, the students usually do not have a phone for themselves. If you let them(students) to use the phone for education, they might use it for something else, such as games. Therefore, the teachers and parents can not relax at all.” (School head, School B, Yueyang)
Left-Behind Children
In the interviews with teachers and school headteachers in Luzhou and Yueyang, left-behind children were widely recognized as a barrier to implementing the AppSalt program in these two sites. The left-behind students usually live with their grandparents, as their parents left the hometown for job opportunities in big cities. The accessibility to smartphones might limit the availability of the intervention content among these left-behind children. A school headteacher of a primary school in Luzhou said in the interview:
“More than 10 students live with their grandparents. Some of these grandparents do not have a smartphone. Therefore, our participation rate is relatively low compared with other schools.” (School head, School B, Luzhou)
Current Health Education Curriculum at Primary Schools
From the interviews with teachers and school headteachers, we perceived the difficulties of health education at primary schools. Most school headteachers said that they did arrange health education courses once every 2 weeks according to education authorities” requirements. However, schools usually do not have trained teachers to deliver health education courses. Therefore, these health education classes were generally delivered by the Physical Exercise (PE) teachers or the school doctors. In addition, there is no standard textbooks and evaluation of the health education delivered at primary schools.
Stakeholders' Suggestions for Scaling up the Program
To scale up the AppSalt program in broader settings, we asked the interviewees their opinions for refining the program design for scaling up. The followings are the suggestions collected.
To Empower the Teachers for Delivering Health Education
In the interviews, some teachers expressed their worries for delivering health education courses to students and their parents as they lacked the expertise and professional training. During the implementation process, the teachers usually relied on the local CDC staff's support to teach the salt reduction knowledge and skills at parents group meetings. For scaling up the program, more professional training are required to empower teachers for delivering health education courses. A representative from the local education department raised her concerns for empowering teachers.
“Health education requires professional training. For example, although salt reduction is not a very difficult topic for our teachers, they still need to prepare for this course in advance. Where should they get the knowledge? Who should provide the textbook? Who should provide the training to teachers?” (Informant A, Education department, Shijiazhuang)
To Design More Effective Motivation Schemes for Teachers and Students/Parents
In the interviews with parents and teachers, some interviewees said that this program caused many extra time demands on families and teachers. To better motivate the students and parents for participation, some teachers designed their student motivation scheme. However, as for teachers themselves, there was no effective motivation scheme to implement this program. In the interviews, the teachers were asked what kind of motivation would be effective for them to implement this program. Most teachers said that the recognition from the local education department, such as certificates or awards, would be more valuable for them compared with monetary motivations.
To Incorporate Salt Reduction Contents Into the Regular Health Education Curriculum
Most teachers and heads of schools agreed that it might be easier for them to deliver the salt reduction courses if the content of salt reduction could be incorporated into the health education curriculum of primary schools. Almost all teachers, school heads, and representatives from local education departments agreed on the feasibility and potential beneficial effects of delivering salt reduction knowledge through a regular health education curriculum.
Discussion
Summary of Major Findings
Salt reduction is an urgent and challenging public health issue for China (25, 26). The AppSalt program proposed an app-based multi-component intervention program through the education system aiming to engage primary school students and their family members to reduce salt intake (11). This article systematically evaluated the implementation process of the AppSalt program in 27 primary schools to enlighten the “blackbox” between design and outcome (27). A summary of key quantitative and qualitative findings is provided in Table 3. The mixed data indicates that all online intervention activities, including health education courses and salt intake monitoring, were delivered to high fidelity against the protocol and achieved high participation rate. This study demonstrated that the smartphone app is widely accessible among the participants and could be a scalable tool for implementing health education through primary schools. Although the variation of implementing offline intervention activities was observed across the intervention sites, the overall dose delivered and participation rate reached an acceptable level at most intervention schools.
Table 3. Quantitative and qualitative findings for explaining the process evaluation dimensions.
Although the implementation of the AppSalt program was generally acceptable at all three sites, higher participation rate in intervention activities and better participants' experience are observed in Shijiazhuang than the other two sites. There might be several possible socio-economic explanations for this phenomenon.
First, the intervention families in Shijiazhuang is generally better off than those in the other two sites, given the 12-month household income collected in the baseline survey (Additional File 2). Public health awareness is closely related to participants' socio-economic background, especially for the low- and middle-income countries (28). Therefore, our participants in Shijiazhuang were possibly more aware of salt's adverse effects and thus more interested in public health programs. Second, people of higher income and higher education are usually more used to using smartphones (29, 30). Therefore, an mhealth program using a smartphone app as the primary intervention tool tends to be better implemented in higher-income regions. Finally, the issue of left-behind children is more prevalent in Luzhou and Yueyang than in Shijiazhuang. The topic of left-behind children often emerged in interviews with key stakeholders in these two regions. The left-behind children usually live with their grandparents and do not have access to smartphones (31). Although the schools in Luzhou and Yueyang achieved a relatively high participation rate in intervention activities, the researchers should not overlook the issue of left-behind children when scaling up the AppSalt program.
Comparison With Existing Studies
The rapid development of mobile technology brings tremendous opportunities for public health interventions, including smoking cessation, chronic disease self-management, weight control (32–35). mHealth apps are also considered promising tools for delivering health education among students in school settings. A systematic review finds that school-based mHealth interventions effectively reduce multiple risk factors, including unhealthy diet, physical inactivity, smoking, and alcohol use (18). Most of these programs are conducted among middle or high school students who have their own mobile phones. However, unlike other mhealth school-based interventions among adolescents, this AppSalt program relies on parental participation for achieving the intended outcome, as the third-grade primary school students were too young to use smartphones by themselves. Therefore, the implementers, especially the teachers, need to effectively motivate not only the students, but also their parents to participate in this program.
The equity issue of using the smartphone app as a primary intervention tool is also a concern that might limit the generalizability of mHealth programs (36, 37). The left-behind children usually live with their grandparents and suffer from multiple neglects compared with their peers (38). This study also indicates that the left-behind children do not have adequate access and support for using smartphone apps for online health education. The national monitoring data of migrant workers indicated that 30.1% of migrant workers left their children at home (38). This issue of left-behind children should be taken into consideration when promoting this program on a broader scale.
Implications for Future Scaling up
This mixed-methods process evaluation systematically examined the implementation of the AppSalt program at the intervention schools, whose findings will be used to speed the translation of the AppSalt intervention to a broader scale-up (39).
Primary school health education is an inter-disciplinary task, which calls for close cooperation of schools, local health departments, and local education departments. In the AppSalt program, the rapport established between the AppSalt implementers, the teachers, and the participants is a crucial factor that facilitated the delivery of the program. For broader scaling-up, the local CDC office might not have the capacity and enough personnel to support every individual school for implementing this program. Therefore, more empowerment training should be provided for teachers and enable them to deliver health education knowledge to students and parents. Besides, standardized course materials, including textbooks, slides, and videos, should be developed and provided to schools to support the delivery of health education courses. What's more, we suggest national health education authorities to incorporate salt reduction knowledge into usual health education curriculum, which would benefit the scaling up of this program and reduce the extra burden on headteachers to deliver relevant knowledge.
Although almost all intervention schools in the AppSalt program managed to achieve an acceptable or higher participation rate, the implementation variation should be considered when promoting this program to other regions. This process evaluation indicates that the AppSalt program is generally better implemented in higher-income regions. This finding is consistent with a nutrition education program conducted at Chinese primary schools (40). For the future scale-up to other provinces in China, the researchers need to accommodate more flexibility in the intervention design for various socio-economic settings and address the barriers identified through this study. For example, a combination of online and offline salt reduction intervention packages should be designed, which would allow schools to choose suitable intervention activities according to their local context and therefore improve the scalability of this program.
Strengths and Limitations
This study is a comprehensive process evaluation that involves mixed methods and multiple stakeholders. Both quantitative and qualitative data were used to evaluate the implementation process, and multiple stakeholders were interviewed in this study.
The study also has some limitations. First, only the participants from the intervention group are interviewed. Without evaluating the control group, we might overlook the contamination of the controlled arm during the study. However, we presume that the contamination effect should be minimal as the access to the App is strictly managed by the project office, and only the students in the intervention group were granted the account to use the App. Second, the selection of the informants from local health and education departments was carried out by local research collaborators (local CDC), which could also result in some recruitment bias. To minimize such bias, the researchers communicated in advance with local collaborators to explain the recruitment criteria. Third, although the schools were selected based on the maximum variation sampling methods, the interviewees, including parents and students, were actually self-volunteered to participate in the interviews from each selected school. We should acknowledge the possibility of recruitment bias brought by this convenience recruitment method. Fourth, we did not perform any observations to subjectively evaluate the implementation of this program in real settings due to time and personnel constraints. All monitoring data were collected with the implementer's/teacher's assistance. Therefore, some implementation problems might be neglected or not reported.
Conclusions
School can and should become a primary setting for children to receive health education and build up healthy life habits (41, 42). However, the health education curriculum in Chinese primary schools is usually overlooked and not followed due to the burden of getting higher marks for “more important” lessons.
The AppSalt program demonstrated the feasibility of a new mHealth model for providing health knowledge by using smartphone apps. This multi-component app-based school-based salt reduction program indicated that smartphone applications are feasible tools to provide salt reduction knowledge and skills to primary school students and their family members. Further work is underway to scale up the program to larger populations in diverse settings.
Data Availability Statement
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.
Ethics Statement
The studies involving human participants were reviewed and approved by Peking University Institutional Review Board (No. IRB00001052-19096) and the Queen Mary Ethics of Research Committee (QMERC2020/22). Written informed consent to participate in this study was provided by the participants' legal guardian/next of kin.
Author Contributions
YS and RL designed this process evaluation with substantial support from the program management team and partners, especially YL, PZ, FH, and HL. CG performed qualitative data collection for this study. YS performed the data analysis and interpretation with JS's support. YS drafted the first version of this manuscript. YL, FH, HL, and PZ reviewed and revised the manuscript critically. All authors read and approved the final manuscript for publication.
Funding
The National Institute of Health Research (NIHR) funded Action on Salt China at Queen Mary University of London using Official Development Assistance (ODA) funding (16/136/77). This study is a sub-study of Action on Salt China project.
Conflict of Interest
FH is a member of the Consensus Action on Salt and Health (CASH) group, a non-profit charitable organization, and its international branch World Action on Salt and Health (WASH) and does not receive any financial support from CASH or WASH.
The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher's Note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Acknowledgments
We acknowledge all participants, partners, and collaborators who were involved in the implementation of this program. The study could not be completed with such a high level of fidelity without their support.
Supplementary Material
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpubh.2022.744881/full#supplementary-material
References
1. World Health Organization. Tackling NCDs: 'best buys' and Other Recommended Interventions for the Prevention and Control of Noncommunicable Diseases. Geneva: World Health Organization (2017).
2. Trieu K, Neal B, Hawkes C, Dunford E, Campbell N, Rodriguez-Fernandez R, et al. Salt reduction initiatives around the world – a systematic review of progress towards the global target. PLoS ONE. (2015) 10:e0130247. doi: 10.1371/journal.pone.0130247
3. Hyseni L, Elliot-Green A, Lloyd-Williams F, Kypridemos C, O'Flaherty M, McGill R, et al. Systematic review of dietary salt reduction policies: evidence for an effectiveness hierarchy? PLoS ONE. (2017) 12:e0177535. doi: 10.1371/journal.pone.0177535
4. Laatikainen T, Pietinen P, Valsta L, Sundvall J, Reinivuo H, Tuomilehto, et al. Sodium in the Finnish diet: 20-year trends in urinary sodium excretion among the adult population. Europ J Clin Nutr. (2006) 60:965–70. doi: 10.1038/sj.ejcn.1602406
5. Anderson CAM, Appel LJ, Okuda N, Brown IJ, Chan Q, Zhao L, et al. Dietary sources of sodium in China, Japan, the United Kingdom, and the United States, women and men aged 40 to 59 years: the INTERMAP study. J Am Diet Assoc. (2010) 110:736–45. doi: 10.1016/j.jada.2010.02.007
6. Gonçalves C, Abreu S, Padrão P, Pinho O, Graça P, Breda J, et al. Sodium and potassium urinary excretion and dietary intake: a cross-sectional analysis in adolescents. Food Nutr Res. (2016) 60:29442. doi: 10.3402/fnr.v60.29442
7. Chen C, Denney L, Zheng Y, Vinyes-Pares G, Reidy K, Wang H, et al. Nutrient intakes of infants and toddlers from maternal and child care centres in urban areas of China, based on one 24-hour dietary recall. BMC Nutr. (2015) 1:23. doi: 10.1186/s40795-015-0019-5
8. Abrignani MG, Lucà F, Favilli S, Benvenuto M, Rao CM, Di Fusco SA, et al. Lifestyles and cardiovascular prevention in childhood and adolescence. Pediatr Cardiol. (2019) 40:1113–25. doi: 10.1007/s00246-019-02152-w
9. He FJ, Wu Y, Feng X-X, Ma J, Ma Y, Wang H, et al. School based education programme to reduce salt intake in children and their families (School-EduSalt): cluster randomised controlled trial. BMJ. (2015) 350:h770–h. doi: 10.1136/bmj.h770
10. Li X, Jan S, Yan LL, Hayes A, Chu Y, Wang H, et al. Cost and cost-effectiveness of a school-based education program to reduce salt intake in children and their families in China. PLoS ONE. (2017) 12:e0183033. doi: 10.1371/journal.pone.0183033
11. He FJ, Zhang P, Luo R, Li Y, Chen F, Zhao Y, et al. An application-based programme to reinforce and maintain lower salt intake (AppSalt) in schoolchildren and their families in China. BMJ Open. (2019) 9:e027793. doi: 10.1136/bmjopen-2018-027793
12. He FJ, Zhang P, Luo R, Li Y, Sun Y, Chen F, et al. App based education programme to reduce salt intake (AppSalt) in schoolchildren and their families in China: parallel, cluster randomised controlled trial. BMJ. (2022) 376:e066982. doi: 10.1136/bmj-2021-066982
13. Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ. (2015) 350:h1258. doi: 10.1136/bmj.h1258
14. Dugas M, Gao GG, Agarwal R. Unpacking mHealth interventions: a systematic review of behavior change techniques used in randomized controlled trials assessing mHealth effectiveness. Digit Health. (2020) 6:2055207620905411. doi: 10.1177/2055207620905411
15. Champion KE, Parmenter B, McGowan C, Spring B, Wafford QE, Gardner LA, et al. Effectiveness of school-based eHealth interventions to prevent multiple lifestyle risk behaviours among adolescents: a systematic review and meta-analysis. Lancet Digital Health. (2019) 1:e206–21. doi: 10.1016/S2589-7500(19)30088-3
16. Becker MH. The health belief model and sick role behavior. Health Educ Monographs. (1974) 2:409–19. doi: 10.1177/109019817400200407
17. Bronfenbrenner U, Morris PA. The bioecological model of human development. In: Handbook of Child Psychology. p. 793–828. doi: 10.1002/9780470147658.chpsy0114
18. Sun Y, Luo R, Li Y, He FJ, Tan M, MacGregor GA, et al. App-based salt reduction intervention in school children and their families (AppSalt) in China: protocol for a mixed methods process evaluation. JMIR Res Protoc. (2021) 10:e19430. doi: 10.2196/19430
19. Steckler AB, Linnan L, Israel B. Process Evaluation for Public Health Interventions and Research. San Francisco, CA: Jossey-Bass (2002).
20. Palinkas LA, Horwitz SM, Green CA, Wisdom JP, Duan N, Hoagwood, et al. Purposeful sampling for qualitative data collection and analysis in mixed method implementation research. Adm Policy Ment Health. (2015) 42:533–44. doi: 10.1007/s10488-013-0528-y
21. Sun Y, Luo R, Li Y, He FJ, Tan M, MacGregor GA, et al. App-based salt reduction intervention in school children and their families (AppSalt) in China: protocol for a mixed methods process evaluation. JMIR Res Protoc. (2021) 10:e19430. doi: 10.2196/preprints.19430
22. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. (2006) 3:77–101. doi: 10.1191/1478088706qp063oa
23. O'Cathain A, Murphy E, Nicholl J. Three techniques for integrating data in mixed methods studies. BMJ. (2010) 341:c4587. doi: 10.1136/bmj.c4587
24. Wilson DK, Griffin S, Saunders RP, Kitzman-Ulrich H, Meyers DC, Mansard L, et al. Using process evaluation for program improvement in dose, fidelity and reach: the ACT trial experience. Int J Behav Nutr Phys Act. (2009) 6:79. doi: 10.1186/1479-5868-6-79
25. Xi B, Hao Y, Liu F. Salt reduction strategies in China. Lancet. (2014) 383:1128. doi: 10.1016/S0140-6736(14)60567-5
26. Wang M, Moran AE, Liu J, Coxson PG, Penko J, Goldman L, et al. Projected impact of salt restriction on prevention of cardiovascular disease in China: a modeling study. PLoS ONE. (2016) 11:e0146820. doi: 10.1371/journal.pone.0146820
27. Harachi TW, Abbott RD, Catalano RF, Haggerty KP, Fleming CB. Opening the black box: using process evaluation measures to assess implementation and theory building. Am J Commun Psychol. (1999) 27:711–31. doi: 10.1023/A:1022194005511
28. Palafox B, McKee M, Balabanova D, AlHabib KF, Avezum A Jr, Bahonar A, et al. Wealth and cardiovascular health: a cross-sectional study of wealth-related inequalities in the awareness, treatment and control of hypertension in high-, middle- and low-income countries. Int J Equity Health. (2016) 15:199. doi: 10.1186/s12939-016-0478-6
29. Jeong J, Huang P, Mithas S. The Great Equalizer? Smartphone Use at Workplace and Income Gaps. New York, NY (2018). doi: 10.5465/AMBPP.2018.18810abstract
30. Neter E, Brainin E. eHealth literacy: extending the digital divide to the realm of health information. J Med Internet Res. (2012) 14:e19. doi: 10.2196/jmir.1619
31. Wen M, Lin D. Child development in rural china: children left behind by their migrant parents and children of nonmigrant families. Child Dev. (2012) 83:120–36. doi: 10.1111/j.1467-8624.2011.01698.x
32. Grady A, Yoong S, Sutherland R, Lee H, Nathan N, Wolfenden L. Improving the public health impact of eHealth and mHealth interventions. Aust N Z J Public Health. (2018) 42:118–9. doi: 10.1111/1753-6405.12771
33. WHO. Global Observatory for eHealth. mHealth: New Horizons for Health Through Mobile Technologies: Second Global Survey on eHealth. Geneva: World Health Organization (2011).
34. Rowland SP, Fitzgerald JE, Holme T, Powell J, McGregor A. What is the clinical value of mHealth for patients? npj Digital Med. (2020) 3:4. doi: 10.1038/s41746-019-0206-x
35. Marcolino MS, Oliveira JAQ, D'Agostino M, Ribeiro AL, Alkmim MBM, Novillo-Ortiz D. The impact of mhealth interventions: systematic review of systematic reviews. JMIR Mhealth Uhealth. (2018) 6:e23. doi: 10.2196/mhealth.8873
36. Mahmood A, Kedia S, Wyant DK, Ahn S, Bhuyan SS. Use of mobile health applications for health-promoting behavior among individuals with chronic medical conditions. Digit Health. (2019) 5:2055207619882181. doi: 10.1177/2055207619882181
37. Yang X, Kovarik CL. A systematic review of mobile health interventions in China: Identifying gaps in care. J Telemed Telecare. (2019) 27:3–22. doi: 10.1177/1357633X19856746
38. Wen YJ, Hou WP, Zheng W, Zhao XX, Wang XQ, Bo QJ, et al. The neglect of left-behind children in China: a meta-analysis. Trauma Violence Abuse. (2020) 22:1326–38. doi: 10.1177/1524838020916837
39. Gallo JJ, Lee SY. Mixed Methods in Behavioral Intervention Research. New York, NY: Springer Publishing Company (2015). p. 195-212.
40. Qian L, Newman IM, Yuen LW, Du W, Shell DF. Effects of a comprehensive nutrition education programme to change grade 4 primary-school students' eating behaviours in China. Public Health Nutr. (2019) 22:903–11. doi: 10.1017/S1368980018003713
41. Young I, Leger L, Blanchard C. Health-promoting schools: working in partnership to address global needs, a collaboration leading to the production of practical tools for practitioners. Glob Health Promot. (2013) 20(4 Suppl):122–7. doi: 10.1177/1757975913501005
Keywords: process evaluation, salt reduction, mHealth (mobile Health), primary school, mixed method approach
Citation: Sun Y, Li Y, He FJ, Liu H, Sun J, Luo R, Guo C and Zhang P (2022) Process Evaluation of an Application-Based Salt Reduction Intervention in School Children and Their Families (AppSalt) in China: A Mixed-Methods Study. Front. Public Health 10:744881. doi: 10.3389/fpubh.2022.744881
Received: 24 November 2021; Accepted: 15 February 2022;
Published: 14 March 2022.
Edited by:
Sonu M. M. Bhaskar, Liverpool Hospital & South West Sydney Local Health District (SWSLHD), AustraliaReviewed by:
Mohammad Reza Anani Sarab, Shahid Beheshti University, IranLorena Saavedra-Garcia, University of North Carolina at Chapel Hill, United States
Copyright © 2022 Sun, Li, He, Liu, Sun, Luo, Guo and Zhang. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Puhong Zhang, zpuhong@georgeinstitute.org.cn