Incorporation of Edutainment Into Intervention and Evaluation: The Jump With Jill (JWJ) Program

Objective: To incorporate media strategies (e.g., edutainment) into a pilot nutrition education intervention, Jump with Jill (JWJ), to (1) change student knowledge and intentions; (2) stimulate enthusiasm; and (3) enhance evaluation.

Methods: Third graders (n = 194) completed a pre-survey, attended a 60-min nutrition education show presented as a school assembly, and then completed a post-survey. Data was collected interactively, where students lined up behind an emoji that best represented their answers.

Results: Statistically significant improvements were reported for drink preferences and enjoyment of nutrition education as well as the experience of taking a survey. All aggregate responses for knowledge, attitudes and intention became significantly more positive (p = 0.05). Furthermore, ~95% reported positive ratings to participating in the survey (P < 0.0001).

Discussion: Use of edutainment may serve to stimulate change in nutrition knowledge and intention in 3rd grade students. Perfecting this teaching strategy into evaluations of health promotion programming may serve the field by increasing accuracy of student responses when language, reading level, or survey inexperience are barriers.

Introduction

Through the use of entertaining strategies, mass media has been successful in impacting food intake in children (1, 2). Marketing techniques such as repetition, music, characters and excitement are well-thought-out, well-funded and incorporate theoretical frameworks from the fields of behavior change, education and social marketing (3). Use of entertainment as an educational tool, termed edutainment (4), has established guidelines (5). In brief, research-based messages are woven into the audiences' lived experiences (6) using compelling plots, songs, and characters. Storytelling, identification and attachment to strong characters, parasocial interaction, social modeling, competition and liking are incorporated to enhance attitudinal and behavioral change (3, 4). To be effective, edutainment messages should be sustained as part of a comprehensive campaign that connects learners to characters and stories showing behavior change while establishing a new social norm. Because health skills develop through practice, the positive experiences derived from learning through play help develop deep intrinsic motivations that positively influence health outcomes (7) and lead to high levels of participant program satisfaction (6). Edutainment has been a successful component of previous nutrition interventions, improving knowledge, attitude and intention (8–10) and developing resistance to persuasive messages (11).

Although amenable to change, habits and food preferences formed during childhood typically track into adulthood and may lead to development of chronic disease (12–14). Hence, it is important to utilize strategies directed at elementary-aged children that influence targeted food and beverage choices known to impact health, such as increasing intake of fruits, vegetables, and low-fat dairy foods as well as decreasing intake of sugar-sweetened beverages (15). However, successful nutrition education aimed at children cannot simply be didactic but should be interactive, enjoyable and developmentally appropriate (7, 16, 17). Moreover, programming should have a theoretical framework (7, 16, 17) such as those used in behavior change, education and social marketing (18). Finally, unique evaluation processes must be considered because children are mastering written language and may be inexperienced with surveys. Edutainment strategies integrated into evaluation protocol might be a tool to addresses this challenge.

Although schools are an obvious venue for educating children about healthy eating (16, 19, 20), many students receive inadequate nutrition education despite its success (21). While teachers consider nutrition education valuable for students (22), Common Core State Standards and high-stakes standardized testing make nutrition and physical education expendable (23). Limited resources (22) and lack of subject expertise make external nutrition education programs attractive (16). Employing media strategies can entice classroom teachers into including nutrition education, thereby overriding cost and time barriers (11).

Furthermore, reliable evaluation strategies are essential parts of any intervention. Assessment strategies for children are generally informal (using some type of written format) and focus on measuring changes in knowledge, attitude, and behavior. Such changes are especially difficult to assess in young children given their cognition levels (24). Moreover, with required standardized testing in elementary schools, questions have arisen regarding student stress in relation to any testing, (23, 25) generating interest in investigating alternative evaluation strategies.

Employing media strategies, Jump with Jill (JWJ) is a music-based, nutrition education program. JWJ creates a unique learning experience using original music, lighting, props, and live characters to inspire both children and teachers (Figure 1). During the 60-min school assembly, students dance and sing to behaviorally-focus songs that address increasing consumption of fruits, vegetables, dairy and water. Following the show, these characters and their messages can be used by classroom teachers via supplemental materials (e.g., music downloads, activity sheets, danceable music videos).

FIGURE 1

Figure 1. The music-based edutainment nutrition education program, JUMP WITH JILL (JWJ), engaging students and teachers in school-wide assembly.

This study was undertaken to determine the impact of incorporating edutainment into both a nutrition education campaign and its evaluation. The hypotheses were such incorporation would (1) change student knowledge and intentions; (2) stimulate enthusiasm toward nutrition education; and (3) enhance the evaluation process.

Methods

Recruitment

An independent booking agency recruited schools for the 2015-2016 school year to receive JWJ at no cost. Eligible schools (n = 50) were within the 10 New Jersey counties that ranked lowest in the 2014 County Health Rankings (Atlantic, Camden, Cape May, Cumberland, Essex, Gloucester, Hudson, Passaic, Salem, and Union). One performance school per county was randomly chosen for evaluation. While the entire school attended the assembly, only one randomly-selected 3rd grade class completed surveys. Parental, student and teacher consent were handled at the district level. As all schools viewed JWJ as an adjunct to their education, the project was considered exempt from written consent. The University of Pittsburgh's Institutional Review Board approved this study as only deidentified evaluation data was used for the evaluation. No demographic information was collected.

Survey Development

A survey was designed incorporating edutainment strategies to assess student knowledge, attitudes and intentions (e.g., questions repeated phrasing used in the show). For the first question, actual products (milk, water, energy drinks, fruit drinks, and soda) were displayed. Milk and Water were classified as a positive response (coded as 1) and Energy Drink/Fruit Drink/Soda (coded as 0) were classified as a negative response.

For questions two through seven, instead of adapting more traditional “yummy/yucky faces” (26) or indication of “yes” or “no,” customized emojis were used to determine responses (Figure 2). To overcome obstacles associated with using a Likert Scale with children (27), emoji faces varied in relationship to the strength of response and were accompanied by words rather than numbers. In the JWJ Likert Scale Survey Technique for Children, responses were recorded such that each emoji represented a point on a 1 to 5 Likert scale with 1 reflecting “ABSOLUTELY NOT” and 5 reflecting “YES!.” In addition, at pre-test students were asked to raise their hands to determine previous experience with survey taking; at post-test, they were asked to stand in front of the emoji that reflected their survey enjoyment.

FIGURE 2

Figure 2. Integration of edutainment strategies into the evaluation of JUMP WITH JILL (JWJ), a nutrition education program.

Prior to administration, cognitive testing with a small, convenience sample (n = 5; grades K-4) was conducted to determine how the expressions would be interpreted. Except for “No,” students indicated agreement with each face label. Originally a face with a tear depicted “No” but students thought that face represented “Sad.” The emoji was changed to a face with a line for a mouth.

Survey Administration

Stations containing either a beverage (energy drink, fruit drink, soda, milk or water) (Question 1) or an emoji (Questions 2–7) were arranged. Before starting the survey, students were told by JWJ characters that there was no right answer and that the survey was not a test. To capitalize on the power of peer influence, students made a “Promise to be Honest” as a commitment to being thoughtful “research subjects” where their authentic opinions were valued. To minimize groupthink, students were asked to select their answers before moving to a station. Students then were asked to fully commit to their answer by lining up in front of the emoji that matched their response (Figure 3). The JWJ cast recorded the number of children at each station for each question. The students then joined the assembly. After the show, students reaffirmed their honesty pledge and repeated the pre-survey process.

FIGURE 3

Figure 3. Third-grade students participating in the JUMP WITH JILL (JWJ) Likert Scale Survey Technique for Children. Written consent was obtained from the parents for all identifiable minors for the publication of this image.

Data Analysis

Frequencies were calculated from recorded responses. Two types of outcomes were determined using aggregated data from all schools and time points (pre-test, post-test): the proportion of responses (children lining up behind an emoji) and an overall score. The proportion of responses was calculated as: sum of children lining up behind an emoji divided by total of children multiplied by 100. Pairwise differences among dependent proportions were tested using simultaneous 95% confidence intervals among dependent proportions (assuming a multinomial distribution) (28, 29). The overall score was calculated (for all but question 1) by multiplying the emoji's response choice (1–5) by the number of children lining up behind it, then summing all multiples for each time point and school. Higher overall scores indicated more positive responses. Differences in scores between pre- and post-tests were assessed. Distribution of overall scores and differences were determined to be non-normal; hence, the Wilcoxon Signed-Ranks test was conducted using Statistical Analysis Software (version 9.3, 2011, SAS Institute Inc, Cary, NC).

Results

JWJ performed 50 shows reaching nearly 14,700 elementary school children. Over 75% of the 194 3rd graders participating in the evaluation reported no previous experience with taking a prior survey. Following the show, statistically significant differences in individual scores were noted for questions 1, 3, 4, and 5 (Table 1). Moreover, all aggregate responses became significantly more positive (Table 2). Furthermore, ~85% reported the highest positive rating (YES!) to survey participation, and 95% reported positive ratings (YES! or Yeah), which differed significantly from those reporting negative ratings (ABSOLUTELY NOT and No) (P < 0.0001).

TABLE 1

Table 1. Proportions of individual score values from third grade students (n = 194) at 10 NJ schools after attending the Jump with Jill show (School Year 2015-16)^a.

TABLE 2

Table 2. Mean of overall scores from third grade students (n = 194) at 10 NJ schools before and after attending the Jump with Jill show (School Year 2015-16)^a.

Discussion

Overall, using media strategies associated with edutainment to change student knowledge and intention as well as to detect these changes was confirmed. Use of enthusiastic endorsement of the expected behaviors (a Social Learning Theory component) by JWJ characters (a marketing strategy) appeared to stimulate change in nutrition knowledge and intention in these 3rd grade students. These findings confirm previous observations that theatrical interventions in schools can be successful (8–10, 30, 31).

JWJ creates positive, realistic impressions of healthy foods, moderates negative responses, and creates commitment for those that start out as “not sure.” Jump with Jill deconstructs the framework that kids have built with their dislikes and breaks ground with newfound aspirational attitudes, as seen by the decrease in the number of extreme negative responses for all questions. The positive experience derived from JWJ helps audiences develop deep intrinsic motivations that lead to high levels of program satisfaction among participants. Moreover, through the many touch points of the program, the students and their teachers attach to the characters. Their relationship deepens as they go from interacting with the show to viewing videos to working in small groups with the characters to complete the survey where their opinion is valued and documented.

While the positive results of the intervention are exciting, a major focus of this pilot study was developing an evaluation tool that aligned with the educational approach of the program. Unique to this study was the integration of media strategies into the evaluation process. Edutainment techniques successfully captured changes in 3rd grade students' knowledge and intention. Making the evaluation process interactive and fun (game-like) resulted in over 95% of students enjoying participation in a study, in contrast to reports of student stress in relation to testing (32, 33). A positive experience can have meaningful measurements for likelihood of healthy behaviors. Researchers have noted high intensity emotional experiences has an increased influence on behavior, so strong it can even overwhelm cognitive processing (34). Nutrition education in the form of a rock show is a strategic choice to empower audiences to action. Additionally, children in 3rd grade are in Piaget's concrete operational phase of cognitive development, where they begin to use inductive logic and become less egocentric (35).

Using the “Promise to be Honest” pledge built upon their developing senses of having unique thoughts of fairness (35) and decreased social desirability. Students were proud to be forthright with their opinions even though they were aired in front of their peers. Setting clear expectations for behavior (aka defining the rules of the game) is crucial. It is suggested that only a small, familiar group (e.g., one class) be used for this data collection method as the researchers experienced a diminished effect of accountability as group sizes grew. For example, as this methodology is scaled to other JWJ projects and sample size is increased, we are still evaluating classrooms one at a time rather than combining classes. Adjustments in the show and travel schedule are made to allow for these multiple classroom visits before and after the show as part of the intervention design.

Perfecting edutainment within nutrition education evaluations in elementary schools may serve the field by increasing accuracy of student responses when language, reading level, or survey inexperience are barriers. These study participants saw the evaluation as part of the intervention, hence creating a positive experience throughout appears to enhance satisfaction. Furthermore, this was an intentionally simple evaluation study design that could fit into the parameters of a school day, requiring no additional staffing from the school or JWJ.

Several limitations must be acknowledged. This current study lacks follow-up to demonstrate students' knowledge retention or behavior change. Other limitations include lack of a control group and unpaired pre- and post- survey data, both of which restrict generalizability. Furthermore, statistically significant proportions that change by a fraction of a decimal to a few points must be interpreted with caution.

Conclusions and Implications

The mass media has proven successful at impacting food intake in children (1, 2). If “celebrities” can shape social norms about unhealthy behaviors, then nutrition educators might be able to use these same strategies to shift youth toward healthy behaviors. If children demand high quality entertainment, then health-related media can compete. The media could be the best guide in developing solutions to address youth health. Suggestions for best practices in incorporating this novel evaluation strategy into other programs include: working in small groups, for example, by classroom; pausing to let students decide answers before reporting to the group; decreasing test anxiety and reducing test fatigue by making the evaluation into a game; and decreasing social desirability bias by using a “promise to be honest” pledge.

To provide additional validation for using media strategies in nutrition education, along with the impact of JWJ on food choice behavior, future studies including a comparison group, more follow-up data points, demographic characteristics of children, and other confounding variables are warranted.

Data Availability

The datasets generated for this study are available on request to the corresponding author.

Ethics Statement

This study was carried out in accordance with the recommendations of the school districts involved in the study. Parental, student and teacher consent were handled at the district level. As all schools viewed JWJ as an adjunct to their education, the project was considered exempt from written consent. The University of Pittsburgh's Institutional Review Board approved this analytical portion of the study as only deidentified evaluation data was used. No demographic information was collected. The University of Pittsburgh Institutional Review Board approved this study.

Author Contributions

The author confirms being the sole contributor of this work and has approved it for publication.

Funding

This study was supported by the Young Audiences of New Jersey and Eastern Pennsylvania that received a grant through the Robert Wood Johnson Foundation.

Conflict of Interest Statement

JJ holds all trademarks, copyrights, and likenesses associated with the property, Jump with Jill.

Acknowledgments

The University of Pittsburgh Institutional Review Board approved this study. Statistical analysis of deidentified data was performed by Salma Musaad MD Ph.D at the University of Illinois at Urbana-Champaign. Madeleine Sigman-Grant provided pivotal guidance. I wish to thank all the administration, teachers and students from all the participating schools for their enthusiastic involvement with this study.

References

1. Frazier S, Harris J. Trends in Television Food Advertising to Young People: 2017 Update. UCONN Rudd Center for Food Policy & Obesity (2018). p. 1–8. Available online at: http://uconnruddcenter.org/files/Pdfs/TVAdTrends2018_Final.pdf

2. Boyland E, Nolan S, Bridget K, Tudur-Smith T, Jones A, Halford J, et al. Advertising as a cue to consume: a systematic review and meta-analysis of the effects of acute exposure to unhealthy food and nonalcoholic beverage advertising on intake in children and adults. Am J Clin Nutr. (2016) 103:519–33. doi: 10.3945/ajcn.115.120022

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Evans WD. Social marketing campaigns and children' s media use. Fut Child. (2008) 18:181–203. doi: 10.1353/foc.0.0009

CrossRef Full Text | Google Scholar

4. Moyer-Guse E. Toward a theory of entertainment persuasion: Explaining the persuasive effects of entertainment-education messages. Commun Theory. (2008) 18:407–25. doi: 10.1111/j.1468-2885.2008.00328.x

CrossRef Full Text | Google Scholar

5. Singal A, Rogers EM. The status of enertianment-education worldwide. In: Singal A, Rogers E, and Sabido M editors. Entertainment-Education and Social Change: History, Reserach and Practie. Mahwah, NJ: Lawrence Eribaum Associates (2004). p. 3–20.

6. Neumark-Sztainer D, Haines J, Robinson-O'Brien R, Hannan PJ, Robins M, Morris B, et al. “Ready. Set. ACTION!” A theater-based obesity prevention program for children: A feasibility study. Health Educ Res. (2009) 24:407–20. doi: 10.1093/her/cyn036

CrossRef Full Text | Google Scholar

7. Gao Z, Podlog L, Huang C. Associations among children's situational motivation, physical activity participation, and enjoyment in an active dance video game. J Sport Heal Sci. (2013) 2:122–8. doi: 10.1016/j.jshs.2012.07.001

CrossRef Full Text | Google Scholar

8. Wechsler H, Basch CE, Zybert P, Shea S. Promoting the selection of low-fat milk in elementary school cafeterias in an inner-city Latino community: evaluation of an intervention. Am J Public Health. (1998) 88:427–33. doi: 10.2105/AJPH.88.3.427

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Perry CL, Zauner M, Oakes JM, Taylor G, Bishop DB. Evaluation of a theater production about eating behavior of children. J Sch Health. (2002) 72:256–61. doi: 10.1111/j.1746-1561.2002.tb07339.x

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Cheadle A, Cahill C, Schwartz PM, Edmiston J, Johnson S, Davis L, et al. Engaging youth in learning about healthful eating and active living: an evaluation of educational theater programs. J Nutr Educ Behav. (2012) 44:160–5. doi: 10.1016/j.jneb.2011.06.005

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Sangalang A, Jessie MQJ, Ciancio KE. Exploring audience involvement with an interactive narrative: implications for incorporating transmedia storytelling into entertainment-education campaigns. Crit Arts. (2013) 27:127–46. doi: 10.1080/02560046.2013.766977

CrossRef Full Text | Google Scholar

12. Grimm KA, Kim SA, Yaroch AL, Scanlon KS. Fruit and vegetable intake during infancy and early childhood. Pediatrics. (2014) 134:S63–9. doi: 10.1542/peds.2014-0646K

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Nicklaus S, Boggio V, Chabanet C, Issanchoua S. A prospective study of food preferences in childhood. Food Qual Prefer. (2004) 15:805–18. doi: 10.1016/j.foodqual.2004.02.010

CrossRef Full Text | Google Scholar

14. Lien N, Lytle LA, Kleppa K-I. Stability in consumption of fruit, vegetables, and sugary foods in a cohort from age 14 to age 21. Prev Med. (2001) 33:217–26. doi: 10.1006/pmed.2001.0874

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Nielsen SJ, Rossen LM, Harris DM, Odgen CL. Fruit and vegetable consumption of u.s. Youth, 2009-2010. NCHS Data Brief. (2014)156:1–8.

Google Scholar

16. Hard A, Uno C, Koch PA. The Importance of Nutrition Education in the 2015 Child Nutrition Reauthorization. (2015) Available online at: http://www.tc.columbia.edu/media/media-library-2014/centers/tisch-center/Nutrition-Ed-White-Paper-09.14.pdf

17. Contento IR. Nutrition education: Linking theory, research and practice. Sudbury, MA: Jones and Barlett (2011).

PubMed Abstract

18. Luca NR, Suggs LS. Theory and model use in social marketing health interventions. J Health Commun. (2013) 0730:1–21. doi: 10.1080/10810730.2012.688243

CrossRef Full Text | Google Scholar

19. Franks A, Kelder S, Dino G, Horn K, Gortmaker SL, Wiecha JL, et al. School-based programs lessons learned from CATCH, planet health and not-on-tobacco. Prev Chronic Dis. (2007) 4:1–9. Available online at: http://www.cdc.gov/pcd/issues/2007/apr/06_0105.htm

PubMed Abstract | Google Scholar

20. Robinson LE, Webster EK, Whitt-Glover MC, Ceaser TG, Alhassan S. Effectiveness of pre-school- and school-based interventions to impact weight-related behaviours in African American children and youth: a literature review. Obes Rev. (2014) 15:5–25. doi: 10.1111/obr.12208

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Kann L, Telljohann SK, Wooley SF. Health education: results from the school health policies and programs study 2006. J Sch Health. (2007) 77:408–34. doi: 10.1111/j.1746-1561.2007.00228.x

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Perera T, Frei S. Improving nutrition education in u.s. elementary schools: challenges and opportunities. J Educ Pract. (2015) 6:41–50.

Google Scholar

23. Saeki E, Pendergast L, Segool NK, von der Embse NP. Potential psychosocial and instructional consequences of the common core state standards: implications for research and practice. Contemp Sch Psychol. (2015) 19:89–97. doi: 10.1007/s40688-014-0043-5

CrossRef Full Text | Google Scholar

24. Seattle PI. The 4 Types of Tests Teachers Give (and Why) _ Articles _ Noodle.

25. Gubi AA, Bocanegra JO. Impact of the common core on social-emotional learning initiatives with diverse students. Contemp Sch Psychol. (2015) 19:98–102. doi: 10.1007/s40688-015-0045-y

CrossRef Full Text | Google Scholar

26. Baxter S, Thompson W. Fourth-Grade children's consumption of fruit and vegetable items available as part of school lunches is closely related to preferences. J Nutr Educ Behav. (2002) 34:166–71. doi: 10.1016/S1499-4046(06)60086-9

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Mellor D, Moore KA. The use of likert scales with children. J Pediatr Psychol. (2014) 39:396–79. doi: 10.1093/jpepsy/jst079

PubMed Abstract | CrossRef Full Text | Google Scholar

28. SAS Institute Inc. SAS. Cary, NC (2015).

29. Berry J, Hurtado G. Comparing non-independent proportions. Observations. Tech J SAS Softw Users. (1994) 3:21–7.

30. Powers AR, Struempler BJ, Guarino A, Parmer SM. effects of a nutrition education program on the dietary behavior and nutrition knowledge of second-grade and third-grade students. J Sch Heal. (2005) 75:129–34. doi: 10.1111/j.1746-1561.2005.tb06657.x

PubMed Abstract | CrossRef Full Text | Google Scholar

31. FoodPlay Productions. Proven Success. (2010). Available online at: http://foodplay.com/downloads/pdf/FP-Eval-Feedback.pdf

32. Lobman C. “I Feel Nervous…Very Nervous” Addressing test anxiety in inner city schools through play and performance. Urban Educ. (2014) 49:329–59. doi: 10.1177/0042085913478621

CrossRef Full Text | Google Scholar

33. Nichols S, Berliner D. Collateral damage: How high stakes testing corrupts American schools. Cambridge, MA: Harvard University Press (2007).

Google Scholar

34. Davidson RJ, Scherer KR, Goldsmith HH. Chapter 31: Handbook of Affective Sciences. New York, NY: Oxford University Press, (2003).

Google Scholar

35. Simatwa EMW. Piaget's theory of intellectual development and its implication for instructional management at pre- secondary school level. Educ Res Rev. (2010) 5:366–71. Available online at: http://www.academicjournals.org/ERR2

Google Scholar