Jonathan J. Nguyen
Danielle L. Jessen Condry- ETC Research Group, Department of Microbiological Sciences, North Dakota State University, Fargo, ND, United States
Service-learning (SL) and community-engaged learning (CEL) are high-impact practices whose ideological foundations are built upon ideas pioneered by philosophers such as John Dewey and William James. Given that one methodology (CEL) directly branched from the other (SL), these practices are expected to have fundamental underpinnings that differentially influence how projects within these practices are carried out. Stufflebeam and Shinkfield’s Context, Input, Process, and Product (CIPP) model for evaluation was applied to assess these two high-impact practices. This narrative review has two goals: (1) discuss the usage of the CIPP model to evaluate established SL and CEL projects, and (2) assess any differences in evaluation garnered from CIPP model usage that may have stemmed from nuances in SL and CEL ideology. Literature covering either practice had shown, in some cases, to be inconsistent with how the implementation and guiding principles of such projects matched the terminology used by project organizers. This discrepancy has implications for how these projects are carried out and evaluated in the future.
Introduction
Throughout the years, scholars and academics have demonstrated tenacity in pursuing new ways to augment our classrooms. Some practices are built upon established ideologies and are designed to enrich students’ learning experiences in specific ways. This narrative review will focus on two established pedagogical techniques that seemed to follow this trend: service-learning (SL) and community-engaged learning (CEL). These pedagogies are based on the established ideologies of early philosophers such as John Dewey and William James (Yontz and McCook, 2003; Toncar et al., 2006; Sotelino-Losada et al., 2021). Service-learning is a form of experiential learning that draws upon course material and intertwines it with the act of community service (Sotelino-Losada et al., 2021). Similarly, community-engaged learning, which has its roots in service-learning, has a focus on collaboration and reciprocation between community and academic stakeholders, which lends itself to a more inclusive approach to experiential learning (Saltmarsh et al., 2009; Strasser et al., 2009; Woodley et al., 2019; Agans et al., 2021). Referred to as SL and CEL for brevity, both are considered “high impact practices” (Anderson et al., 2001; Cain, 2013; Malotky et al., 2020) and are similar in principle. SL and CEL have distinct nuances between them that directly influence the manner in which each is carried out.
As with any pedagogical technique, there needs to be a means for evaluating the effectiveness of implementation. How do instructors know that what they did affected the results in the way they think it did? SL and CEL are no exceptions to this rule. Given the inherent differences between the ideologies of SL and CEL, discussed below, it is proposed that the methods to evaluate them will reflect their differences. We propose that the Stufflebeam and Shinkfield’s Context, Input, Process, and Product (CIPP) model can be applied to evaluate SL and CEL contexts (Meurer et al., 2011; Zhang et al., 2011). Although the CIPP model was not designed to evaluate SL and CEL projects, the use of formative and summative assessment elements in the CIPP model lends to continuous improvement and effective implementation of SL and CEL projects.
To explore the nuances of evaluation of SL and CEL, a narrative review was deemed appropriate because narrative reviews feature a broader scope and may describe the state of literature in a more general manner while seeking to reinterpret or highlight connections or disconnections (Ferrari, 2015; Graulich et al., 2021). As such, narrative reviews usually result in less rigorous topic coverage (Ferrari, 2015). Therefore, the scope of this paper will be non-exhaustive and more exploratory by nature, lending itself to future research to expand upon the ideas presented herein.
Foundational thinking of SL and CEL
John Dewey was an American philosopher, educator, and large proponent of social reform, especially within academia (Williams, 2017). He was highly active around the early twentieth century, producing such works as Dewey (1916) and How We Think (1910). He believed that community-building, democracy-building, and learning were not only interdependent upon each other but the same in essence (Cummings, 2000). With this in mind, one of his biggest influences was connecting experience and education in a way that incorporated democratic values into society (Pacho, 2015). He believed that every individual has a role to play in contributing toward a democratic community (Dewey, 1916). Because of all these beliefs, many consider him to have laid much of the foundation for SL and CEL (Giles and Eyler, 1994). Similarly, William James was a psychologist-philosopher whose ideology also helped shape SL and CEL (Deutsch, 1995; Sotelino-Losada et al., 2021). In his piece, “The moral equivalent of war,” James envisioned emphasizing nonmilitary programs geared toward involving the country’s youth in service-related programs (Deutsch, 1995; James, 1995). This would later be reflected in the establishment of organizations such as the Peace Corps and VISTA (Yontz and Mccook, 2003).
Dewey and James, among other influences, are regarded by David A. Kolb as key contributors to the philosophical foundation behind his Experiential Learning Theory (Kolb, 2014). Experiential learning, as learning through lived experiences, is a foundational pillar on which SL and CEL rest (Petkus, 2000; Makani and Rajan, 2016). As the coming sections will demonstrate, SL and CEL both lean heavily into the practice of working with the community to accomplish academic and communal goals. The extent to which the community is involved can vary depending on the implementation and type of pedagogy, but in general, these two methodologies build off of what Dewey and James pioneered (Giles and Eyler, 1994; Meurer et al., 2011; Zhang et al., 2011; Pacho, 2015).
Service-learning: community service with pedagogical spice
SL is a pedagogical technique that has persisted in academia for several decades (Driscoll et al., 1996). This is more evident in the graph below (Figure 1) from the Web of Science.1 Also shown in the graph is the noticeable traction this pedagogy has gained in recent years. Due to the wide variation in how it is practiced, SL does not have a universally recognized definition but rather a collection of overlapping and intertwined meanings (Sotelino-Losada et al., 2021). This review will draw upon the definition of SL that Sotelino-Losada et al. describe. The decision to use the definition by Sotelino-Losada et al. was influenced by their holistic approach to synthesizing the definition. After analyzing several definitions used by different studies, they arrived at a meaning that best encompassed what they discussed. That is, “service-learning is a pedagogical methodology... that requires the explicit connection between curricula or educational plans and the performance of a community service in a single project. Thus, SL participants develop complex cognitive strategies that require questioning what they have learned and their role in the social and environmental framework” (Sotelino-Losada et al., 2021). Common outcomes of SL include opportunities for students to gain real-world application (Cain, 2013; Hwang et al., 2019; Ylitalo and Meyer, 2019), increase in engagement (Peterson et al., 2014; Hwang et al., 2019), improved critical thinking and problem-solving skills (Cain, 2013; Peterson et al., 2014), and professional development (Cain, 2013; Ylitalo and Meyer, 2019). SL’s influence has spread to a vast number of disciplines. This can be seen below in the analysis (Figure 2) visualized from the Web of Science. This breadth of coverage includes disciplines such as teacher education, public health, medicine, and STEM (Table 1; Cashman and Seifer, 2008; Lake and Jones, 2008; Long et al., 2011; Adkins-Jablonsky et al., 2021). Across the globe, countries in Europe, Africa, and South America, to name a few, have embraced its community-focused principles (Johnson et al., 2008; Gaines-Hanks and Grayman, 2009; Sotelino-Losada et al., 2021). This demonstrated prevalence of SL across many disciplines and continents is a testament to its value and will be discussed in the following section.
Figure 1. Web of science visualization of service-learning Publication Trends. The web of science was used to generate a graphical representation of the publications involving service-learning. Web of science, copyright clarivate 2023. All rights reserved.
Figure 2. Discipline coverage of articles involving service-learning. Service-learning has been adopted by many different disciplines, demonstrating its broad applicability. Web of science, copyright clarivate 2023. All rights reserved.
Table 1. Evaluation methods and extent of community involvement in SL.
CEL’s emergence from SL
While the ideological underpinnings of SL have remained relatively consistent over the years, one aspect has seen its fair share of debate: reciprocity. The idea of reciprocity is not necessarily new to SL (Henry and Lynn Breyfogle, 2006); rather, the framing and interpretation of reciprocity within SL have gradually shifted. Earlier in its cycle, SL had maintained an expectation of reciprocity between stakeholders (Henry and Lynn Breyfogle, 2006). This expectation, however, was limited in its depth in that reciprocity, for many projects, simply meant that the parties involved, such as the academic partner and the community partner, were beneficiaries of the partnership (Henry and Lynn Breyfogle, 2006; Petri, 2012). The extent to which each member benefitted was not universally articulated, nor was it necessarily equitable (Henry and Lynn Breyfogle, 2006; Petri, 2012). This potential imbalance likely contributed to the fluctuation in the popularity of SL in the late 1900s (Bailey et al., 2002). As SL regained traction, new criteria for implementation began to receive emphasis. These criteria, such as the importance of the community in being a partner in the needs-defining process, further leaned toward a genuine reciprocal relationship (Mintz and Liu, 1994). Furthermore, authors such as Henry and Breyfogle have called into question the traditional interpretation of reciprocity in SL (Henry and Lynn Breyfogle, 2006). They argue that true, authentic reciprocity is not something that is achieved by simply having each involved party receive something; instead, reciprocity should be viewed with the understanding that, holistically, change that transcends barriers is happening on all fronts (Henry and Lynn Breyfogle, 2006; Petri, 2012). Such change is not necessarily to be viewed as isolated events but instead with the interconnectedness of the greater community in mind (Henry and Lynn Breyfogle, 2006; Petri, 2012). The emergence of community-engaged learning (CEL), described later, coincides with the shift in the framing of reciprocity. Likely, as CEL features principles that more closely align with the newly proposed guidelines, the discussions of reciprocity ultimately cascaded into the new pedagogy that sets itself apart from its predecessor.
Community-engaged learning: collaborate and listen!
As with other pedagogical techniques, SL has evolved over the last decade, with its values further emphasized in one area: the community. As discussed above, a strong desire to prioritize fostering collaboration between the community and academia likely gave rise to CEL. Defined as “a second-generation formulation of SL that emphasizes collaboration between students and community stakeholders around a civic issue,” CEL builds upon the principles of SL, adding focus on the interaction and teamwork that SL can entail (Saltmarsh et al., 2009; Strasser et al., 2009; Woodley et al., 2019; Agans et al., 2021). It is worth noting that the phrase and, by extension, the practice of SL have not been phased out. A steady flow of literature and research still keeps them relevant (Adkins-Jablonsky et al., 2021; Sotelino-Losada et al., 2021). As such, it is perhaps inaccurate to say that SL has evolved into CEL. Rather, a more appropriate way to phrase it might be that SL has branched out into what is now seen as CEL. Given the relatively newer emergence of CEL (when compared to SL), the available pool of literature pertaining to CEL remains dwarfed by that of SL. Separate analyses, also done on the Web of Science and similar to the ones shown earlier for SL, were done for CEL and can be seen below (Figures 3, 4). The narrower timeframe should not take away from the increasing trend in popularity.
Figure 3. Web of science visualization of community-engaged learning. The web of science was used to generate a graphical representation of the increasing popularity of publications involving community-engaged learning. Web of science, copyright clarivate 2023. All rights reserved.
Figure 4. Discipline coverage of articles involving community-engaged learning. Community-engaged learning has been adopted by many different disciplines, demonstrating its broad applicability. Web of science, copyright clarivate 2023. All rights reserved.
At this point, attention should be drawn to differentiating between the phrases “community-engaged learning” and “community engagement.” While “community-engaged learning” ties in specifically with a course setting, “community engagement” can entail broader contexts. For example, Participatory Action Research (PAR) is a research framework in which researchers work with a given population within a community to conduct research alongside community partners to address an issue (Trott et al., 2019). In this instance, PAR can be considered “community engagement” because of the interaction between academic and community partners (Trott et al., 2019). Beyond the classroom, community-engagement programs extend participation to students across programs as well as in clubs (Jones et al., 2020).
Given the close relationship and overlap that SL and CEL have with each other, it should be no surprise that CEL’s common outcomes are reportedly similar to those seen in SL. To reiterate, those outcomes included: (1) real-world application, (2) increased engagement, (3) increased critical thinking and problem-solving skills, and (4) professional development (O’Connor et al., 2011; Hou, 2014; Woodley et al., 2019; Agans et al., 2021). It is worth noting that although there are similarities in outcomes seen in SL and CEL, the extent to which they differ remains to be thoroughly explored in apparent literature. Outcomes aside, the differences and similarities in practicing SL and CEL can be visualized in the Venn diagram (Figure 5). As shown in Figure 4 and Table 2, CEL, similarly to SL, has been seen across multiple disciplines (Figure 4 and Table 2).
Figure 5. Venn diagram comparing service-learning and community-engaged learning. Despite service-learning and community-engaged learning having similar foundations, a few key differences set them apart. Most notably, service-learning places an emphasis on “providing a service” while community-engaged learning emphasizes partnership, cooperation, and equal contribution across all parties. Graphic created with Canva.
Table 2. Evaluation methods and extent of community involvement in CEL.
Evaluation of SL/CEL
As with any pedagogical technique that gains any semblance of traction, evaluation of effectiveness is critical. The following that SL and CEL have garnered has resulted in many different SL and CEL projects being carried out at varying levels of education (Billig, 2011; Malotky et al., 2020; Adkins-Jablonsky et al., 2021). With this level of participation, there is bound to be a spectrum of “effectiveness” that each project falls on. Whether the project is successful or not (and to what extent) relies heavily on how it is implemented within a curriculum. After all, even the greatest pedagogy will crumble under the pressure of poor implementation. With this in mind, the necessity of evaluation and its role in maintaining a certain level of quality is accentuated not only in SL and CEL but in pedagogy as a whole. Upon perusal of the literature, there were no universally accepted means of evaluating these pedagogies with the goal of future improvement (Tables 1, 2). Instead, a multitude of studies saw different means of evaluating different outcomes. For example, researchers like Adkins-Jablonsky et al. gained insight into the benefits these pedagogies bring to the community through thorough interviews with community partners (Adkins-Jablonsky et al., 2021). In contrast, other studies, such as that by Homkes, aim to evaluate the projects’ cognitive outcomes but do not explicitly address other dimensions, such as the impact on the community partner or overall programmatic effectiveness (Homkes, 2008). While these examples used evaluation methods geared more toward the scopes of their respective studies, there are still methods that utilize a more holistic approach to evaluating SL and CEL projects. One worth noting is the Context, Input, Process, and Product (CIPP) model for evaluation (Zhang et al., 2011).
Methods
Literature search strategy
The primary search engine used in this review was Google Scholar. This engine was chosen due to ease of use and overlap in content seen in the databases listed below. When an entire article was inaccessible on Google Scholar, a follow-up search was conducted on the university library’s other databases (i.e., EBSCO, Scopus, and Web of Science). Several key terms were used to explore the topic, including, but not limited to, combinations of the following terms: “service learning,” “community-engaged learning,” “medical education,” “public health,” “preservice teachers,” “STEM”(Science, Technology, Engineering, Math), “framework,” “assess,” “assessment,” “John Dewey,” and “William James.” The initial two search terms were intended to establish the foundations of SL and CEL. After establishing a foundation, several more articles were chosen under each search term; this time, the articles were housed in distinct fields to understand better how SL and CEL were accessed across disciplines. The search terms “assess” and “assessment” covered this review’s direction to analyze both pedagogies. Finally, the latter search terms covered the theoretical underpinnings that heavily influence this pedagogy. No timeframe constraint was imposed on the search to maximize the potential coverage of relevant articles.
Results
Using the CIPP model to evaluate SL/CEL
As previously mentioned, there is no singular, universally accepted method for evaluating SL and CEL projects. Instead, studies may highlight what should be evaluated within such projects. For example, Long et al. describe the necessity for SL projects to be formatively and summatively evaluated (Long et al., 2001). The formative evaluation, happening partway through the project, allows for adjustments to be made and is crucial for the flexibility and adaptability of the project (Long et al., 2001). Meanwhile, summative evaluation, done upon project completion, is essential for assessing how well outcomes matched the expectations set before the project’s implementation (Long et al., 2001). We postulate that the CIPP model is a prime candidate for a universal means of evaluating SL and CEL projects due to its seamless incorporation of formative and summative evaluation. Initially developed by Stufflebeam and Shinkfield, the Context, Input, Process, and Product (CIPP) model for evaluation is “a comprehensive framework for conducting formative and summative evaluations of projects, personnel, products, organizations, and evaluation systems” (Stufflebeam, 2000). Each of the four components within the CIPP model focuses on a different aspect of a project, ranging from planning to monitoring and, ultimately, improvement (Zhang et al., 2011). A visualization of the model (Figure 6) can be found below, but each component will be outlined in the following text as applied to SL and CEL.
Figure 6. Context, input, process, product model for evaluation visualized. The CIPP model for evaluation is an excellent tool for evaluating projects of all kinds. Its comprehensive approach allows for quality monitoring of all stages of a project. Graphic created with Canva.
Starting with the Context component, it is critical to understand the needs of each constituent involved and whether the project adequately meets those needs (Zhang et al., 2011). Within the context of SL and CEL, failure to address context would mean that there may be a disconnect in the objectives of the project organizers, the needs of the community, or the pedagogical needs of the students. For example, this could lead to a poor relationship with the community, thus potentially prompting a cessation of the partnership. The Input component ensures that every constituent can contribute toward a potential approach to the identified problem, whether conceptually in the planning phase or physically during implementation (Zhang et al., 2011). In SL and CEL especially, the community must be able to contribute significantly toward the project. Otherwise, the academic stakeholders risk overstepping control boundaries, and the project fails to break the mold of “community service” (Burns, 1998; Meurer et al., 2011). The Process component is geared toward periodic project assessment and allows for mid-project adjustments to ensure it stays on the correct path (Zhang et al., 2011). This ensures that, should the need ever arise, SL and CEL projects maintain the flexibility to account for most circumstances, expected or otherwise. Finally, the Product component evaluates all of the project’s outcomes, to what extent the project was “successful,” and how insight gained could be used to inform future projects (Zhang et al., 2011). The outcomes might look different depending on which stakeholder is being evaluated. For instance, student outcomes might focus on content knowledge, skill acquisition, or attitudes (Zhang et al., 2011). This component is crucial for the sustainability of a SL and CEL project and allows ample room for improvements for the next iteration.
To reiterate, using the CIPP model to evaluate SL/CEL projects can lead to improved assessed outcomes because the model acts as a quality assurance test at every stage of a project. Applying the model in such a way acts as a guiding principle that allows stakeholders more control over how their project turns out. This means outcomes can be actively worked toward rather than looked at after everything is said and done. Though not originally intended to be specifically a tool to design or evaluate SL and CEL, project organizers of both SL and CEL alike have looked to the CIPP model as a means of thoroughly assessing the outcomes of their projects.
CIPP usage within literature
While some insight into what the CIPP model might look like in the context of SL and CEL was explored earlier, it is necessary to emphasize that the model itself was not explicitly developed to assess SL and CEL projects. Rather, it can apply to projects in general (Zhang et al., 2011). Its comprehensive nature piqued the interest of Zhang et al., who advocated using the CIPP model within the context of SL (Zhang et al., 2011). After an exhaustive breakdown of 26 different evaluation methods, the authors asserted that the CIPP model was the best suited for guiding SL projects (Zhang et al., 2011). In their study, Zhang et al. managed to effectively map each of the four components of the CIPP model to measures taken in a preservice teacher training program (Zhang et al., 2011). For the Context component, a “service-learning faculty task force” was assembled to examine institutional records, standards, and curriculum to identify the necessity to improve retention of teachers in the workforce (Zhang et al., 2011). On the community side, the task force collaborated with adjunct faculty who taught in the school system to highlight the need to help at-risk readers in elementary school (Zhang et al., 2011). To assess the level of preparation from both the preservice teachers and elementary students, several quantitative assessments were distributed that provided insight into concepts like self-efficacy, proficiency in relevant skills, and confidence (Zhang et al., 2011). The Input component tasked the team with consulting university faculty, elementary school reading specialists and teachers, and national experts on SL to ensure the development of a properly prescribed project (Zhang et al., 2011). Regular meetings continued between task force members and employees of the elementary school, establishing an adaptive workflow that befits the Process component (Zhang et al., 2011). Finally, the Product component assessed the outcomes through reflections, quantitative assessments, interviews, observations, and stakeholder feedback (Zhang et al., 2011).
Similarly to Zhang et al. Meurer et al. took the CIPP model and applied it to their study, this time within the field of medicine (Meurer et al., 2011). Meurer et al. describe a portion of the Urban and Community Health Pathway (UCHP) program at the Medical College of Wisconsin. This program aims to foster medical students’ competency to adequately care for urban, underserved settings and ultimately improve overall health in those communities (Meurer et al., 2011). Throughout the experience, medical students worked with expert faculty and, if possible, community partner liaisons to carry out projects that addressed topics such as homelessness, Hmong and Latino health, HIV, and violence prevention (Meurer et al., 2011). Such projects might have entailed needs assessment, outreach education, and facilitating training sessions for other health workers (Meurer et al., 2011). A key component of their program is labeled as “service learning” (Meurer et al., 2011). However, upon closer inspection, Meurer et al. place emphasis on close interaction with the community using phrasing such as “community-identified needs” or asserting that the community-engaged activities must be guided “in collaboration with a community partner” (Meurer et al., 2011). Meurer et al. actually debated the terminology that was appropriate for their pedagogy, coming to call it a “community service learning,” an in-between label, highlighting the confusion that exists in the literature on terminology. Their focus and priorities here arguably lend themselves more to the nature of CEL than the label of SL. For the Context component, Meurer et al. describe the background information of the medical program as well as the institutional resources that allow many partnerships with the community to happen (Meurer et al., 2011). As for the Input component, many curricula combined with the vision of a council consisting of community-engaged faculty and medical students informed the project planning steps (Meurer et al., 2011). The Process component saw several core informative sessions where students, advisors, experts, and community partners could interact, present, and co-facilitate (Meurer et al., 2011). Finally, the Product component was captured in end-of-semester reflections written by the students (Meurer et al., 2011).
Comparing how the CIPP model is used between SL and CEL literature
After discussing the differences between SL and CEL, it was intriguing how differently the two studies incorporated the CIPP model in their projects. Notably, Zhang et al. were much more detailed than Meurer et al. in describing how each component was laid out. Most significantly, however, was the continued emphasis that was placed by Zhang et al. on the participation of the community partners at each step. Meurer et al. touch on collaborating with their community at various points but to a lesser extent than Zhang. Additionally, Zhang et al. more closely follow Stufflebeam’s original descriptions of the CIPP model. For example, Zhang’s Context component outlined several steps the researchers took to ensure enough background information to carry on with a SL project. In contrast, Meurer et al. only stated the background information of the program and the institutional resources. Building off this, Meurer et al. differed in the Product component in that they did not bring in additional perspectives, such as those from faculty members, experts, and community partners. It’s worth noting that some of these discrepancies could be due to the fact that Zhang et al. focused on a specific SL project, whereas Meurer et al. encompassed a whole program. The variance in how these studies involve their community partners creates an intriguing dissonance between semantics and execution. For example, the usage of the CIPP model in the study by Zhang et al. suggests that their project was more aligned with the label of CEL rather than SL, perhaps more so than Meurer et al. This provides further dialog on the existence of a spectrum on which SL and CEL are carried out. Even though a study may claim that it involves a SL project, how it’s carried out may be more in tune with CEL. Likewise, a project labeled as CEL might lack elements of adequately involving the community in the way that the “community-engaged” aspect demands. With this in mind, caution should be taken regarding labels used to describe projects in the facilitation and publication because future stakeholders may refer heavily back to these as examples to build their projects.
Applying the CIPP model to published SL/CEL projects
To demonstrate the CIPP model’s ability to evaluate SL and CEL projects post hoc, this section will show the model applied to a few of the examples of SL and CEL discussed in previous sections. The articles chosen to be included were not determined by any particular line of reasoning but rather on the sole basis of demonstration.
The first article we apply the CIPP model to is by Lee et al. In this case of SL, for the Context component, the needs of the academic partner can be seen in the objectives for the course and project. In this case, the course objectives for the Imi Ho’ōla’s Scientific Basis of Medicine course were:
1. “Explain and give examples of varying global perspectives regarding health and disease” (Lee et al., 2016).
2. “Describe the need for health-care services in rural and underserved populations.”
3. “Articulate the roles of health professionals working in the health care setting.”
4. “Predict the epidemiological and psychosocial impact diseases/health conditions have on society” (Lee et al., 2016).
5. “Recognize and apply professional behaviors and attitudes as recommended by the Association of American Medical Colleges (AAMC)” (Lee et al., 2016).
On the community side, a longstanding community leader consistently provided dialog with the university detailing community needs and activity suggestions for the students (Lee et al., 2016). While the article does not go into specifics on what some of the community needs were, both sides were able to articulate their needs. Since the community needs were not made clear by the authors, it is unknown whether the project was suited to those needs. With the Input component in mind, each side was able to contribute to some extent. The students researched topics, gave talks/presentations, and conducted service in the form of cleaning yards, painting, and visiting residents, while the community members organized hospital visits, tours of historical sites, and informed student activities (Lee et al., 2016). Looking at the Process component, mid-project evaluations nor adjustments were reported in the article. It is then presumed that any reflection was saved until the end. For the Product component, the students saw benefits in areas such as civic awareness, career development, and teamwork (Lee et al., 2016). The article did not communicate the extent to which the community partner saw “successful” outcomes.
For comparison, the article by Vance-Chalcraft and Osborne Jelks involving CEL will be revisited using the lens of the CIPP model. For their case study, the Context component was embodied on the academic side by the course objectives, which included the following:
1. “Students will be able to generally explain the potential fate and effects of a contaminant in the environment”(Vance-Chalcraft and Jelks, 2022).
2. “Students will be able to appreciate and discuss the role that technology and industrial population plays in our society and its capacity to alter the quality of the environment as well as solve problems caused by human impact”(Vance-Chalcraft and Jelks, 2022).
3. “Students will be able to make informed decisions regarding the toxicity of pollutants in the environment, their origins, and mitigation strategies that are protective of the environment and the health of the public” (Vance-Chalcraft and Jelks, 2022).
The authors did not include whether community partners articulated any problems they wanted to be addressed. For the Input component, contributions were weighed equally on both the side of academia and the community. Students worked with community members to sample and analyze water from the creek, which ultimately resulted in developing strategies to improve overall health, quality of life, and the environment (Vance-Chalcraft and Jelks, 2022). Looking at the Process component, while a formal mid-project review or adjustment was not reported, students did meet regularly with community members during events such as community meetings and sampling dates, which may have afforded both parties to reflect on the project (Vance-Chalcraft and Jelks, 2022). As for the Product component, the “success” of the project is not apparent since the case study results entailed strategies and actions for the community to take in the future. In addition, there was no included data on students’ reflections, so the extent to which the project was successful for the students is unclear. That being said, any implications for future improvement are included in the strategies and plans proposed from the Input component.
Discussion
Insight gained from CIPP usage to evaluate published projects
Based on the information provided by the study conducted by Lee et al. applying the CIPP model highlights the benefits/contributions of the SL project on the students’ part. For example, each activity described in the paper was from the students’ point of view. In contrast, the extent of the community contribution was only captured by the main community figure’s suggestions for student activities (Lee et al., 2016). However, the needs of the community were not made explicit by the authors (Lee et al., 2016).
When the CIPP was applied to the case study by Vance-Chalcraft & Osborne Jelks, it was clear that the community played a more significant role in this project. Between the joint sampling, analysis, and troubleshooting, students and community partners worked hand-in-hand to achieve a common (though not explicit) goal. However, the CIPP model also revealed the need for more product communication on the side of the student participant population.
This section shows that the CIPP model can be applied retroactively to past projects. Applying the CIPP model to past projects offered structured insight into how each project was carried out, highlighting the nuanced difference between SL and CEL in the role of community in each pedagogy. Applying the CIPP model also highlighted how SL and CEL are described in the literature; some components are more detailed in description than others, which may indicate more details would help to distinguish SL from CEL, improve outcome measures, and implementation of published pedagogies in other environments.
Summary, concluding thoughts, and implications
SL and CEL are grounded in theory that was initiated by ideas from individuals like John Dewey, William James, and David Kolb (Deutsch, 1995; Petkus, 2000; Pacho, 2015; Williams, 2017; Sotelino-Losada et al., 2021). These two pedagogical techniques both entail combining classroom-based objectives with community involvement yet possess specific characteristics that distinguish them (Agans et al., 2021). SL studies, despite showing a degree of reciprocity, might have seen academic SL project stakeholders being responsible for much of the work (Buchanan et al., 2002). Conversely, CEL studies might have had a further degree of interaction between academic and community stakeholders, with consistent sharing of responsibilities (Vance-Chalcraft and Jelks, 2022). That being said, there remain instances in which projects declared as SL contain facets of true reciprocity, lending themselves to align more closely with CEL (Santas, 2009). The same can be said in the opposite direction about CEL (Woodley et al., 2019). This can also be seen in Tables 1, 2 in that not every study under either SL or CEL provide evaluation of the community impact or partnership, further showing a muddling in terminology. The blurring of the line between SL and CEL is further exacerbated by the missing consensus on how to evaluate these projects (Tables 1, 2), which leaves room for sporadic and inconsistent assessment of the quality of projects as a whole. Because of these intrinsically different characteristics, it was proposed that a central framework (i.e., the CIPP model for evaluation) could be used to evaluate specifically how SL and CEL projects were carried out and how well each project fit the characteristics laid out in their respective term’s definition.
Though the pool of available literature using the CIPP model to evaluate SL and CEL was scarce, there were still instances in which usage of the model could be compared (Meurer et al., 2011; Zhang et al., 2011). Upon further analysis, there was a difference in how the CIPP model was used in each study. However, the manner in which each study incorporated the model surprisingly reflected characteristics of the other pedagogical techniques. In other words, the SL study showed traits that spoke more to a CEL framework. This result relates to goal (2) identify any differences in the evaluative process that arise due to the nuances that distinguish SL from CEL. It was unanticipated that the literature search would yield several examples of literature in which authors use the terms (SL vs. CEL) interchangeably. There were also hints of a spectrum that exists where two projects that might be considered both SL or both CEL could be wildly different due to differences in their implementation. This indicates a need for more consensus on the definitions of the terms SL and CEL, causing some projects to use either term while perhaps implying facets of the other one. The specific wording used in a project (i.e., SL vs. CEL) might affect the manner in which it is carried out. For example, would a project labelled as CEL prompt stakeholders to consider the project from a more collaborative/community-oriented mindset? This could be explored in future research and may support future use of the CIPP model to evaluate SL/CEL and ensure the proper term is used to describe the project.
Despite the CIPP model showing SL studies had CEL principles, the model is appropriate to use to evaluate SL and CEL because of its ability to piece apart and highlight critical areas that makeup and, ultimately, define each methodology, which addressed goal (1) discuss how SL and CEL could be assessed/evaluated using the CIPP model. This was seen by applying the CIPP model to published SL/CEL projects that did not already utilize the model in the creation of the projects. However, note that this review showed the effectiveness of using the CIPP model on published SL/CEL projects is highly dependent upon the level of detail used to describe the projects. Using the CIPP model post-implementation of a SL/CEL project shows the model’s effectiveness as an evaluation tool for those implementing these methods in the classroom setting. The post-analysis with CIPP may call for the application of CIPP in the planning stage of current projects, as it serves as a valuable tool to monitor quality, influence outcomes, and effectively plan the experience. Although the CIPP model’s usage is not limited to SL and CEL projects, it is a promising tool to use as a guide in the planning and evaluating SL and CEL projects.
Scope statement
This manuscript intends to reach all practitioners and supporters of service-learning and community-engaged learning and those interested in implementing these pedagogies. Consequently, the content of this article resides comfortably in the field of education. Since these pedagogical techniques have only increased in traction over the years, the growing pool of participants also increases the risk of suboptimal implementation and mislabeling of the practices of community-engaged learning and service learning. In the current climate of service-learning and community-engaged learning literature, this manuscript serves as a call to realign the two pedagogies with their ideologies in hopes that there is a more apparent distinction between the two in future published work. This manuscript also demonstrates and advocates for using the CIPP model to evaluate service-learning and community-engaged learning projects to optimize project success. It is imperative that the points discussed here are communicated and considered by practitioners and supporters of these pedagogies. As such, the authors deem it appropriate that a publication in Frontiers in Education adequately aligns with achieving this goal.
Author contributions
JN: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Resources, Supervision, Visualization, Writing – original draft. DC: Conceptualization, Funding acquisition, Project administration, Supervision, Writing – review & editing.
Funding
The author(s) declare financial support was received for the research, authorship, and/or publication of this article. Publishing costs contributed by the Department of Microbiological Sciences, North Dakota State University.
Acknowledgments
The authors would like to thank Kathryn Wissman, Sarah Sletten, Laura Dahl, and Samiran Banerjee for their insight, advice, and guidance in the creation of this article. The author would also like to thank the North Dakota State University Department of Microbiological Sciences for its support in research activities.
Conflict of interest
The authors declare that the research was conducted without any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Footnotes
References
Adkins-Jablonsky, S., Fleming, R., Esteban, M., Bucio, D., Morris, J. J., and Raut, S. (2021). Impacts of a COVID-19 E-service-learning module in a non-major biology course †. J. Microbiol. Biol. Educ. 22, 1–10. doi: 10.1128/jmbe.v22i1.2489
Agans, J., Rosenthal, E., Lesnick, J., Sloan, M., Connor, S., Majano, R., et al. (2021). Research-practice partnerships as community-engaged learning: lessons learned from a collaborative project with youth development programs. J. Commun. Engag. Scholarsh. 13, 1–10. doi: 10.54656/pbya5587
Anderson, J., Swick, K., and Yff, J. (2001). Service-learning in teacher education: enhancing the growth of new teachers, their students, and communities. In AACTE.
Bailey, P. A., Carpenter, D. R., and Harrington, P. (2002). Theoretical foundations of service-learning in nursing education. Available at: http://www.compact.org
Billig, S. (2011). Implementing the K-12 service-learning standards for quality practice. The Prevention Researcher.
Buchanan, A. M., Baldwin, S. C., and Rudisill, M. E. (2002). Service learning as scholarship in teacher education. Educ. Res. 31, 30–36.
Burns, L. T. (1998). Make sure it’s service learning, not just community service. Educ. Dig. 64, 38–41.
Cain, D. M. (2013). Impact of a service-learning project on student success in allied health microbiology course. J. Microbiol. Biol. Educ. 14, 129–130. doi: 10.1128/jmbe.v14i1.541
Cashman, S. B., and Seifer, S. D. (2008). Service-learning. An integral part of undergraduate public health. Am. J. Prev. Med. 35, 273–278. doi: 10.1016/j.amepre.2008.06.012
Chambers, D. J., and Lavery, S. (2012). Service-learning: a valuable component of pre-service teacher education. Australian. J. Teach. Educ. 37, 128–137. doi: 10.14221/ajte.2012v37n4.2
Cummings, C. K. (2000). John Dewey and the rebuilding of Urban Community: engaging undergraduates as neighborhood organizers. Michigan J. Commun. Serv. Learn. 7, 97–109.
Deutsch, M. (1995). William James: the first peace psychologist peace and conflict. J. Peace Psychol. 1, 27–35. doi: 10.1207/s15327949pac0101_5
Dewey, J. (1916). Democracy and education. An introduction to the philosophy of education. New York: MacMillan.
Driscoll, A., Holland, B., Gelmon, S., and Kerrigan, S. (1996). An assessment model for service-learning comprehensive case studies of impact on faculty, students, community, and institution. Michigan J. Commun. Serv. Learn. 3, 66–71.
Elam, C. L., Sauer, M. J., Stratton, T. D., Skelton, J., Crocker, D., and Musick, D. W. (2003). Service learning in the medical curriculum: developing and evaluating an elective experience. Teach. Learn. Med. 15, 194–203. doi: 10.1207/S15328015TLM1503_08
Ferrari, R. (2015). Writing narrative style literature reviews. Med. Writ. 24, 230–235. doi: 10.1179/2047480615z.000000000329
Gaines-Hanks, N., and Grayman, N. (2009). International service-learning in South Africa and personal change: an exploratory content analysis. NASPA J. 46, 72–93. doi: 10.2202/1949-6605.5006
Giles, D. E., and Eyler, J. (1994). The theoretical roots of service-learning in John Dewey: toward a theory of Self-learning. Michigan J. Commun. Serv. Learn. 1, 77–85.
Graulich, N., Lewis, S. E., Kahveci, A., Nyachwaya, J. M., and Lawrie, G. A. (2021). Writing a review article: what to do with my literature review. Chem. Educ. Res. Pract. 22, 561–564. doi: 10.1039/d1rp90006d
Henry, S. E., and Lynn Breyfogle, M. (2006). Toward a new framework of “server” and “served”: De(and re)constructing reciprocity in service-learning pedagogy. Int. J. Teach. Learn. High. Educ. 18, 27–35.
Homkes, R. (2008). Assessing IT service-learning. SIGITE’08: Proceedings of the 9th ACM SIG-Information Technology Education Conference, 17–21.
Hou, S.-I. (2014). Integrating problem-based learning with community-engaged learning in teaching program development and implementation. Univ. J. Educ. Res. 2, 1–9. doi: 10.13189/ujer.2014.020101
Hwang, C., Liu, H., and Salusso, C. J. (2019). Social responsibility initiative: examining the influence of a collaborative service learning project on student learning. Int. J. Fash. Des. Technol. Educ. 12, 356–363. doi: 10.1080/17543266.2019.1652854
James, W. (1995). The moral equivalent of war? Peace Conf. J. Peace Psychol. 1, 17–26. doi: 10.2307/j.ctt1ffjh9n.9
Johnson, P. D., Johnson, P., Johnson, P. D., Johnson, P. W., and Shaney, N. (2008). Developing contemporary engineering skills through service learning in Peru. J. Commun. Engage. Scholarsh. 1, 81–84.
Jones, J. L., Gallus, K. L., Manning-ouellette, A., and Affairs, S. (2020). Exploring the impact of community-engaged programs on undergraduate students. Attitud. Toward Intellect. Disabil. 14, 19–33.
Knapp, B. J., Stoner, J., Lang, J., Johnson, R., Flenner, R., and Gathambo, M. (2022). An emergency medicine based model for community-engaged learning. J. Am. Coll. Emerg. Phys. Open 3, 1–5. doi: 10.1002/emp2.12752
Kolb, D. A. (2014). Experiential learning: Experience as the source of learning and development. New Jersey: FT Press.
Lake, V. E., and Jones, I. (2008). Service-learning in early childhood teacher education: using service to put meaning back into learning. Teach. Teach. Educ. 24, 2146–2156. doi: 10.1016/j.tate.2008.05.003
Lee, W. K., Harris, C. C. D., Mortensen, K. A., Long, L. M., and Sugimoto-Matsuda, J. (2016). Enhancing student perspectives of humanism in medicine: reflections from the Kalaupapa service learning project. BMC Med. Educ. 16, 1–9. doi: 10.1186/s12909-016-0664-7
Lo, K. D., Holler, D., Iglesias, T., and Katz, S. R. (2015). Profiles in community-engaged learning. San Francisco: McCarthy Center Faculty Publications.
Long, A. B., Larsen, P., Hussey, L., and Travis, S. S. (2001). Organizing, managing, and evaluating service-learning projects. Educ. Gerontol. 27, 3–21. doi: 10.1080/036012701750069012
Long, J. A., Lee, R. S., Federico, S., Battaglia, C., Wong, S., and Earnest, M. (2011). Developing leadership and advocacy skills in medical students through service learning. J. Public Health Manag. Pract. 17, 369–372. doi: 10.1097/PHH.0b013e3182140c47
Makani, B., and Rajan, N. M. (2016). Understanding business concepts through application: the pedagogy of community-engaged learning. J. Bus. Educ. Leadersh. 6, 82–92.
Malotky, M. K. H., Mayes, K. M., Price, K. M., Smith, G., Mann, S. N., Guinyard, M. W., et al. (2020). Fostering inclusion through an interinstitutional, community-engaged, course-based undergraduate research experience. J. Microbiol. Biol. Educ. 21:11. doi: 10.1128/jmbe.v21i1.1939
Meidl, C., Chao, X., and FitzGerald, A. M. (2018). Establishing asset thinking with preservice teachers: community-engaged learning with young children from public housing. Handb. Res. Serv. Learn. Initiat. Teach. Educ. Prog., 257–277. doi: 10.4018/978-1-5225-4041-0.ch014
Meredith, G. R. F. (2020). Community engaged teaching, research and practice: a catalyst for public health improvement. Michigan journal of community service. Learning 26, 75–100. doi: 10.3998/mjcsloa.3239521.0026.106
Meurer, L. N., Young, S. A., Meurer, J. R., Johnson, S. L., Gilbert, I. A., and Diehr, S. (2011). The urban and community health pathway: preparing socially responsive physicians through community-engaged learning. Am. J. Prev. Med. 41, S228–S236. doi: 10.1016/j.amepre.2011.06.005
Mintz, S., and Liu, G. (1994). Service learning: an overview. Washington, DC: Corporation for National and Community Service, 12.
O’Connor, K. M., McEwen, L., Owen, D., Lynch, K., and Hill, S. (2011). Literature review: embedding community engagement in the curriculum: an example of university-public engagement. National Coordinating Centre for Public Engagement.
Pacho, T. O. (2015). Unpacking John Dewey’ s connection to service-learning University of Hamburg. J. Educ. Soc. Policy 2, 8–16.
Peterson, J. J., Wardwell, C., Will, K., and Campana, K. L. (2014). Pursuing a purpose: the role of career exploration courses and service-learning internships in recognizing and developing knowledge, skills, and abilities. Teach. Psychol. 41, 354–359. doi: 10.1177/0098628314549712
Petkus, E. (2000). A theoretical and practical framework for service-learning in marketing: Kolb’s experiential learning cycle. J. Mark. Educ. 22, 64–70. doi: 10.1177/0273475300221008
Petri, A. N. (2012). Reciprocal exchange: Understanding the community partner perspective in higher education service-learning Kansas City: University of Missouri-Kansas City.
Saltmarsh, J., Giles, D. E. Jr., Ward, E., and Buglione, S. M. (2009). Rewarding community-engaged scholarship. N. Dir. High. Educ. 147, 25–35. doi: 10.18060/21128
Santas, A. J. (2009). Reciprocity within biochemistry and biology service-learning. Biochem. Mol. Biol. Educ. 37, 143–151. doi: 10.1002/bmb.20291
Self, J., Handforth, B., Hartman, J., McAuliffe, C., Noznesky, E., Schwei, R., et al. (2012). Community-engaged learning in food systems and public health. J. Agric. Food Syst. Commun. Dev. 3, 113–127. doi: 10.5304/jafscd.2012.031.006
Sotelino-Losada, A., Arbués-Radigales, E., García-Docampo, L., and González-Geraldo, J. L. (2021). Service-learning in Europe. Dimensions and understanding from academic publication. Front. Educ. 6, 1–13. doi: 10.3389/feduc.2021.604825
Strasser, R. P., Lanphear, J. H., McCready, W. G., Topps, M. H., Hunt, D. D., and Matte, M. C. (2009). Canada’s new medical school: the northern Ontario school of medicine: social accountability through distributed community engaged learning. Acad. Med. 84, 1459–1464. doi: 10.1097/ACM.0b013e3181b6c5d7
Stufflebeam, D. L. (2000). The CIPP Model for Evaluation, In: Evaluation Models. Evaluation in Education and Human Services. Eds. D.L. Stufflebeam, G. F. Madaus, and T. Kellaghan. Dordrecht: Springer. 49. doi: 10.1007/0-306-47559-6_16
Toncar, M. F., Reid, J. S., Burns, D. J., Anderson, C. E., and Nguyen, H. P. (2006). Uniform Assessment of the Benefits of Service Learning: The Development, Evaluation, and Implementation of the Seleb Scale, J. Mark. Theory Pract. 14, 223–238. doi: 10.2753/MTP1069-6679140304
Trott, C. D., Sample McMeeking, L. B., and Weinberg, A. E. (2019). Participatory action research experiences for undergraduates: forging critical connections through community engagement. Stud. High. Educ. 45, 2260–2273. doi: 10.1080/03075079.2019.1602759
Vance-Chalcraft, H. D., and Jelks, N. O. (2022). Community-engaged learning to broaden the impact of applied ecology: a case study. Ecol. Appl. 33. doi: 10.1002/eap.2768
Woodley, S. K., Freeman, P. E., and Ricketts, T. D. (2019). Combining novel research and community-engaged learning in an undergraduate physiology laboratory course. Adv. Physiol. Educ. 43, 110–120. doi: 10.1152/advan.00177.2018
Ylitalo, K. R., and Meyer, A. R. (2019). Bringing service into science: community-engaged service-learning for undergraduate and graduate epidemiology students. Pedag. Health Promot. 5, 89–98. doi: 10.1177/2373379918794970
Yontz, E., and Mccook, K. D. L. P. (2003). Service-learning and LIS education. J. Educ. Libr. Inf. Sci. 44:58. doi: 10.2307/40323942
Zhang, G., Zeller, N., Griffith, R., Metcalf, D., Williams, J., Shea, C., et al. (2011). Using the context, input, process, and product evaluation model (CIPP) as a comprehensive framework to guide the planning, implementation, and assessment of service-learning programs. J. High. Educ. Outreach Engag. 15, 57–84.
Keywords: context, input, process, product, Community-engaged learning, service-learning
Citation: Nguyen JJ and Condry DLJ (2023) Evaluating differences in community-engaged learning and service-learning via the context, input, process, and product model. Front. Educ. 8:1289322. doi: 10.3389/feduc.2023.1289322
Edited by:
Dolores Ortiz-Masià, University of Valencia, SpainReviewed by:
Dulce Carolina Macias Ceja, University of Valencia, SpainBrigita Miloš, University of Rijeka, Croatia
Copyright © 2023 Nguyen and Condry. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Jonathan J. Nguyen, am9uYXRoYW4uai5uZ3V5ZW5AbmRzdS5lZHU=