Sharing Experiences in Designing Professional Learning to Support Hydrology and Water Resources Instructors to Create High-Quality Curricular Materials

The creation of high-quality curricular materials requires knowledge of curriculum design and a considerable time commitment. Instructors often have limited time to dedicate to the creation of curricular materials. Additionally, the knowledge and skills needed to develop high-quality materials are often not taught to instructors. Furthermore, similar learning material is often prepared by multiple instructors working at separate institutions, leading to unnecessary duplication of effort and inefficiency that can impact quality. To address these problems, we established the HydroLearn platform and associated professional learning experiences for hydrology and water resources instructors. HydroLearn is an online platform for developing and sharing high-quality curricular materials, or learning modules, focused on hydrology and water resources. The HydroLearn team has worked with three cohorts of instructors from around the world who were dedicated to creating high-quality curricular materials to support both their students and the broader community. In order to overcome some of the aforementioned barriers, we tested and revised several different models of professional learning with these cohorts. These models ranged from (a) instructors working individually with periodic guidance from the HydroLearn team, to (b) small groups of instructors collaborating on topics of shared interests guided through an intensive HydroLearn training workshop. We found the following factors to contribute to the success of instructors in creating modules: (1) instructor pairs co-creating modules enhanced the usability and transferability of modules between universities and courses, (2) dedicating an intensive block of time (∼63 h over 9 days) to both learning about and implementing curriculum design principles, (3) implementing structures for continuous feedback throughout that time, (4) designing modules for use in one’s own course, and (5) instituting a peer-review process to refine modules. A comprehensive set of learning modules were produced covering a wide range of topics that target undergraduate and early graduate students, such as: floodplain analysis, hydrologic droughts, remote sensing applications in hydrology, urbanization and stormwater runoff, evapotranspiration, snow and climate, groundwater flow, saltwater intrusion in coastal regions, and stream solute tracers. We share specifics regarding how we structured the professional learning models, as well as lessons learned and challenges faced.

Introduction

Creating high-quality curricular materials can be challenging for instructors, given that the creation of these materials requires both knowledge of curriculum design, as well as a considerable time commitment (Borrego et al., 2010; Bourrie et al., 2016; Habib and Deshotel, 2018). Many university-level instructors completed doctoral coursework that did not cover the knowledge and skills needed to develop high-quality curricular materials (DeChenne et al., 2012). Moreover, instructors often have multiple commitments, including teaching, conducting research, and service to the university and field. This leaves limited time for the creation of curricular materials. Additionally, when instructors do invest time in creating curricular materials, they often do this work alone and for their own courses. While developing curricular materials is an important part of the teaching process in higher education, multiple instructors around the world creating similar curricular materials is inefficient and duplicative, and may impact quality. In addition to the issues around the creation of high-quality curricular materials (Ruddell and Wagener, 2015), the recent COVID-19 pandemic generated a need for high-quality curricular materials (Loheide, 2020) that can be accessed online and are openly available. This rapid transition to online instruction was challenging for many faculty. For instance, Johnson et al. (2020) found that 97% of higher education administrators reported that at least some of their faculty had no online teaching experience and 61% of administrators reported that the greatest need was increased access to online digital materials. Many instructors were not just looking for materials online (e.g., repositories of slides), but rather modules that students could engage in.

We have sought to address these problems within the field of hydrology and water resources by establishing the HydroLearn platform.¹ HydroLearn allows instructors to find, adapt, and use high-quality online modules. Although the HydroLearn platform was designed prior to the COVID-19 pandemic, it was positioned to support instructors in the rapid transition to online instruction and serves as a useful resource of online modules to support hydrology and water resources content. To support instructors in creating high-quality curricular materials, the HydroLearn team designed online professional learning experiences, both synchronous and asynchronous, to support instructors in learning about research-based practices in curriculum design. We refer to instructors who participated in these learning experiences as fellows. The purpose of this article is to describe two approaches to professional learning experiences the HydroLearn team created to support fellows’ use of research-based practices to design online modules. We first describe the research upon which our model for curriculum design and professional learning is based, then the two approaches of the professional learning experiences that we designed, and lastly we share lessons learned.

Pedagogical Frameworks

High-quality curricular materials are defined as those that have evidence of student learning, follow research-based methods of curriculum design, and are accessible to students with a variety of learning needs. The modules that were the outcomes of these professional learning experiences have evidence of student learning (Byrd et al., under review; Roundy et al., under review), were designed using the research-based methods described below, and as part of the review process, were required to incorporate features to make them more accessible (e.g., including captions on all videos, making sure figure captions were readable by screen readers, etc.). Thus, we consider the HydroLearn modules to be high-quality curricular materials. To support fellows in the creation of HydroLearn modules, we brought together two pedagogical frameworks: one from research in curriculum design (i.e., Backward Design) and one from research in professional learning (i.e., workshops). We describe the literature related to each of these in turn below.

Curriculum Design

The framework for curriculum design that we used is Backward Design (Wiggins and McTighe, 2005):

“One starts with the end—the desired results (goals or standards)—and then derives the curriculum from the evidence of learning (performances) called for by the standard and the teaching needed to equip students to perform.”

Backward design is an iterative process of curriculum design in which instructors first define their learning objectives, then create assessments that align with those learning objectives, and lastly design the content to be taught which will set students up to be successful with the assessments (see Figure 1). At each step, curriculum designers are constantly considering the constructive alignment of their materials, including carefully examining that the stated learning objectives match the assessed learning objectives and that the content taught will allow students to learn the content and skills needed to be successful in the assessments.

FIGURE 1

Figure 1. Backward design model used to develop HydroLearn modules.

Learning objectives specify “not only what is to be learned, the topic, but how it is to be learned and to what standard” (Biggs and Tang, 2011, p. 97–98). Effective learning objectives are written with specific principles in mind, including that the learning objectives be written with measurable verbs (i.e., verbs that can be observed, such as identify instead of verbs that cannot be observed, such as learn or understand) and that instructors take into account the level of cognitive demand required of the learning objectives included within their modules, specifically using Bloom’s Levels of Cognitive Demand (Krathwohl, 2002) to classify each learning objective. Hollowell et al. (2017) found that online courses that included clear learning objectives and constructive alignment, among other characteristics, were correlated with higher student learning, as measured by course grades.

Within the framework of Backward Design, once the learning objectives have been written, instructors should design assessments that align with those objectives. Authentic, high cognitive demand tasks can be used as assessments and are helpful in measuring learning objectives at the higher levels of Bloom’s Taxonomy. High cognitive demand tasks include: (a) “guidance for working with [the] practices [of a discipline] but require students to access their own content knowledge;” (b) multiple possible “correct” answers where correctness is based on accurately applying content and justifying decisions; and (c) “engaging in practices to make sense of content and recognize how a scientific body of knowledge is developed” (Tekkumru-Kisa et al., 2015, p. 663). Authentic tasks are tasks that have real world relevance and may be representative of the task a learner of the subject may need to undertake with the knowledge learned. They are a subset of high cognitive demand tasks, and allow competing solutions and a variety of outcomes (Herrington et al., 2003). In engineering, activities that include the use of online computational and analysis tools, such as Jupyter Notebooks and Google Colab, offer opportunities for students to use real world open and accessible data to solve authentic, high cognitive demand engineering tasks.

Following Backward Design, once the learning objectives and assessment are crafted and aligned, instructors must then design the content to be taught (i.e., the content that will get students from their knowledge and skills at the beginning of the course to the knowledge and skills needed to complete the authentic task). The design of online materials allows for the inclusion of video, text, images, and animations to support students’ comprehension of the content (Kumar et al., 2019). Additionally, when content is presented in online modules, students have the opportunity to revisit content as needed, which is typically not possible when content is delivered in-person (Mok, 2014).

Backward Design affords a specific organizational structure that allows instructors new to curriculum design to begin to create high-quality curricular materials. However, the format of the professional learning experience in which instructors learn about curriculum design can also impact their success in curriculum writing. Therefore, we purposefully provided training on Backward Design within a workshop model to support fellows’ professional learning.

Professional Learning

Research on the professional learning of instructors highlights the need for those experiences to focus on the specific content instructors will be teaching and how students learn that content, to align with instructors’ experience in the classroom, to use curriculum materials and assessments, and to be spread over time (Garet et al., 2001; American Educational Research Association [AERA], 2005; Desimone, 2011). Moreover, Walpole and McKenna (2015) described the importance of working with instructors in teams, given that they are influenced by their colleagues. Workshops are one way to create collegial environments that support participants in learning from one another (Loucks-Horsley et al., 2010). Researchers agree that lecturing is often not a useful pedagogical approach for professional learning experiences (Loucks-Horsley et al., 2010). Mundry et al. (2000, p. 6–8) suggest several features of effective workshops, including making sure participants are aware of the goals and that the goals align with those of the participants, integrating a variety of activities, and creating space for participants to create products that are useful for their goals. We considered these features in designing the HydroLearn professional learning experiences to support fellows in learning about curriculum design.

Learning Environment

The design of the professional learning experiences offered through the HydroLearn program went through two different approaches and engaged three cohorts of fellows over a period of 3 years. In education literature, the term “learner” often refers to K-20 students, but in this paper, the fellows were the learners, as they were the participants in our professional learning experiences. Given that time is a major constraint in the development of high-quality curricular materials, we asked participating fellows to dedicate time to this work and we compensated them for that time.

Structure of Workshops

Recruitment for the workshops varied by cohort. For Cohort 1, we used a targeted approach and invited specific individuals to apply for the fellowship. For Cohorts 2 and 3, we expanded our methods to include social media outreach, requests for applications via partnership channels, and direct email invitations. Some applicants to Cohorts 2 and 3 learned of the opportunity through word-of-mouth from friends and colleagues.

Cohort 1: The first approach, which we used with Cohort 1, was based on inviting individual instructors to develop teaching modules and deploy them on the HydroLearn platform. Each participating fellow worked individually to develop a module on a topic of interest that they planned to use in their respective courses. The guidance took place via bi-weekly virtual meetings and iterative review rounds of the modules throughout the academic year. The meetings were attended by the cohort participants (see Table 1 for details); however, the development of the modules and the review process were primarily done on an individual basis, on their own time, for each participant. Interaction amongst the different participants was minimal and was limited to the time when they co-participated in the periodical meetings.

TABLE 1

Table 1. HydroLearn fellows by cohort.

At the conclusion of Cohort 1, we hosted a virtual meeting with the participants to solicit their feedback about how to improve our model for professional learning. We also met with our project external evaluator to review our progress with Cohort 1 and gather his feedback regarding modifications we could make to Cohort 2. Based on this feedback, we revised our approach to foster more collaboration between the fellows and facilitate a process for improvement through interactions between the fellows (see Figure 2). This revised approach adopted a workshop structure to facilitate an intensive, collaborative experience. Also, recognizing the value of and limitations on fellows’ time, we strove to have most work completed during the summer workshop (although in many cases there was considerable post workshop work). We called these workshops hackathons in reference to their intensive, collaborative, and online nature. We had initially planned to host the Cohort 2 hackathon in-person, however, just as we were preparing to announce the workshop, the COVID-19 pandemic took hold and many states went on lockdown. We pivoted to an online hackathon for Cohort 2, which we found to be quite effective, and so repeated this format the following summer with Cohort 3.

FIGURE 2

Figure 2. Two approaches to professional learning used to develop HydroLearn modules.

Cohorts 2 and 3: Following a hackathon approach, the fellows in Cohorts 2 and 3 came together, virtually, from across the world (Table 1). We placed the fellows in groups of 2–3 to collaboratively develop modules. The groups worked collaboratively to design and build a module of joint interest both synchronously and asynchronously. The worktime was setup following an iterative design approach with multiple sharing points and checking-in with other participants and the HydroLearn guides during the assigned hackathon time. In a few instances (in Cohort 2), some groups decided to create individual modules but still interact in their groups for feedback.

Differences in the nature and the timing of the cohorts resulted in a larger number of participants in Cohorts 2 and 3 compared to Cohort 1. For example, the pre-defined timeframe of Cohorts 2 and 3, compared to a rather loose participation time in Cohort 1, probably encouraged more instructors to commit and participate as fellows. Also, the recruitment announcement for Cohort 2 was sent out at the onset of the first COVID-19 wave, at which time faculty had already switched to remote instruction and the concept of co-developing sharable curricular material was most appealing. The success of Cohort 2 probably propagated into the community and colleagues encouraged each other to participate in Cohort 3, during which COVID-19 conditions were still highly present.

The hackathons lasted 9 and 8 days for Cohorts 2 and 3, respectively. During each day fellows spent 2–4 h receiving training on the development of teaching content using research-based pedagogical approaches (described below), 3–5 h working in their groups, and 1 h receiving feedback from peers. The workshop leaders included a team of three hydrology and water resources professors who acted as “content guides,” two education professors who acted as “education guides,” and four graduate student researchers with expertise in the functionality of the HydroLearn platform who acted as “technical guides.” Each group of fellows was assigned one content, one education, and one technical guide who provided them with feedback throughout the hackathon and beyond.

The workshop was conducted with the overall goal of developing high-quality modules that could be used as-is or adapted by other instructors (across the world) in their courses. Therefore, and based on the HydroLearn team’s prior research in developing effective and adoptable learning modules (e.g., Habib and Deshotel, 2018; Habib et al., 2019) and other existing studies (e.g., Henderson et al., 2015; Shekhar and Borrego, 2016), the workshop participants were advised to consider the following aspects when developing their modules:

• Develop a module that follows evidence-based active learning pedagogical practices and that you, as the instructor, could use in your courses.

• While the primary and immediate users of the modules will be the ones who developed them, please develop the modules for potential use by other instructors.

• As you are working, think to yourself, “Is this something a colleague could use without my assistance?”

• Use open-source textbooks, readings, and software rather than copyrighted or subscription-based when possible.

The workshop interwove guidance and instruction on developing learning objectives, authentic tasks, rubrics, and content following evidence-based pedagogical practices, described in detail in the next section. We also incorporated time and support for hands-on content development using the HydroLearn platform allowing collaboration, discussion, and feedback around the effectiveness of the content being developed for achieving learning objectives.

Elements of Curriculum Design

One key aspect of the HydroLearn hackathon was the engagement of the fellows in intensive experiences to learn and apply processes of high-quality curriculum design. Before the start of the hackathons, fellows were asked to engage in a module on HydroLearn which was developed to be a primer in curriculum design (Gallagher et al., 2019). The concepts of Backward Design and authentic tasks were first introduced in this module and then reinforced during the live hackathons. Fellows were introduced to the backward design process articulated by Wiggins and McTighe (2005).

This process of beginning with the end in mind (i.e., identifying desired results) provided opportunities for fellows to clearly define the most essential learning objectives for their module by revisiting course outcomes, program goals, and professional standards. Next, fellows determined acceptable evidence that demonstrated that students met the desired learning objectives. Finally, fellows developed the instruction and hands-on learning experiences needed to move students toward demonstration of key learning performances. To operationalize the backward design process, fellows were introduced to the concept of constructive alignment (Biggs and Tang, 2011; Biggs, 2014). Figure 3 shows the key components fellows were asked to consider during module design: learning objectives, assessment task, and instruction. Throughout the module design process, fellows were asked to evaluate their module design for constructive alignment between these key components.

FIGURE 3

Figure 3. Components of constructive alignment used to develop HydroLearn modules.

Learning Objectives

In order to scaffold fellows in developing high-quality learning objectives, they were asked to use Bloom’s Taxonomy as a means to ensure (a) each learning objective was properly structured [i.e., (CONDITION), the student will be able to (ACTION) (TASK) (DEGREE)] and (b) that at least some of the learning objectives were at the upper end of Bloom’s Taxonomy (i.e., analyze, evaluate, create). Below are two example learning objectives from the Introduction to Floodplain Analysis module:

• Delineate watersheds and measure their associated properties/characteristics (Understand, Apply)

• Formulate a floodplain analysis that considers alternative design criteria (Create)

• Compare design alternatives under a changing climate (Evaluate)

In developing these learning objectives, fellows drew from their own domain and teaching experience, degree program outcomes, and professional standards. They then used their learning objectives to drive the design of their authentic assessment tasks.

Assessment Tasks

In HydroLearn, culminating assessments are performance tasks that demonstrate students have met the learning objectives. When scaffolding fellows in designing performance tasks, we considered the research on the cognitive demand of tasks (Stein and Lane, 1996; Boston and Smith, 2009; Tekkumru-Kisa et al., 2015) and modified a framework for evaluating cognitive demand developed by Tekkumru-Kisa and colleagues to include two categories: low and high cognitive demand (Table 2). Low cognitive demand tasks are aligned with Bloom’s Taxonomy levels: remember, understand, and apply. High cognitive demand tasks are aligned with analyze, evaluate, and create. The juxtaposition of low and high cognitive demand tasks provided valuable insights to fellows for the design of their own module learning tasks.

TABLE 2

Table 2. Characteristics of low and high cognitive demand tasks adapted from Tekkumru-Kisa et al. (2015).

To further deepen fellows’ understanding of the characteristics of a high cognitive demand performance task, they were introduced to the qualities of authentic tasks. Authentic, high cognitive demand tasks included in HydroLearn are expected to mimic the types of problems that engineers may be asked to solve. For instance, the HydroLearn module entitled Introduction to Floodplain Analysis (Polebitski and Smith, 2020) engages students in authentic high-cognitive demand tasks. This module guides students through the analysis of a flood prone area on the Pecatonica River near Darlington, Wisconsin. In this real-world context, students:

• delineate the Pecatonica Basin using StreamStats and data from National Water Information System,

• apply principles of frequency analysis to determine peak discharge for the Pecatonica River,

• create, execute, and analyze a HEC-RAS model, and

• create a design and recommendation for the property of interest.

This engaging module provides an authentic context and tasks for students.

Instruction

Once fellows had well-articulated learning objectives at low and high levels of Bloom’s Taxonomy that were clearly aligned to authentic learning tasks, they turned their attention to consider the content and learning experiences students would need to be successful on those tasks. For example, in the Introduction to Floodplain Analysis (Polebitski and Smith, 2020) module, the instructional materials to prepare students for the task of delineating a watershed include:

• videos defining watershed delineation, and watershed classification (HUC system), soil characteristics (e.g., texture, compaction, depth), geomorphology, land use and land cover;

• video, text, and images on using StreamStats to delineate watersheds, retrieve basin properties, and use exploration tools (e.g., measure, elevation profile); and

• video, text, and images on using the National Water Information System.

The content knowledge and “how to” videos prepare students for the culminating task of delineating the Pecatonica Basin.

Review Process

Completed learning modules were then shared with a three-person review team consisting of a content guide, an education guide and a technical guide. This team used a detailed review form designed to evaluate the occurrence and quality of (1) relevant content, (2) authentic tasks, (3) clear learning objectives, (4) engaging and accessible delivery, and (5) clear and engaging learning activities with constructive alignment (see Supplementary Material for full HydroLearn Module Review Form). The review form included specific targeted questions such as “Do the learning objectives in this module represent different levels of Bloom’s Taxonomy?” and “Is sufficient text/video presented to clearly explain key ideas?” as well as space for open-ended feedback/comments related to each module section. Reviews by each of the guides were returned to the module author(s), who subsequently submitted a revised module including responses to reviewer comments for final approval by the project team. This review process provided an opportunity to evaluate key pillars of HydroLearn modules presented in the hackathon and promote consistency among the modules.

Results to Date

Given that our focus is on the professional learning experiences of hydrology and water resource engineering instructors, our results focus on the products and experiences of the HydroLearn fellows. The outcomes of HydroLearn’s workshop/hackathon approach include 34 modules (completed to date) completed by the fellows that span a broad range of crucial topics in the field of hydrology and water resources. The subjects of the modules include, but are not limited to, Fluid Mechanics, Open Channel Flow, Physical Hydrology, Groundwater, Irrigation, Hydraulics, and Water Resources Management (see Supplementary Material for a list of each module and its learning objectives). Each module is designed around an authentic, high-cognitive demand task that emulates the work of professionals in the field. Of these 34 modules, 30 were implemented during the COVID-19 pandemic in the fellows’ own classes. Additionally, 4 fellows chose to also implement modules written by other fellows in their courses. In addition to creating modules, several fellows have written about the unique contributions made by their modules, particularly highlighting the affordances of authentic tasks in an online format, which allows the integration of sophisticated software used by engineers in the field (Maggioni et al., 2020; Lane et al., 2021; Roundy et al., under review).

Over the course of the 2 years in which we implemented the HydroLearn professional learning experiences for Cohorts 1–3, we learned many lessons. Here we present aspects of the learning experiences that worked and challenges we faced in the hope that we can inform future efforts which also seek to design professional learning experiences to support instructors in designing high-quality sharable curricula.

What Worked

As previously mentioned, we made major revisions to our professional learning approach between Cohorts 1 and 2. We found that the quality of the modules submitted by fellows for review, was higher for those in Cohorts 2 and 3, as compared to Cohort 1. We assess this quality based on the amount of feedback and revisions we needed to ask fellows for before accepting their modules. There are several possible reasons for the higher quality of the first submitted modules by the cohorts which participated in the hackathon style workshop, including the intensive nature of the workshop, the commitment fellows made to attend all sessions, and the collaborative nature of the workshops.

We found the intensive workshop structure of the hackathons to be much more fruitful for the fellows than the periodical meetings that fellows in Cohort 1 experienced. We posit that combining training on evidence-based pedagogical practices and hands-on student activities during the hackathon enabled fellows to learn and then enact their learning immediately. The hackathon event also imposed an organized structure and a schedule over a specified time frame that led to fellows finishing the modules successfully. For instance, 17% of Cohort 1 fellows finished within 6 months of the end of the meetings, whereas 67% of Cohort 2 and 40% of Cohort 3 finished within 6 months. Unlike the approach used with Cohort 1, as part of their acceptance into the hackathons, fellows in Cohorts 2 and 3 were asked to commit to attend the designated days and times of their workshops. This dedicated time for both learning about and creating the modules seems to have supported fellows in the timely completion of their modules. In addition to the change to the structure of the meetings for Cohorts 2 and 3, another shift we made was to ask fellows to create modules collaboratively (i.e., two or three fellows working together to create one module), whereas each of the Cohort 1 fellows created their own modules. We found that the collaborative approach was highly effective and had a positive impact on the quality of the final products, possibly because it imposed peer evaluation and discussion and validation of the pedagogical structure of the ideas. Moreover, the pairings promoted sharing of content and cross checking that strengthened the modules that were developed. This process also made the modules more transferable/modular since they had to meet the needs of two distinct fellows and their associated courses and students. Although the module designs and some content were developed in pairs, some of the paired teams in Cohort 2 produced separate modules, which were also of high quality. Ultimately, we found that each fellow had to have ownership of the module they were using in their class and adapt it to their specific needs.

An unexpected positive outcome from Cohorts 2 and 3 was the sharing of modules within and between cohorts. We speculate that there is more within cohort sharing amongst our latest cohorts due to the structure of the workshop. We had daily check-ins and activities across small working groups, and modules of similar topics (e.g., climate change and drought) were grouped together for these discussions. For Cohort 3, a few fellows decided to expand upon the modules created by previous fellows which may also account for the sharing of modules between cohorts. In our most recent and extensive study (see Byrd et al., under review), the majority of the modules were implemented by the fellows who developed them. The exceptions to this were instances where fellows were sharing modules, as described above, and the use of three modules developed by the HydroLearn team that were used by a professor who was not a developer. Unfortunately, we cannot track how many faculty members who did not participate in the fellowship have adopted HydroLearn modules.

Although we had initially wanted to host an in-person hackathon for Cohort 2, the pivot to an online format had unanticipated positive impacts. First, this format forced us to find a structure for the workshop that kept fellows engaged for 7 h per day, which lead to the structure of: training, work time, feedback which we found to be successful. The pivot to an online format, rather than having an in-person weeklong workshop, likely also allowed primary caregivers to attend the workshop. Although the commitment was 7 h per day, there were many breaks throughout the day when fellows would stop to check on their children. In spite of these affordances, it may be that instructors who lost childcare due to the pandemic may have chosen not to apply to the fellowship program. However, the percent of the fellows who identified as female was higher in 2020, as compared to 2021, suggesting that perhaps childcare was not more of an obstacle in 2020 than in 2021. Most importantly, though, was that the move to an online format broadened participation. In Cohorts 2 and 3 we had fellows from Sweden, New Zealand, Ethiopia, and Turkey, among others. Without the impetus of the COVID-19 pandemic, we would likely have kept an in-person format and missed the opportunity to connect with this broader community.

Surprisingly, we did not see an immediate spike in new users when the pandemic hit. However, it does seem that HydroLearn has more than doubled in popularity since that time. Examining the Google Analytics for our homepage, we had nearly 20,000 page views between March 1 and May 31, 2020. Comparatively, the page views from the last 3 months (January 1 to March 31, 2022) were roughly 47,000. We cannot say definitively that this increase is due to the pandemic alone; However, we suspect that it likely played a role in this gain.

Lastly, the peer-review process, and the reviews provided by the guides, was key to strengthening the modules and make them a useful teaching tool. Although the review process was time-consuming, the reviews written through three different lenses (i.e., content, education, and technical guides) provided fellows with rigorous feedback and opportunities to revise.

Challenges Faced

Overall, we felt that the HydroLearn professional learning experiences, and in particular the hackathon workshops, were effective in supporting fellows in developing high-quality modules related to water resources and hydrology for undergraduate courses. Indeed, there is evidence that these modules have supported student learning gains (Byrd et al., under review; Roundy et al., under review). However, we also faced some challenges throughout this process related to time, over-committing, and collaboration. The most notable challenge was the considerable time commitment on the part of the fellows as well as the HydroLearn guides. Developing rich content, with active-learning components and real-world applications, is time-consuming and requires commitment from instructors. Designing and running the workshops, following up with fellows after the workshops, and engaging in peer reviews took considerable time for the HydroLearn guides. Additionally, although the fellows produced 34 modules across the 3 cohorts, 5 modules are still in progress, and 2 modules have been abandoned. For some fellows, the time commitment proved too great, especially as weighed against other demands on their time.

Relatedly, some fellows over-committed in the early design stages of their modules. Defining a reasonable scope for a module or designing it to be modular enough that some portion could be completed well in a reasonable amount of time was a challenge - both for the fellows and for the guides. Fellows were often excited about the modules and potential of HydroLearn and thus laid out a plan for a module that was larger than they had the capacity to finish in a reasonable amount of time. Most of the fellows who encountered this challenge ended up creating (and completing) smaller modules with the intent to build on additional sections in the future. We also feel that more guidance and more work upfront supporting fellows to plan a reasonable scope for their modules might have helped more fellows to complete their modules within 6 months. For Cohorts 1 and 2, no specific instructions were given to the fellows on the expected length or scope of the modules, other than an overall guidance on the intended audience and purpose of the modules. Building on the experience of the first hackathon, and given the intense nature of the online hackathon format, the HydroLearn team revised their expectations for the second hackathon (Cohort 3) and communicated to the fellows upfront that the scope of a certain module should be such that it can be covered within 2–3 weeks of class time. While we do not have direct evidence on whether this helped the fellows complete the modules more successfully, we believe that it resulted in a more positive participation experience by the fellows and led to better quality of the modules overall.

Lastly, collaboration between fellows was successful in most cases, but some challenges emerged. At times these challenges were related to time zone differences, which made communication challenging. In other instances, some fellows changed positions after the hackathon and were no longer teaching and thus were unable to support their groups in finishing their module. Some of the groups assigned by the guides did not work out because the fellows had need of different content for the courses they were teaching. If we had been able to host the hackathons in person, rather than virtually, perhaps some of these collaboration issues could have been avoided, as fellows would have been in the same location to build rapport and also to avoid time zone challenges. However, an in-person workshop might have been a barrier to participation for some of our international fellows. In spite of the challenges we faced, the workshop approach seemed to work well, as evidenced by the number of completed modules as well as the learning reported by students (Byrd et al., under review).

Discussion

In order to support student learning, instructors need high-quality curricular materials. However, limited time to develop such materials and little training in the research behind curriculum design means that instructors may need additional support to create these materials. Additionally, the rapid shift to online teaching required by the COVID-19 pandemic forced many instructors to search for online curricular materials. HydroLearn was well-positioned to support faculty with a library of online modules. Additionally, the second cohort of fellows was recruited just at the start of the COVID-19 pandemic and lockdowns, which enabled us to provide fellows with learning experiences around developing online instructional materials in time for their fall 2020 courses. The HydroLearn professional experience model, in particular the HydroLearn hackathons, was successful in supporting fellows to develop high-quality curricular materials. Through the HydroLearn hackathons, we created dedicated time and space for fellows to learn about and enact principles of curriculum design, while supported by guides in engineering, education, and the technical platform. We encourage others interested in creating professional learning experiences for instructors to consider research in this field that supports the use of a workshop model and collaborative teams (Mundry et al., 2000; Loucks-Horsley et al., 2010; Walpole and McKenna, 2015). We also found the peer review process following the hackathons to be key to ensuring the modules deployed on the platform were of high-quality. The barriers to designing high-quality curricular materials (e.g., time, training, funding) need to be surmounted. We encourage others to adopt and adapt our hackathon approach to support instructors and improve the design of online curriculum materials. We offer our openly available HydroLearn module on curriculum design (Gallagher et al., 2019) as a starting point. We are also happy to collaborate with others to share additional details regarding our workshop design, slides, templates, and review documents, among others.

Although the hackathon approach was successful in supporting faculty to develop high-quality learning modules, its long-term sustainability would require resources for supporting key components such as training of participants and external review of the modules developed, both of which were critical in developing high-quality modules. Next steps for this project include working with the hydrology and water resources engineering community to scale up this model of professional learning experiences for instructors, and extending this model to include doctoral students in the field. We hope that by working in collaboration with the broader community we are able to establish a sustainable model for professional learning experiences and for the continued development of modules to meet the ever-changing needs of the field.

Data Availability Statement

The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author/s.

Author Contributions

MG and EH conceptualized and outlined the manuscript. MG, EH, DW, BL, and JB wrote sections of the manuscript. MG, EH, BL, and DT contributed to manuscript revision. JB aided with formatting. All authors contributed to the article and approved the submitted version.

Funding

The HydroLearn project was supported by the National Science Foundation (NSF) under collaborative awards DUE IUSE 1726965 and 1725989. Previous NSF awards, including I-Corps 1644493, TUES 1122898, and CCLI 0737073 provided support for the earlier research on HydroLearn.

Author Disclaimer

Any opinions, findings, conclusions, or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the NSF.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

We acknowledge and appreciate the efforts of all the HydroLearn fellows who participated in the hackathon and authored modules.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/feduc.2022.890379/full#supplementary-material

Footnotes

1. ^ www.hydrolearn.org

References

American Educational Research Association [AERA] (2005). Teaching teachers: professional development to improve student achievement. Res. Points 3, 1–4.

Google Scholar

Biggs, J. (2014). ‘Constructive alignment in university teaching’. HERDSA Rev. High. Educ. 1, 5–22.

Google Scholar

Biggs, J., and Tang, C. (2011). Teaching for Quality Learning at University, 4th Edn. New York: McGraw-Hill Education.

Google Scholar

Borrego, M., Froyd, J. E., and Hall, T. S. (2010). Diffusion of engineering education innovations: a survey of awareness and adoption rates in U.S. engineering departments. J. Eng. Educ. 99, 185–207. doi: 10.1002/j.2168-9830.2010.tb01056.x

CrossRef Full Text | Google Scholar

Boston, M. D., and Smith, M. S. (2009). ‘Transforming secondary mathematics teaching: increasing the cognitive demands of instructional tasks used in teachers’ classrooms’. J. Res. Math. Educ. 40, 119–156.

Google Scholar

Bourrie, D. M., Jones-Farmer, L. A., and Sankar, C. S. (2016). ‘Growing the intention to adopt educational innovations: an empirical study’. Knowl. Manag. E Learn 8, 22–38. doi: 10.34105/j.kmel.2016.08.003

CrossRef Full Text | Google Scholar

Byrd, J., Gallagher, M., and Habib, E. (under review). ‘Assessments of students’ learning gains in HydroLearn online modules for teaching hydrology and water resources’. Front. Educ.

Google Scholar

DeChenne, S. E., Enochs, L. G., and Needham, M. (2012). Science, technology, engineering, and mathematics graduate teaching assistants teaching self-efficacy. J. Scholarship Teach. Learn. 12, 102–123.

Google Scholar

Desimone, L. M. (2011). ‘A primer on effective professional development’. Phi Delta Kappan 92, 68–71. doi: 10.1177/003172171109200616

CrossRef Full Text | Google Scholar

Gallagher, M. A., Byrd, J., LaHaye, O., and Habib, E. (2019). Using Hyrdolearn, why and how? HydroLearn. Available online at: https://edx.hydrolearn.org/courses/course-v1:HydroLearn+HydroLearn101+2019_S2/about

Google Scholar

Garet, M. S., Porter, A. C., Desimone, L., Birman, B. F., and Yoon, K. S. (2001). ‘What makes professional development effective? Results from a national sample of teachers’. Am. Educ. Res. J. 38, 915–945. doi: 10.3102/00028312038004915

CrossRef Full Text | Google Scholar

Habib, E., and Deshotel, M. (2018). ‘Towards broader adoption of educational innovations in undergraduate water resources engineering: views from academia and industry.’. J. Contemp. Water Res. Educ. 164, 41–54. doi: 10.1111/j.1936-704X.2018.03283.x

CrossRef Full Text | Google Scholar

Habib, E., Deshotel, M., Lai, G., and Miller, R. (2019). ‘Student perceptions of an active learning module to enhance data and modeling skills in undergraduate water resources engineering education.’. Int. J. Eng. Educ. 35, 1353–1365.

Google Scholar

Henderson, C., Cole, R., Froyd, J., Friedrichsen, D., Khatri, R., and Stanford, C. (2015). ‘Designing educational innovations for sustained adoption: A how-to guide for education developers who want to increase the impact of their work. Kalamazoo, MI: Increase the Impact.

Google Scholar

Herrington, J. A., Oliver, R. G., and Reeves, T. (2003). ‘Patterns of engagement in authentic online learning environments’. Australas. J. Educ. Technol. 19, 59–71. doi: 10.14742/ajet.1701

CrossRef Full Text | Google Scholar

Hollowell, G. P., Brooks, R. M., and Anderson, Y. B. (2017). ‘Course design, quality matters training, and student outcomes.’. Am. J. Dis. Educ. 31, 207–216. doi: 10.1080/08923647.2017.1301144

CrossRef Full Text | Google Scholar

Johnson, N., Veletsianos, G., and Seaman, J. (2020). ‘U.S. faculty and administrators’ experiences and approaches in the early weeks of the COVID-19 pandemic.’. Online Learn. 24, 6–21. doi: 10.24059/olj.v24i2.2285

PubMed Abstract | CrossRef Full Text | Google Scholar

Krathwohl, D. R. (2002). ‘A revision of Bloom’s Taxonomy: an overview’. Theory Pract. 41, 212–218. doi: 10.1207/s15430421tip4104_2

CrossRef Full Text | Google Scholar

Kumar, S., Martin, F., Budhrani, K., and Ritzhaupt, A. (2019). ‘Award-winning instructors online teaching practices: elements of award-winning courses’. Online Learn. J. 23, 160–180. doi: 10.24059/olj.v23i4.2077

PubMed Abstract | CrossRef Full Text | Google Scholar

Lane, B., Garousi-Nejad, I., Gallagher, M. A., Tarboton, D. G., and Habib, E. (2021). ‘An open web-based module developed to advance data-driven hydrologic process learning.’. Hydrol. Process. 35:e14273. doi: 10.1002/hyp.14273

CrossRef Full Text | Google Scholar

Loheide, S. P. II (2020). ‘Collaborative graduate student training in a virtual world. Eos 101. doi: 10.1029/2020EO152183 [Epub ahead of print].

CrossRef Full Text | Google Scholar

Loucks-Horsley, S., Stiles, K. E., Mundtry, S., Love, N., and Hewson, P. W. (2010). Designing professional development for teachers of science and mathematics. Thousand Oaks, CA: Corwin press. doi: 10.4135/9781452219103

CrossRef Full Text | Google Scholar

Maggioni, V., Girotto, M., Habib, E., and Gallagher, M. A. (2020). ‘Building an online learning module for satellite remote sensing applications in hydrologic science.’. Remote Sens. 12:3009. doi: 10.3390/rs12183009

CrossRef Full Text | Google Scholar

Mok, H. N. (2014). ‘Teaching tip: the flipped classroom’. J. Inf. Syst. Educ. 25, 7–11. doi: 10.3138/jvme-2021-0043

PubMed Abstract | CrossRef Full Text | Google Scholar

Mundry, S., Britton, E., Loucks-Horsley, S., and Raizen, S. A. (2000). Designing successful professional meetings and conferences in education: Planning, implementation, and evaluation. Thousand Oaks: Corwin press.

Google Scholar

Polebitski, A., and Smith, T. (2020). Introduction to floodplain analysis. HydroLearn. Available online at: https://edx.hydrolearn.org/courses/course-v1:UW_Platteville+CEE4300+F2020/about

Google Scholar

Roundy, J. K., Gallagher, M. A., and Byrd, J. L. (under review). An innovative active learning module on snow and climate modeling. Front. Educ.

Google Scholar

Ruddell, B. L., and Wagener, T. (2015). ‘Grand challenges for hydrology education in the 21st century. J. Irrig. Drain. Eng. 20:A4014001. doi: 10.1061/(ASCE)HE.1943-5584.0000956

PubMed Abstract | CrossRef Full Text | Google Scholar

Shekhar, P., and Borrego, M. (2016). ‘After the workshop: a case study of post-workshop implementation of active learning in an electrical engineering course.’. IEEE Trans. Educ. 60, 1–7. doi: 10.1109/te.2016.2562611

CrossRef Full Text | Google Scholar

Stein, M. K., and Lane, S. (1996). ‘Instructional tasks and the development of student capacity to think and reason: an analysis of the relationship between teaching and learning in a reform mathematics project’. Int. J. Phytoremediation 21, 50–80. doi: 10.1080/1380361960020103

CrossRef Full Text | Google Scholar

Tekkumru-Kisa, M., Stein, M. K., and Schunn, C. (2015). ‘A Framework for Analyzing Cognitive Demand and Content-Practices Integration: Task Analysis Guide in Science’. Wiley Periodicals Inc. J Res Sci Teach 52, 659–685. doi: 10.1002/tea.21208

CrossRef Full Text | Google Scholar

Walpole, S., and McKenna, M. C. (2015). Best practices in professional development gor improving literacy instruction in schools, Best practices in literacy instruction. New York: Guilford Press.

Google Scholar

Wiggins, G., and McTighe, J. (2005). Understanding by Design. Alexandria, VA: Association for Supervision and Curriculum Development.

Google Scholar