Designing for Trust-Building Interactions in the Initiation of a Networked Improvement Community

Kallio, Julie M.; Halverson, Richard R.

doi:10.3389/feduc.2019.00154

ORIGINAL RESEARCH article

Front. Educ., 14 January 2020

Sec. Leadership in Education

Volume 4 - 2019 | https://doi.org/10.3389/feduc.2019.00154

This article is part of the Research Topic Social Innovation in Education View all 11 articles

Designing for Trust-Building Interactions in the Initiation of a Networked Improvement Community

$\nJulie M. Kallio$ Julie M. Kallio^*

Richard R. Halverson

Department of Educational Leadership and Policy Analysis, University of Wisconsin–Madison, Madison, WI, United States

Networked Improvement Community (NICs) are increasingly recognized as a social innovation for orchestrating sustained change in education. NICs are one type of a research-practice partnership that provides a model for researchers and educators to bring insights about what works locally to scale. A critical aspect of NIC success is the emergence of relational trust across the participant network. At initiation, therefore, NIC leaders must create the conditions for long-term development of relational trust, which can be operationalized to be the existence of reciprocated, help-based interactions. To understand how NIC leaders foster these reciprocated, help-based interactions, this paper leverages social network and qualitative data to explore how the core activities of a NIC might foster help-based interactions amongst participants. This paper is a case study of how social network and qualitative data analysis might be applied to the design and development of NICs, and social innovation more broadly. We apply social network and qualitative data analysis in the context of the Personalization in Practice-Networked Improvement Community, which brought together 21 educators from five schools around a common challenge. Focusing on the initial activities that took place over 3 months, we use social network analysis to connect the patterns and progressions of interactions with design activities and qualitative data to examine the quality of those interactions. Our paper highlights how collaborative design activities created the three conditions for relational trust to emerge: sparking interactions around shared practices, creating situations for participants to ask for help, and encouraging reciprocated, help-based interactions. The application of social network and qualitative data allows us to capture (1) the creation of meaningful ties amongst educators across schools and strengthening of ties between same-school colleagues, and (2) instances of reciprocated, help-based researcher-educator and educator-educator interactions. These findings demonstrate how specific collaborative design activities can foster the kinds of trust-building networks necessary for NIC success. This paper presents an applied case of using analytic research methods for the design of social innovation. The triangulation of social network and qualitative data provided insight into the internal dynamics of the partnership and has implications for development measures of network health. We found that the social network data described that interaction changed, but did not indicate which activities led to these changes. Triangulation with qualitative data was necessary to understand the quality of the interactions that were possible as the social network emerged. This case contributes to emerging research on how to measure the effects Networked Improvement Communities on participants and their practices. In doing so, we demonstrate, on a practical level, how social network and qualitative data might be used to generate network-level data for improvement, and we contribute theoretical insight into the way collaborative design creates the conditions for the long-term development of relational trust.

Introduction

Solving the complex problems of educational systems requires rethinking how researchers and educators work together. For many years, education researchers and policy makers devised technical innovations to improve learning at scale, and formulated processes to ensure the appropriate implementation of these programs (Slavin, 2002). In recent years, however, the education policy and research community has come to realize the promise of social innovations, particularly through including educators in the change process (Cohen-Vogel et al., 2015). This realization is seen in the emergence of research-practice partnerships as a promising pathway to engage in systems-level change (Coburn and Stein, 2010).

One type of research-practice partnership is the Networked Improvement Community (NIC). NICs are a social reorganization of traditional research and development activities that leverage data-informed, collective action for social innovation (Bryk et al., 2011). When the first NICs achieved outsized success, NICs became an increasingly popular model for reform (Bryk et al., 2015). Toward this end, research on models for NIC initiation (Russell et al., 2017) and execution (LeMahieu et al., 2017) have focused primarily on identifying the organizational structures, methods, and tools that support NIC progress. What is less clear is how NICs foster the social capacities, such as relational trust, that are needed for sustained reform.

This paper focuses on designing for interactions that build social capacities for sustained reform. Relational trust describes the capacity for successful, professional interaction (Bryk and Schneider, 2002) and has been promoted as an indicator of research-practice partnership effectiveness (Henrick et al., 2017). Relational trust springs from recurrent, reciprocal help-based interactions that, over time, build communities of practice where participants can take risks together and experiment with new practices. If relational trust is a key capacity for long-term change, then how can a NIC act as a catalyst to move practitioners toward the kinds of interactions that can spark relational trust?

To this end, this paper answers the question, how do NIC collaborative design activities foster reciprocated, help-based interactions? Focusing on the first 3 months of NIC initiation, we illustrate how the collaborative design activities sparked help-based interactions among NIC participants. The paper begins with a review of collaborative design, relational trust, and social network theory. We then provide a narrative of the early stages of PiPNIC, a NIC with educators around the challenges of implementing personalized learning. Next, we detail the applied research design. The findings trace (1) the creation of meaningful ties amongst educators across schools and strengthening of ties between same-school colleagues, and (2) instances of reciprocated, help-based researcher-educator, and educator-educator interactions. The paper concludes with an exploration of how the findings from this analysis inform an understanding of designing NIC initiation and the use of social network and qualitative data to inform the development of social innovation efforts.

Collaborative Design

Collaborative design¹ is a user-centered problem-solving approach that emphasizes the inclusion of users in both what problems to solve and how they will be solved (Schuler and Namioka, 1993; Muller, 2003). NICs appropriate many collaborative design ideas, such as problem-identification, iterative testing, and reflection cycles (Bryk and Gomez, 2007). NICs begin with identifying a common problem of practice important to the educator participants and leverage collaborative design as a core interaction mechanism (Dolle et al., 2013). Through collaborative design, a NIC invites practitioners to examine how problems occur in local contexts and identify measurable goals, develop robust data pathways to iteratively inform design process and outcomes, build and test solutions, and create a theory of action that reveals the problem and possible solution paths (LeMahieu et al., 2017).

Successful collaborative design requires help from other people. In a successful NIC, problems are solved when researchers ask for design help from educators, and educators are open to research precedents and design options. The perspectives of researchers and practitioners are then integrated into the collaborative design of an artifact that addresses the problem of practice. As a result, participants feel mutual ownership over the process and product, and recognize why each kind of expertise included in the design was necessary for the resulting solution.

While the design process is aimed at creating a useful solution to a shared problem, collaborative design research also focuses on how interaction is coordinated to support authentic participation (Ehn, 2008). The connection between the collaborative design activities and interactions is labeled infrastructuring (Penuel, 2019). Infrastructuring describes the “network of tools, relationships, standards, and protocols on which an individual or group relies to carry out day to day tasks and accomplish particular goals” (Penuel, 2015, p. 5). From a social perspective, NIC initiation is infrastructuring, where the collaborative design activities foster the kinds of social capacities that will support long-term partnership success.

Relational Trust

Relational trust is a specific form of social capacity that has been studied extensively in organizational theory (Mayer et al., 1995) and education (Tschannen-Moran, 2018) and associated with successful school reform (Bryk and Schneider, 2002). Relational trust is a critical resource for solving organizational problems as it supports asking and answering hard questions, risk taking, and the collaborative vetting of proposed solutions (Levin and Cross, 2004).

Bryk and Schneider (2002) defined the concept of “relational trust” as a form of social trust² that is built through the interactions amongst educators over time within a community. Relational trust develops between two individuals when they ask each other for help, and the bid for help is fulfilled by the other, repeatedly, over time. When people ask for and receive help from one another across an organization (or a partnership), trust networks begin to form that can support participants to engage in tasks that require more risk (Mayer et al., 1995; Halverson and Kelley, 2017). Distributed relational trust emerges when there are redundant, reciprocal trust-networks develop in an organization around key professional tasks. Tracing the development of a network of reciprocated, help-based interactions across the participant community operationalizes the conditions for relational trust to emerge in the long term.

The role of trust in building successful research-practice partnerships is well-known. Henrick et al. (2017), for example, propose five indicators under the category of “building trust and cultivating relationships”: researchers and practitioners (1) routinely work together, (2) establish routines that promote collaborative decision making and guard against power imbalances, (3) establish norms of interaction that support collaborative decision making and equitable participation in all phases of the work, (4) recognize and respect one another's perspectives and diverse forms of expertise, and (5) decide partnership goals that take into account team members' work demands and roles in their respective organizations. Identifying these indicators is an important feature of partnership research. The next step in partnership research is then to identify the kinds of activities that promote collaborative decision-making and the strategies leaders use to build capacity for participants to recognize and respect one another's perspectives. This paper is aimed making this connection between NIC activities and the patterns and progressions of interactions that might yield these indicators of long-term development of trust.

Social Network Theory

The patterns and progressions of interactions during NIC initiation can be traced by social network analysis tools, which are built from social network theory. Social network theory provides a method to track the patterns and progressions of interactions amongst members of a group (Daly, 2010). Social network theory foregrounds the overall structure of the group, the ties between actors, and the quality of the ties as important factors in understanding actor and network outcomes (Lin, 2002; Borgatti et al., 2014).

Networks and trust have been studied extensively, with higher network density associated with greater network cohesion, trust, and capacity for change (Mohrman et al., 2003; Moolenaar and Sleegers, 2010). Strong ties are correlated with benevolence-based trust (Currall and Judge, 1995) and with trust and trustworthiness (Glaeser et al., 2000) (see Table 1 for a summary). Trust is typically examined for its impact on interactions, rather than as an outcome of interactions (Liou and Daly, 2014).

TABLE 1

Table 1. Theoretical connection between help-based interactions, the facet of trust they align with, and related research.

In education, social network theory has been used to explicate how ties among network members evolve during reform efforts (Daly and Finnigan, 2010), mediate professional learning (Penuel et al., 2012), and support principal innovation (Moolenaar and Sleegers, 2010). Recent work to apply social network methods to NICs has examined how knowledge is transferred across the network (Cannata et al., 2017a,b) and how organizational positions affect NIC participation (Sherer and Feldstein, 2018).

Social network theory allows for collecting data on the shape and intensity of the network of help-based interactions that characterize a successful NIC. NIC initiation typically brings together people who have limited prior connections. Limited prior connections is part of the NIC strategy to learn from implementations across contexts of practice. The lack of relational trust between people who have few shared connections could limit participation in NIC activities. In this scenario, interaction may be centralized among the small number of actors who had prior connections, which could result in a one-way network where certain actors provided help to a large number of participants. Even though the NIC may be organized around shared design activities, the patterns of help-based interactions may come to be unevenly distributed across participants. The application of social network and qualitative analytic tools to examine the network as it is forming can be used to provide insight into the patterns and progressions of emerging connections among participants.

In the next section we provide a narrative of the larger research and practice context for the NIC presented in this paper.

A Networked Improvement Community Around Personalized Learning

The Personalization in Practice (PiP) research group was formed in 2014 to study school-wide efforts to design and implement personalized learning strategies in K-12 schools (Halverson et al., 2015). PiP is a research alliance between the University of Wisconsin-Madison School of Education, the CESA 1 Institute for Personalized Learning (IPL), and the Wisconsin Department of Public Instruction. The partnership is supported by the Joyce Foundation and by the US Department of Education Institute of Education Sciences. The goal of PiP was to document how public schools engage in personalized learning, then transform these insights into opportunities for professional learning for interested educators through a NIC.

There is a strong and growing network of personalized learning schools in Wisconsin, and IPL has been a regional and national leader in designing, supporting, and scaling this grass-roots movement since 2009 (CESA1, 2011). IPL defines personalized learning as,

an approach to learning and instruction that is designed around individual learner readiness, strengths, needs and interests. Learners are active participants in setting goals, planning learning paths, tracking progress and determining how learning will be demonstrated.

The PiP research team conducted ethnographic studies of 20 IPL schools engaged in personalized learning. They identified three key personalized learning practices:

• Educators designed cultures of agency to engage students as active participants in their learning;

• Educators acted as facilitators of learning by regular conferring with students to construct learning pathways and set learning goals; and

• Schools developed socio-technical ecologies of digital tools, such as productivity tools, learning management systems, computer adaptive testing and curriculum tools, and digital media and design tools to coordinate instructional, assessment and learning tasks (Halverson et al., 2015).

In the fall of 2016, the Personalization in Practice-Networked Improvement Community (PiPNIC) was launched to bring together expert educators to identify, document, and improve core personalized learning practices. PiPNIC sought to engage expert practitioners and researchers in collaborative design around common, meaningful problems of practice that would produce practical and theoretical knowledge about cutting-edge personalized learning practices. The PiPNIC theory of action was that engaged practitioners and researchers in collaborative design would spark help-based interactions, which would in turn generate solutions to the problem of practice and develop the capacity to support further improvement.

A network hub was established, led by the two authors of this paper, and the formation of a Networked Improvement Community was conceptualized as three stages: problem identification, participant recruitment, and a 90-day collaborative design cycle³.

Stage 1: Problem Identification

This stage involved contacting and interviewing schools across the state to identify shared problems of practice in personalized learning. Identifying a problem of practice from the field established the interdependence and authenticity of the partnership from the start as researchers would have to rely on the practice-based knowledge of educators, while educators would have to rely on the researchers to structure the common inquiry. To do this, our network initiation team drew on Gawande's (2008) idea of listening to those closest and most knowledge about the work in order to identify meaningful insights and challenges. Through phone calls, visits, and discussions at conferences, over 60 educators were consulted from traditional public, charter and private school communities. Schools were nominated through the PiP researchers' existing connections for their expertise in personalized learning. In this way, the research team leveraged its existing social capital to better understand the challenges faced by personalized learning educators and leaders.

We identified conferring as central, shared problem of practice on the frontier of personalized learning. When personalized learning educators ranked their practices, conferring emerged as the practice that had the highest utility for their work (Rutledge, 2017). Conferring came to be defined as the regular one-on-one conversations between an educator and student. For example, in a project-based learning school, a student might sign up to meet with their teacher to discuss their ideas for an interest-based project. In a competency-based school, a student and teacher might meet to look at their progress on a computer adaptive platform and discuss next steps. Educators from different contexts described the purpose of conferring as developing learning relationships, individualizing learning, and/or capturing evidence about learning. While the instructional origins of conferring are found in other pedagogies (e.g., Calkins and Harwayne, 1991), it also roots in the individualized educational program meetings of special education. Despite near universal agreement that conferring was a core practice, teachers reported little consensus on process documentation or evidence of impact. This made conferring an excellent problem of practice for NIC design.

Stage 2: Participant Recruitment

The Stage 1 listening effort resulted in a long list of potential NIC educator participants. We narrowed this list by identifying practitioners who had well-established conferring practices, district leadership support, and the capacity to engage in the project. We also sought to recruit educators from a range of schools in terms of student age (kindergarten through twelfth grade) and context (school size, locale, age of program). We ultimately invited 21 educators from five K-12 public schools based on their expressed expertise in conferring and their willingness to spend four Saturdays over the course of 3 months working on a collaborative design project. Each participant received a stipend and the option of continuing education credit. Ten UW researchers agreed to help coordinate the collaborative design process.

Stage 3: 90-Day Design Cycle

In the spring of 2017, 10 UW researchers, including the authors, and 21 educators from five schools (see Appendix B for school descriptions) came together to participate in collaborative design activities. The collaborative design activities were organized by the 90-day design cycle (Park and Takahashi, 2013), a way to prototype an innovation through leveraging knowledge of those within and outside of the field associated with the topic; coordinating the development and “testing” of a product by at least one of several means; begin and conclude within a span of 90 days … [and] deliver needed knowledge in a timely fashion (Park and Takahashi, 2013, p. 6–7). The 90-day cycle also provided a strategy to synchronize the work of participants from different organizations, a key challenge of RPP (Coburn and Penuel, 2016). The outcome of the design task was to develop and validate conferring protocols that could be used across the schools. Each school team that participated ultimately produced a protocol to guide their local conferring practice, and all protocols were published in a final 90-day report (Kallio and Halverson, 2017). Meetings were held on four Saturdays, hosted by participant schools.

The research team developed a series of five key activities to engage participants in collaborative design. Each school team of educators:

1. Created videos of their own practices to share current conferring strategies;

2. Pitched a plan for the improvement of conferring to the whole group;

3. Developed protocols that described the context, conversation, and documentation strategies for their desired conferring practices;

4. Engaged in a user-testing cycle where each educator tried out their school's protocol and experimented with protocols from other schools; and

5. Contributed to a final report and community discussion where experts in personalized learning and student-focused instruction commented on the presentation of new practices.

To facilitate these activities, each PiP research team member was assigned as a liaison, or “site captain,” as primary points of contact for each school. PiP site captains visited participant schools prior to and throughout to support design activities.

The NIC collaborative design activities were held during the Saturday whole-group meetings. Each activity included (1) sharing activities in mixed-school groups and (2) reflecting, planning, and testing activities within same-school teams (see Table 2 for a summary).

TABLE 2

Table 2. Collaborative design activity sequence during the 90-day design cycle.

In sharing activities, participants presented representations of their conferring practice or plans for improvement to participants from other schools. The goal of these structured sharing activities with mixed-school groups was (1) to maximize opportunities for each participant to ask for and receive help from educators from other schools and to minimize the possibility of one person serving as the de facto spokesperson for the school, and (2) to require participants to take a risk in sharing their practice publicly and allowing others to comment on it. These kinds of sharing activities were repeated each week as school teams refined their representations of conferring, created and executed improvement plans that incorporated feedback, and ultimately user-tested each other's conferring protocols. This repetition built a history of interactions between educators from different schools and with researchers.

In reflecting, planning, and testing activities, participants from the same school plus their site captain had time to make sense of their feedback. Participants then negotiated how they would incorporate individually-received feedback into future collective action. The repetition of these activities also provided opportunities for same school colleagues to build a history of interactions with each other and with their site captain.

The conferring protocols that resulted from the NIC process have since been shared through the state Department of Public Instruction, as well as presented by participants and researchers at state-wide educator conferences. There has also been follow up between researchers and schools as part of a continuation of the Personalization in Practice study, specifically focused on the implementation of the conferring protocols and the instructional systems that support it. The remainder of the paper focuses on the interaction structures and emerging network in Stage 3.

Research Design

This study uses a mixed methods approach that draws on social network and qualitative data to answer our qualitative research question (Plano Clark and Ivankova, 2016): how do NIC collaborative design activities foster reciprocated, help-based interactions? We collected social network and qualitative data to determine the patterns and progressions of these interactions, and connect these to the design of the initiating activities of the NIC. We operationalized relational trust with a survey that allowed participants to identify who they valued in the NIC process, then explored key design tasks for evidence of reciprocated, help-based interactions. While all members of the research team engaged in data collection, the two authors of this paper were primarily responsible for both the design of the NIC activities and the collection of data.

Social Network Data Collection and Analysis

The research team developed a social network survey to collect data about participant interaction. The survey was given at the conclusion of the 90-day cycle, to all participants (n = 31, 21 educators and 10 researchers), built and delivered through Qualtrics (Appendix A). To ensure 100% response rate, participants were given time during the final meeting to complete the survey, and the network coordinator verified that each person had submitted it before the meeting concluded.

Participants indicated who they had interacted with about conferring prior to participation and who they interacted with during. We used a roster with the names of all the participants as a feature of the survey. They then rated how important that person was to their “learning about conferring.” The question was phrased as “learning” to capture the range of interactions across the different design activities but rooted in collaborative and professional interactions. Because relational trust is a condition for learning with someone in the context of a professional learning community (Louis, 2006) and other studies have used tie strength as a proxy for trust (Gulati, 1994), we interpreted higher “important to my learning” as a valued interaction.

In the analysis of social network data, we use whole network and dyadic characteristics (Borgatti et al., 2013). Responses from the social network survey⁴ were imported into Excel, anonymized, and uploaded into UCINET (Borgatti et al., 2002). Operationalizing what the tie represents is critical for interpretation. The ties we capture indicated who an individual interacted with and how that interaction was valued. This provides insight into the distribution and quality of ties across the network and situates qualitative observations, providing evidence about the ways in which participants relied on one another for help in the design activities.

Qualitative Data Collection and Analysis

We also collected data to capture the emergence of help-based interactions in the collaborative design activities. A number of written documents served as data sources, including the initial grant application, an advisory committee report, a research group presentation, meeting agendas, and a final report. During the listening sessions, the research team had a Google form that team members filled out with the answers from their calls. We collected planning documents and agendas, with notations for how meetings were modified in the moment, as well as email communications, internal and external presentations. Observation data was recorded by research team members as they participated in and/or facilitated activities related to the project, including notes and participant reflections from all collaborative design meetings.

We also collected direct feedback from participants. We asked participants for feedback about the activities, their thinking about conferring and personalized learning, and what each individual was interested to improve. After the second meeting, participants emailed one thing that they found to be the most meaningful during the morning's activities. On the last Saturday, we had teams do a focus group debrief of the user testing process and reflect on their progress up until then. The audio from these focus groups was transcribed.

Finally, we used the qualitative data to write detailed design narratives for each site and the NIC as a whole. The site narratives were semi-structured, describing each school's context and participating team members, what conferring and personalized learning looked like prior to PiPNIC, then a chronology of actions and design moves during the 90-day cycle. The network narrative focused on the meta-design of the processes that brought this work together. At a research meeting, these narratives were cross-checked with other members of the research team. The narratives were also presented with participating educators at a conference.

The analysis in this paper began with the social network then qualitative data. The social network analysis was used to identify patterns in interactions, then the qualitative data provided the quality of the interactions. For example, when the network analysis showed that participation in PiPNIC created interactions between participants from different schools, we turned to the qualitative data to understand the progression and function of those interactions. This led us to identify a pattern of authentic requests for help between educators and researchers and amongst educators. Our attention was drawn to examples of participant interaction that could illustrate the kinds of help-based interactions indicated by the social network analyses. In our discussion (below), we consider the explanatory relation between the activities and the social network data as part of a larger argument about designing for relational trust.

In this way, the social network data provided selection criteria for the qualitative data, though not all of the qualitative data demonstrated the existence of help-based interaction. Many of the fieldnotes, for example, described information presentation activities, or documented participants involved in discussions or non-project related interactions. Our attention was drawn to examples of participant interaction that could illustrate the kinds of help-based interactions indicated by the social network analyses. In our discussion (below), we consider the explanatory relation between the activities and the social network data as part of a larger argument about designing for relational trust.

Because of the focus of this case on a context that includes educators as partners in social innovation, we constrain the analysis in this paper to educator-researcher and educator-educator interactions. While the researchers had some previous connections with each other, their pattern of interactions would have been confounded by other meetings beyond the collaborative design activities. Future analyses could examine how the ties between the researchers changed, but that is beyond the scope of this case.

Findings

Our findings focus on identifying the emergence of help-based interactions through the PiPNIC collaborative design activities, then on describing the conditions that sparked these interactions. We present three kinds of collaborative design activities that produced the observed help-based interactions: (1) activities that created meaningful cross-school connections among educators and activities that strengthened ties between same-school colleagues and between researchers and educators; and (2) activities to build reciprocal interactions across the network.

Meaningful Cross-School Connections

The first kind of activity we highlight is the process of bringing together educators from different schools and school contexts interested in shared problems of practice to spark new professional interactions. Bringing together a group of weakly-connected educators and researchers can be a source of new ideas and resources for the NIC (Granovetter, 1973; Johnson, 2011), and it is integral to the process of finding what works, for whom, and under what conditions (Bryk et al., 2011). However, the lack of existing ties between educators from different schools and the differences in their school contexts can also produce barriers to collaborative problem solving if they perceive that other people's practices would not apply⁵.

In PiPNIC, recruitment began with researchers who had existing ties with potential partners. Recruitment via existing ties has implications for the initial structure of the relational network, how new ties might be created, and the function of these new ties in terms of building the relational resources for innovation. Once a school indicated they were interested in participating, the school leader and the PiP team worked together to invite other educators from the school. Asking the school leader to identify someone meant that the educator might have a strong connection to their own school colleagues, but probably would not have a direct connection to the research team. This created a situation where one person from each school was connected to the PiP team, but most of the educators did not have prior connections with the PiP researchers or with other school educators.

Our social network data reflected this initial condition. The social network survey asked participants to identify who they knew prior to PiPNIC⁶. Ties prior to participation show a weakly-connected researcher-practitioner network with six subgroups: the five school teams and the research team (Figure 1). Researchers were central and densely connected to each other with ties to a few educators, but educators were not connected to each other (with a few exceptions). The ties between researchers and educators are characterized as weak because the average importance rating of the research team prior was 2.83/5 whereas the average importance rating of educators to their own colleagues at each school was 3.64/5 (Table 3).

FIGURE 1

Figure 1. This graph shows the network prior to the 90-day cycle. Participants answered the survey question, “Who did you know prior to PiPNIC?” The graph, constructed in UCINET, includes all participants (n = 31). The non-circle nodes are educators, circles are researchers, and the shape indicates organizational affiliation. More highly connected nodes are moved to the center, while people with fewer connections are moved to the periphery.

TABLE 3

Table 3. This table shows the average importance rating for different types of ties.

A weakly-connected researcher-practitioner network is an important initiating condition for many NICs. The research team recognized we would need to initiate meaningful interactions across school groups early in the process by creating stable mixed-groups of educators from across schools. We intentionally created stable mixed-groups so that educators would come to know one another and have opportunities to consult each other over time.

The first mixed-group activity was sharing videos of their current conferring practice. Prior to the first Saturday meeting, PiP site captains had met with educators at each school to video the existing local classroom conferring practice of most teachers. Site captains worked with educators to shoot and edit the video to present at the first group meeting. Then on the first Saturday, each educator in the mixed-group shared their video of the school's conferring practice. A PiP Site Captain facilitated the discussion process. This activity allowed each educator to see how their conferring expertise compared with the other school's educators. Each educator could ask about the practices of the other schools and serve as an expert for the practices represented in their school videos.

Researchers noticed that this first look into each other's practices sparked discussion about the similarities and opportunities they saw in how conferring happened in other schools, rather than focusing on the differences in their contexts. One participant observed how remarkably similar the goals of conferring were, despite the variation in the implementation, age levels, pedagogical models, or student populations.

These mixed groups met again on the second Saturday where educators “pitched” their plans to improve their conferring practices, again facilitated by a researcher. Participation in these activities meant they needed to interact with participants from other schools to complete the design task, and next we show how we know these interactions were meaningful.

At the end of the 90-day NIC design activities, we saw changes in number of meaningful connections that each participant reported with others. The density of the network⁷ increased from 0.28 to 0.44. This increase in density indicates the creation of new ties amongst the same number of nodes. The structure of the resulting network showed a distributed network of interactions (Figure 2). Educators became more centrally located in the network, whereas most of the PiP researchers had moved to the periphery, which confirmed our intention to design activities that would foster help-based interactions amongst participating educators. The network graph illustrates how educators were creating new connections with educators from other schools and with researchers. This creation of interactions across school groups reflects research on how networks can close structural holes and create access pathways to the ideas and resources of the other actors (Burt, 2017).

FIGURE 2

Figure 2. This graph shows the network during to the 90-day cycle. Participants answered the survey question, “During PiPNIC, who was important to your learning about conferring?” The graph, constructed in UCINET, includes all participants (n = 31). The circles are researchers, non-circles are educators, and the shape indicates the different schools. More highly connected nodes are moved to the center, while people with fewer connections are moved to the periphery.

We also observed that the new ties that were created were valued by participants. Half (12/21) of the educators indicated that “interactions with innovative educators from other schools” was the most important aspect of their participation. Participating educators shared the following reflections on their most meaningful moments:

• “The process of sharing with others who have similar missions but very different contexts and different practices around a common question has been even more powerful and effective than I imagined it would be. I have greatly enjoyed this experience and feel as if I am gaining a lot.”

• “I think the most meaningful part of the PiPNIC was when we were partnered with one person from a different school to share our protocol and talk through it with them so that they could try it in their context.”

• “Meeting with different age levels teachers to see the connection with what is happening in my classroom”

• “The most meaningful part of the PiPNIC process was the interaction and conversations amongst the different schools.”

• “The connections, stories and experiences shared from other professionals. The honest and open vulnerability everyone had through the process.”

Every educator reported meaningful interactions with at least three educators from other schools. The number of meaningful interactions that each person receives is called in-degree centrality, and is often used as a measure of status in a network (Siciliano, 2016). This is because a person with more nominations from others has an influential position (Moolenaar, 2012). We considered a rating of 3 or higher (5 being the highest possible) for the question, “During PiPNIC, how important was this person to your learning about conferring?” That each person developed meaningful connections with educators from other schools is significant in establishing the conditions that support help-based interactions, as similar levels of status support the development of relational trust (Tschannen-Moran and Hoy, 2000).

Strengthening Ties Between Same-School Colleagues and Between Researchers and Educators

Sparking new ties among new colleagues is an initial challenge for the NIC process, but strong ties within schools, and with network initiators, are needed for innovation to take root in practice (Coburn and Russell, 2008). In many ways, it is not surprising that time spent together increased tie strength. A quarter of the educators responded that spending time working with their colleagues and other educators was meaningful. In this part of the analysis, however, we consider how the quality of interactions across the network strengthened ties within schools and ties with researchers.

The NIC leaders designed activities to foster help-based interactions amongst same-school colleagues. Saturdays began with an hour for school teams and their site captains to meet and prepare for the mixed-group session and concluded with an hour for them to prepare for their work going forward. Providing teams with this time was at first a way to reduce the demand for their participation on time outside of NIC activities, but it also gave them designated time to complete collaborative tasks. For example, one school recognized that it had two approaches to conferring. They had to come to a consensus as to which version they would focus on or whether they would try to blend the two. The interactions to complete this task required a willingness to be vulnerable by sharing one's own practice, being open to change based on new information, and acknowledging each other's competence and expertise. The site captains noted that the team ultimately decided to go with one protocol, citing that “they saw themselves as one school.” In this way, this design task forged a sense of shared commitment to this direction and engagement in the collaborative design process.

Other educators shared how important it was to have the time to work with their colleagues, including the drive to and from the meetings. One educator shared that the best part Saturday was “The ability to sit with my team, free of distractions, and have a conversation. Coming up with conferencing/project protocols has been on our to-do list for far too long. Often we are just going full speed and have little time to stop and just do ONE thing. I also appreciate seeing and hearing about what others are doing with their students. The experience is encouraging us to keep moving forward.”

Looking across the network at the change in tie strength, participation in NIC activities strengthened all types of ties and ties across schools increased the most, but same-school ties were the most important (Table 3). The survey asked participants to indicate how important the person was to their learning about conferring. “More important” here indicated that the interaction supported them in the task they were trying to accomplish and suggests that the person was judged as competent in helping the respondent learn. The deepening of ties amongst colleagues is an important indicator that the NIC process sparked meaningful interactions among educators from the same schools.

Because the goal of NICs is to work across research and practice boundaries, the ties between a school group and their site captain provides an examination of a particularly important type of interaction in the context of the NIC. During design activities, site captains were included in the same-school groups, often helping participants clarify of the task or make sense of feedback, as we draw out in more detail later in the paper. All participants indicated that they had meaningful interactions with their site captains. The average importance rating of the tie between the school members and their site captain was 4.4/5, on par with their ratings for their own colleagues. The importance of the educator-researcher relationship and the deepening of ties across all subgroups suggests that the strong ties that were created represent meaningful, distributed interactions where help could be sought and received.

Activities to Build Reciprocal Interactions Across the Network

The social network and qualitative analysis presented provides insight into the patterns of interactions across the network. In the next section, we look at progressions of interactions. We highlight examples from the first NIC activity (sharing the video of current conferring practices in mixed-groups and deciding on a conferring improvement focus) and the fourth NIC activity (user-testing each other's conferring protocols). Though there were other examples of help-based actions, these two examples were the richest in participant perspective and provide examples of what was possible in the context of the activities.

The first example illustrates how researchers and educators began to ask for help from one another in developing representations of conferring practice. As described earlier, prior to the first meeting, PiP site captains had met with their school to help record and edit a representation of how conferring happens in each educator's school. These videos were then discussed at the first meeting's mixed-group sharing session. This session presented a moment of high vulnerability for educators who had just met each other. The interactions in the video, one-on-one conversations between a teacher and student, are perhaps the most intimate part of a teacher's practice. The researchers sought to create a safe space by prompting participants to notice, rather than evaluate, each other's practices. For example, the activity encouraged participants to describe carefully what they saw in each other's practice rather than making suggestions about how practice might be improved.

The educators from Franklin Elementary School recorded a conferring session in which two teachers met with one student to talk about her idea for an interest-based project. After presenting the video in the mixed-school groups, the Franklin educators reconvened with their site captain to make sense of the feedback they had gotten from educators at other schools. They discussed that other educators had noticed a lack of structure in their conferring process and that the teachers did most of the talking. Their site captain wrote down that their main takeaway from this discussion was that “We talk too much. We need to do something that helps students find their voice.” This design task facilitated their ability to get feedback on their practice, then make sense of the feedback with their colleagues. At the conclusion of the first Saturday, the teachers decided to develop a conferring protocol that would structure conferring around interest-based projects, hoping that the structure would provide the scaffolding for students to speak more.

Before the second Saturday NIC meeting, the PiP site captain analyzed the Franklin conferring video and created a representation of the turn-taking between teachers and students (Figure 3). This visualization affirmed what had been observed the previous meeting and prompted continued conversation among the Franklin teachers about student ownership of the learning process. The team agreed that their conferring sessions were too teacher-led, and they decided to focus their conferring improvement efforts on supporting student-led conversations.

FIGURE 3

Figure 3. Visualization of the Franklin conferring process created by the site captain. This figure represents the conferring process of one advisory session with two advisors and one student. In this instance, the student mostly responded to questions from the advisors and the student spoke less often toward the end of the meeting.

Notably, the interaction between the Franklin teachers and the PiP site captain lacked any signs of defensiveness that could limit reciprocal learning (Argyris, 2000). At the end of this second Saturday, one Franklin teacher shared in an email that she was thankful for “having the chance and opportunity to chat with my colleagues and hash out the details of our project process, having [our site captain] organize our thoughts and make sense of them, and creating meaningful project/conferring opportunities for the students we serve.”

This example illustrates the PiPNIC collaborative design strategy. First, participants engaged in activities grounded in their current practices. Then they received feedback from other schools and from their site captain, and reflected on how to integrate feedback into their own protocol design process. Help was requested and received by educators and researchers alike in an effort to design a solution to the problem of practice.

The second example spans the third and fourth NIC Saturdays to show how the collaborative design activity of user-testing sparked authentic, help-based interactions between educators from two different schools. At this point, each school had a rough draft of their conferring protocol. The PiP team developed a user-testing activity for school teams to test and give feedback on each other's protocols. Each participant was paired with an educator from their mixed-school group in order to continue to build on their history of interactions. Pairs traded protocols, tested each other's designs with students in their own classrooms, then wrote feedback on the protocol. They then discussed their feedback in person at the final meeting.

One pair included Allison, an educator from Jackson High School, and David, an educator from Grant Elementary. Allison had 10 years of experience working in an alternative high school, and David was in the first year of creating a personalized program within his elementary school for disengaged students. Allison and David were assigned to work together because one of the research team members had noticed that they had often engaged each other in conversation during the mixed-group activities. The focus of their protocols, however, was different: Allison's school team had focused on protocol questions to elicit evidence of student growth, whereas David's school team focused on questions to guide the development of interest-based projects with students.

On the third NIC Saturday, tasked with exchanging protocols and making a plan for user-testing, their conversation started with the context and logistics of using the protocol, but soon turned to probing each other's expertise. David asked Allison for help thinking about two questions when she was testing his protocol: “Are there questions that are better predictors of student success?” and “Are there questions that lead to student self-awareness and potential for success on a project?” Allison noted these on her planning document.

A few weeks later, when Allison made comments on David's protocol (via Google documents), she wrote that two of the questions from his protocol, “Who is your audience?” and “How will you measure the quality of your work?” elicited the response from her students, so she suggested that the two questions could either be combined or one of them eliminated. To David's question about student success with the project, Allison challenged him to define his criteria for success better, noting that his use of the term “reasonable” could be interpreted in different ways. In her feedback, Allison responds to David's request for help and shares her expertise with David through a high level of specificity in her feedback combined with recommendations for improvement.

This help was reciprocated by David. He wrote a page of comments on Allison's protocol about his testing process. The goal of Allison's protocol was to develop the relationship between teacher and student through questions about the student as a learner. David described how his conversation with a fourth grader went and that through the protocol, he “learned a lot of new information about the student as a person and learner and think that this protocol … could really strengthen the relationship between the teacher and the student.” This reflection provides evidence of mutual appropriation, where David is adopting some of Allison's perspectives on the goal of conferring.

On the final NIC Saturday, Allison and David met to discuss each other's feedback. They engaged in 45 min of animated conversation, sharing their experience with testing out each other's protocols. David later shared this reflection with his colleagues, “This whole idea of knowing your students better … we were talking about [this as] the key to creating robust personalized learning projects because they are so connected to who the students are as people.” This quote shows a converging understanding of ideas that can be traced through the series of interactions that were set up by these collaborative design activities. Additionally, on the social network survey, Allison and David both indicated higher than average importance ratings for each other. This supports that David and Allison asked each other for help, reliably received help, and valued the quality of the help they received.

These two examples illustrate the progression of interactions that was possible within the 90-day cycle. As the cycle progressed, participants increasing relied on the authenticity of their colleagues' expertise as designers in similar situations to guide their actions. The NIC process created the conditions for building a distributed network of expertise where help-seeking interactions happened across participants.

Discussion

Networked Improvement Communities and other social innovation approaches require educators and researchers to work together to solve complex problems. We explore how collaborative design could serve as the interaction structure at the heart of a NIC process. This case study is a first effort to test using social network and qualitative data analytic techniques to understand the network of relationships that were fostered through the initiating NIC activities. Our research question, how do NIC collaborative design activities foster reciprocated, help-based interactions? helps us understand how social network and qualitative data might be applied to explore the development of network ties amongst participants. We use the idea of reciprocated, help-based interactions as an indicator to point toward how NIC activities could lead to the development of relational trust. Tracing the patterns and progressions of reciprocated, help-based interactions opens a window into the capacity that emerges from NIC collaborative design activities.

Answering this question leads us to explore how we could use social network tools and qualitative data to trace the emergence of help-based interactions across the participant network in order to contribute to the growing literature on the effects of NICs on research-practice partnerships, and social innovation more broadly. The NIC initiation framework described by Russell et al. (2017) states that “coordinated action among the partners should align with the core design activities and that the development of social infrastructure, such as the “culture, norms, and identity” (p. 5) is a desired outcome. We find Penuel's (2019) insight that infrastructuring establishes the “configurations of conditions needed” (p. 2), a helpful way to think about the role of NIC design in sparking emergent relational trust networks. The social infrastructure of PiPNIC emerged from the strategic action of the initiation team to create a sequence of help-based interactions across participants.

Leveraging social network and qualitative data illustrates the path from configuration conditions to the emergence of the PiPNIC social infrastructure. The mixed-group assignments provided proximity for interaction. An early task, watching and annotating other participants' conferring videos, created an initial condition for asking for and receiving feedback from other educators. The grouping strategies built a history of professional interactions over time and a network structure that positioned educators to connect with each other within and across schools.

The social network data showed that researchers moved to the periphery during implementation. A peripheral position for a researcher is different from a traditional research-practice model of researcher as source of knowledge. The peripheral position instead aligns with more collaborative design approaches that center the user (the educator, in this case). Positioning the researcher this way may suggest a structural component to the shift in researcher role in the context of research-practice partnerships that are described (e.g., Cohen-Vogel et al., 2015). This insight presents an important follow up investigation of the networked position of the researcher.

The triangulation of social network and qualitative data was crucial in our study. The social network data helped trace how the patterns of interaction emerged. Our argument suggests that social network analysis can serve as a valuable indicator and as a strategy for how NIC designers might measure emergent network capacities. While the social network analysis demonstrated that relational ties were being developed, we needed the qualitative data to understand how and why help-based interactions occurred. We find that the feedback on which activities mattered, and why, called for qualitative information about participants, designers and the work produced. Together, the social network analysis and qualitative data provided feedback to PiPNIC designers on how and why the planned activities worked, and gave voice to the participants on the effects of the activities on the growth of their knowledge and skills.

Attention to the social infrastructure is not meant to replace measuring the targeted outcome of the NIC. The key outcome for a successful NIC, of course, is a change in the targeted behavior highlighted in the core problem of practice. If the NIC is organized around collaboratively designed solutions to improve student outcomes, then evaluation should be focused on how the solutions change the outcomes. Our work here is meant to investigate the social dynamics during initiation and propose how interactions might be seen in terms of fostering long-term relational trust networks.

Limitations of This Analysis

We would like to note several limitations of this paper. First, there are limitations in terms of how we operationalized relational trust, and more work is needed to establish a causal link between reciprocated, help-based interactions and the development of relational trust over time. Initially, we aimed at a NIC strategy that would create relational trust among participants. Careful reflection led us to understand that relational trust emerges as a result of long-term engagement in help-based interactions, and that the 90-day NIC cycle may not create enough opportunities to create relational trust. Although our paper emphasizes the role that the design of help-based interactions can plan in shaping professional interaction, we would need to continue the study by returning to the daily practices of educators in order to document whether the PiPNIC spark sustained into professional interaction in their schools.

Second, because the study was limited to documenting the PiPNIC 90-day cycle, we are also unable to answer the ultimate question of the PiPNIC process of whether everyday conferring practices changes in the participant school communities. The goal of a NIC is to change everyday practices, and the design of this study, which focused on the NIC process itself, did not allow us to subsequently track the effects of PiPNIC into participant classrooms. Anecdotally, we have checked in with each participant group since the PiPNIC experience. We found that each team experimented with new conferring practices in some way, ranging from formalizing all conferring practices across the school to trying new kinds of conferring (project-based, or interest-driven) protocols in everyday practice. Still, without a systematic study of post-NIC outcomes, we are unable to make claims about the impact of the NIC on practitioner work.

Third, there was a significant limitation to the social network data we collected. We were only able to give the survey near the end of the PiPNIC process (rather than at the beginning and at the end). Thus, the data that we report about the network at the outset was based largely on the recollection of participants near the end of the PiPNIC process (as well as on interview data collected near the beginning of PiPNIC). It would have been better for us to have surveyed people several times throughout to indicate where and how interactions were happening and to have asked specifically about who they were getting help from.

A final limitation of our study is the positionality of the designers. The authors of this paper also acted as the designers of the PiPNIC process. While this situation provided a unique perspective on the design process, it also limited our ability to see beyond the design choices that appeared obvious to us. Our use of interview, observation and social network data helped to triangulate our positionality, but did not remove our role in the design process as a factor in the analysis. In future studies, we would use the social network and qualitative inquiry models to study other NIC implementations to create a distance between the actors interested in the success of the NIC and those interested in measuring its results.

Conclusion

Our paper uses social network analysis and qualitative data analysis to trace how the collaborative design activities of a Networked Improvement Community create the capacity for participants to work together to solve their common problem of practice. The PiPNIC project used a NIC model to create an RPP around the emerging challenges in personalized learning, bringing together 31 participants from five schools and a university for a 90-day collaborative design cycle. In our findings, we document the emergence of network interactions across participants, as well as examples of the kinds of help asked for and received within and across participant groups. We showed that when help-based interactions are reciprocated, the emerging relationships allow the necessary risk-taking required for the kind of experimentation with practices characteristic of successful research-practice partnerships.

With increasing interest in using social network analysis as a form of network-level data to assess the health or effectiveness of a research-practice partnership, we conclude that the social network data alone described that interaction changed, but qualitative data supported a stronger connection with the design activities. The aim of this study is to contribute to emerging ideas of how to use social network analysis to understand how Networked Improvement Communities, and social innovations more generally, are initiated and developed. In doing so, we demonstrate, on a practical level, how social network and qualitative data might be used to generate network-level data for improvement, and we contribute theoretical insight into the way collaborative design creates the conditions for the kinds of interactions associated with long-term development of relational trust.

Social innovation requires attention to interactions. Collaborative design provided a helpful guide to developing activities that lead participants to ask for and to receive help from one another. Network designers can use collaborative design to create opportunities for participants to make their own practice public, engage in collaborative revision of their practices, try out each other's solutions, and create dissemination networks for resulting insights. Our efforts to trace the emergence of help-based networks across participants, and to use qualitative data to illustrate important occasions for interaction, provided insight on how the conditions for relational trust started to emerge in PiPNIC. We hope that our study of how participants helped each other design and test solutions to a shared problem of practice can provide an example for NIC leaders on network design, and for NIC researchers on network evaluation and guidance. More broadly, understanding how collaborative design activities can create these characteristics suggests that, by attending to how people come together, social capacity for innovation can be built in and through orchestrating meaningful help-based interactions among professionals.

Data Availability Statement

The datasets generated for this study will not be made publicly available. The data only available to members of the research team per the institutional ethics board policy.

Ethics Statement

The studies involving human participants were reviewed and approved by Education and Social/Behavioral Sciences Institutional Review Board University of Wisconsin-Madison, Wisconsin, USA. The patients/participants provided their written informed consent to participate in this study. Written informed consent was obtained from the individual(s) for the publication of any potentially identifiable images or data included in this article.

Author Contributions

RH and JK contributed to the conception and design of the partnership. JK organized the data collection. JK did the social network analysis and wrote the first draft of the manuscript. RH contributed the substantive revisions and framing. Both authors contributed to the manuscript revision, read, and approved the submitted version.

Funding

The research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through Grant R372A150031. The opinions expressed are those of the authors and do not represent the views of the Institute or the U.S. Department of Education.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/feduc.2019.00154/full#supplementary-material

Footnotes

1. ^Co-, collaborative, and participatory design all have their roots in Scandinavian tradition (Sanders and Stappers, 2008). We choose to use collaborative design as a term that is more commonly used in education research in the United States.

2. ^There are other many other dimensions and factors of trust that may impact long-term development of relational trust, such as perceived trust/mistrust of research organizations, universities, schools, parent groups, or individual propensity to trust others, especially those of the same profession, etc., however these dimensions of institutional, contractual, and individual trust are beyond the scope of this investigation.

3. ^In this paper, we focus on stage 3, the collaborative design cycle. For more detail on the first two stages of the process.

4. ^For understanding the structure of the network during implementation, we omitted the ties between researchers. The reason for this was 2-fold: we wanted to see the structure of the network around the design work as the educators experienced it and the interactions amongst researchers were primarily focused on the organizational design of the network.

5. ^When educators encounter new ideas for practice, such as from research, they are likely to use their context as a primary filter for whether that new information will fit (Tseng, 2012).

6. ^The accuracy of their recall was corroborated with what members of the research team knew as well. We knew there were very few ties across schools, but we did know about a few preexisting cross school connections and those were accurately reported. Likewise, we compared whether participants indicated interaction with their site captain, and this was consistent with what we knew, providing an external measure of accuracy that participant responses match observed interactions (Kashy and Kenny, 1990).

7. ^Network density is calculated by the number of indicated ties divided by the total number of possible ties.

References

Argyris, C. (2000). “Teaching smart people how to learn,” in Strategic Learning in a Knowledge Economy: Individual, Collective and Organizational Learning Process, eds R. Cross and S. Israelit (Oxford: Butterworth-Heinemann), 279–295.