Does SWPBIS Increase Teachers’ Collective Efficacy? Evidence From a Quasi-experiment

Teachers’ collective efficacy is predictive of students’ success. School-Wide Positive Behavior Interventions and Supports implementation requires the whole team to set itself common goals regarding behavior management. The main purpose of this study was to investigate the medium-term effects of a SWPBIS intervention on teachers’ collective efficacy. Nine schools and 139 teachers and staff members (n intervention = 74, n control = 65) took part in the study. The study shows that SWPBIS implementation has a positive effect on teachers’ collective efficacy both for primary and secondary schools at post-test 1 (ES = +0.80) and 2 (ES = +0.71). Differences are observed at baseline and at posttests according to the educational level. The link between subscales of a school climate instrument and teachers’ collective efficacy is also investigated. The “structure for learning” subscale explains the greatest variance in collective efficacy.

Collective teacher efficacy is one of the most impressive predictors of student achievement, but the remaining question is how to improve it.

According to several authors (Donohoo, 2018; Donohoo et al., 2018; Eells, 2011; Sun et al., 2017), collective teacher efficacy can counteract the negative impact of socioeconomic conditions on student learning. Hope therefore exists for principals of schools that enroll many students from minority, disadvantaged backgrounds.

We posit that a school-wide program working on school culture and common goals can improve teachers' collective effectiveness. Research by Sørlie and Torsheim (2011) has demonstrated this previously. We intend to do so in turn, but in a different context. The novelties brought by our research are the validation of a measure of teachers' collective efficacy, in a French-speaking primary and secondary education context, and to examine the effects of the implementation of SCP (the French name of SWPBIS) in a quasi-experimental and longitudinal design including two post-tests.

The aim of this paper is to measure the evolution of collective efficacy in four schools where SCP was implemented, and to contrast it with the evolution of collective efficacy in five control schools.

Introduction

What is Teachers’ Collective Efficacy?

Collective efficacy is defined as teachers’ beliefs about the educational team’s ability to educate students. These beliefs constitute a norm that influences the actions and outcomes of schools. This definition comes from Bandura’s own original research on this topic in 1993. According to Bandura, teachers operate collectively rather than as isolated individuals within an interactive social system. As such, the author notes that principal leadership contributes significantly to the development and maintenance of effective schools through their ability to bring their teams to work together by understanding the value of collaboration and believing in their ability to overcome obstacles as a team on the road to student success. The belief system of the educational team therefore creates a school culture that can have either vitalizing or demoralizing effects on the way schools function as a social system: vitalizing if the team believes itself to be collectively capable of promoting the academic success of its students, otherwise demoralizing. Bandura (1993) therefore postulates a link between collective effectiveness and academic success. The other definitions found in the literature are all congruent with Bandura’s definition.

For example, Goddard et al. (2000; 2001; 2004a, b) define collective teacher efficacy as the judgment of teachers in a school about the ability of the educational team to organize and execute the courses of action required to have a positive impact on students. Tschannen-Moran and Barr (2004) define collective teacher efficacy as the collective perception that teachers in each school have of themselves as making an educational difference for their students, beyond the educational impact of their families and communities. Skaalvik and Skaalvik (2007) define collective efficacy as teachers’ beliefs in the ability of the educational team to implement strategies that will enable students to succeed. According to Skaalvik and Skaalvik (2007), schools with high perceived collective efficacy set ambitious goals and demonstrate persistence in their efforts to achieve those goals. These ambitious goals create normative pressure that encourages all teachers in the school to do everything they can to excel and discourages them from giving up when faced with difficult situations.

What is known about the sources and shaping of collective teacher efficacy?

According to the first research conducted on this subject by Bandura (1993), it seems that the collective efficacy perceived by teachers evolves according to the level at which they teach. Fairly low when children enter school, then increasing once children are more acclimated to school routines; when the complexity of academic demands increases in the senior years and some gaps are not filled, teachers perceive the decline of their school in terms of teaching effectiveness. This is even more true in schools attended by a disadvantaged audience. According to Bandura’s (1993) research, students’ unfavorable socioeconomic conditions affect their academic performance more because of the deleterious effect they have on the educational team’s beliefs in its ability to motivate and educate students than because of direct links between these unfavorable conditions and academic performance. But the reverse is also true. With educational teams that strongly believe that, through their efforts, students can be motivated and learn regardless of their social background, schools attended by minority and disadvantaged student populations can achieve high scores on standardized tests in reading and mathematics.

Goddard et al. (2000) developed a model and a measure of collective teacher effectiveness. The foundations of their model are based on the concept of self-efficacy formulated by Bandura in 1993 and on the model of teacher effectiveness developed by Tschannen-Moran et al., 1998. The sources of teachers’ collective efficacy are, according to the authors who draw on Bandura (1993), the same as those of self-efficacy and are equally fundamental to the development of this collective belief: mastery experience, vicarious experience, social persuasion, and affective states (Goddard et al., 2000; 2004a, b). While the four sources of information are central to the shaping of collective efficacy, the cognitive processes of analyzing and interpreting information are also crucial. Consistent with the model of teacher efficacy described by Tschannen-Moran et al. (1998), Goddard and colleagues posit that there are two key elements in the development of teachers’ collective efficacy: analysis of the teaching task and assessment of teaching skills. Teachers feel effective in teaching certain subjects to certain students in specific situations. They may feel more or less effective when circumstances differ. The authors therefore hypothesize that the development of perceptions of the group’s ability to educate students successfully occurs when teachers consider the level of difficulty of the teaching task in relation to their perceptions of group competence. And, although the analysis of the teaching task and perceptions of group competence could be considered separately, perceptions of collective efficacy are formed only after teachers weigh the two elements presented above in relation to each other (Goddard et al., 2000). For the authors, perceived collective efficacy influences both individual and team behaviors.

Like Bandura (1993) before them, Goddard et al. (2000) believe that there is reason to believe that although collective efficacy is a relatively stable property, once it is developed, it can grow. According to the authors, this potential growth is consistent with the natural cycle of efficiency induced by reciprocal causality. For example, if gains in collective efficiency generate benefits, reciprocal causality suggests that these benefits can, in turn, enhance collective efficiency. However, this change in collective efficacy requires substantial effort.

In their 2011 systematic review on teacher efficacy, Klassen et al. note, however, that more research is needed to further investigate the sources and shaping of collective efficacy. As the link between teachers’ collective efficacy and students’ success has been highlighted by research and specifically in two meta-analyses (Eells, 2011; Sun et al., 2017), some authors have examined possible ways to increase it. School leadership appears to play a role in improving teachers’ collective efficacy (Goddard et al., 2000; 2001; 2004a, b; Leithwood et al., 2020, Sun and Leithwood, 2017; Tschannen-Moran and Barr, 2004).

What Has Already Been Shown About Collective Teacher Efficacy?

A first wave of research on collective teacher efficacy aims to shed light on the predictive link between collective teacher efficacy and student achievement in a school. Like Bandura in his 1993 study, Goddard et al. (2000; 2004b) postulate a link between teachers’ collective efficacy and students’ academic success, mainly via the behaviors that this perception of efficacy induces. To this extent, they believe that there is much to be gained by leading schools along a path that will systematically develop the collective efficacy of educational teams. The authors postulate that the consequences of a high level of collective efficacy will be the acceptance of ambitious goals, great organizational efforts and persistence that will lead to better performance. But the opposite is, therefore, also true. Low collective efficacy will lead to less effort, a propensity to give up, and lower performance.

Goddard et al. (2000; 2004b) developed a questionnaire based on their model and tested and validated it in a first study they did on a sample of 452 teachers from 47 elementary schools in the Midwestern United States. In their study, the authors hypothesized that collective teacher efficacy would be positively associated with school differences in student achievement, based on the theory that collective teacher efficacy can positively influence many teacher behaviors that would tend to improve student achievement. The results of the study show that collective teacher efficacy is a significant predictor of student achievement in both math and reading. The effect of collective teacher efficacy is larger in magnitude than any other demographic on both reading and math achievement. For math, the score on the collective efficacy measure is associated with an average gain of 8.62 points (out of 100). For reading, the average gain associated with the score on the collective efficacy measure is 8.49. Put another way, a one-unit increase in teacher collective efficacy is associated with a more than 40% standard deviation increase in student achievement. These results are consistent with Bandura’s 1993 research and posit perceived teacher collective efficacy as predictive of student achievement. The authors (Goddard et al., 2004b) replicated their first study in High schools and the results suggest the same tendency: for a one standard-deviation increase in collective efficacy a gain of 0.25 standard deviation is associated in terms of number of students who pass high-stakes assessments in 12th grade. The paper published by Donohoo et al. (2018) confirms the previous results and brings to the forefront the reciprocal causality between collective teacher efficacy and student progress ratings, already pointed out by Bandura in the first place (1993).

The results detailed above are confirmed by two meta-analyses. First, that of Eells (2011) who investigated the content of 26 studies on the effects of collective teacher efficacy on student achievement, and second, that of Sun and colleagues (2017) who, in turn, examined 11 studies on the effects of collective teacher efficacy and student academic outcomes. In both meta-analyses, on average, a strong positive correlation exists between collective teacher efficacy and student academic achievement. The conclusion that can be drawn from these meta-analyses is that collective teacher efficacy is a strong predictor of student achievement.

A second wave of research investigates the links between collective efficacy and other variables either at the teacher level or at the school level. Two Finnish researchers have investigated the extent to which teacher self-efficacy and collective efficacy mediate the effect of perceived school climate on teacher job satisfaction and burnout. Malinen and Savolainen (2016) investigated the issue through large scale longitudinal follow-up. The structural equation modeling indicates that perceived school climate is a significant predictor of teacher self-efficacy (ß = 0.26) and collective efficacy (ß = 0.51). According to Malinen and Savolainen (2016), this relationship indicates that collective efficacy shares elements with the constructs listed above: school climate and self-efficacy. The results of the study also highlight that self-efficacy correlates with teachers’ collective efficacy (r = +0.46), as other researchers had demonstrated before them, for example: Goddard et al. in their 2000 study and, after them, Skaalvik and Skaalvik in 2007. In this study, however, collective efficacy did not show any mediating effect on job satisfaction or burnout.

Skaalvik and Skaalvik (2007), two Norwegian researchers, investigated the links between teachers perceived collective efficacy and their self-efficacy. The authors postulate that collective efficacy is predictive of self-efficacy. To explore this question, Skaalvik and Skaalvik (2007) developed a measure of teacher self-efficacy and a measure of collective efficacy. They used both measures in a study of a sample of 244 teachers from 12 primary and middle schools. Their results indicate that self-efficacy is strongly related to collective efficacy (r = +0.64), according to the structural equation modeling. Collective efficacy is not directly related to burnout, but the authors nevertheless found a moderate indirect link between collective efficacy and burnout, with this relationship being mediated by self-efficacy (r = −0.49). The authors, consistent with previous research, indicate that self-efficacy and collective efficacy are two different constructs and should be measured as such, that the relationship between the two is strong, positive, and reciprocal or bidirectional.

It should be noted, however, that the fit between the concept of collective teacher efficacy itself and the measurement tools used in the research conducted on the subject is not always present (Klassen et al., 2011). According to the authors, only two measures of collective teacher efficacy fit the conceptual definition: Skaalvik and Skaalvik (2007), and Tschannen-Moran and Barr (2004).

School-Wide Positive Behavior Interventions and Supports

School-Wide Positive Behavior Interventions and Support is a systemic approach - the term systemic is used here in the sense of “present at all levels” and implemented by all educational stakeholders - designed to establish support for both the social culture and individual behaviors, both of which are necessary for a school to be a safe and effective learning environment for all students (Sugai and Horner, 2009).

Universal prevention tools are introduced to all students: 1) to support prosocial behaviors, 2) to maximize educational opportunities and academic success, and 3) to prevent the onset of behavior problems (Sugai and Horner, 2009).

SWPBIS organizes a dual three-tiered system of support increasing in intensity according to students’ needs and operating in parallel to address both behavioral and learning issues. The system is built around a primary intervention, known as universal prevention, targeting all students in the school. Secondary interventions are designed to reduce or eliminate risk factors for certain students (±15–20% of students) by providing them with “protective” factors. Tertiary interventions are aimed at reducing the complexity, intensity, and severity of behavioral problems in students with identified risk factors (±5% of students). This latter type of intervention is most often multidisciplinary and various experts are brought in. These interventions are always highly individualized to match students’ needs most closely (Sugai and Horner, 2002, 2006, 2009; Sugai and Simonsen, 2012).

SWPBIS, or SCP in its French version, is therefore a flexible, contextually, and culturally appropriate system for the school in which it is implemented, with the goal of creating a positive, safe, and effective learning environment by preventing and reducing behavioral problems through the development of values from which behavioral expectations are derived. These expectations must be clearly written in positive terms and observable by the educational team. They are then explicitly taught, displayed, and supported by a system of verbal and/or tangible feedback to build a positive school culture that reinforces students’ positive behaviors. Another key element of the system is the collection and use of data to guide the implementation and regulation of interventions in terms of both behavior and learning (Sugai and Horner, 2009).

SWPBIS and Teachers’ Sense of Efficacy

The topic has been little investigated. Only one longitudinal study conducted by Sørlie and Torsheim in Norway in 2011 examined the relationship between collective efficacy and the management of inappropriate student behavior. The purpose of Sørlie and Torsheim (2011) was to conduct a multilevel analysis of the relationship between teachers’ collective efficacy and behavior problems at school. The authors conducted this study as part of longitudinal research to question the effectiveness of the implementation of the Norwegian version of SWPBIS known as PALS. More than 1,000 teachers and principals from 48 Norwegian elementary school of different sizes participated in the study, which was conducted in two waves: the first in the spring of the 2006–2007 school year (Time 1) and the second, 6 mo later, at the beginning of the following school year, 2007–2008 (Time 2). Of the 48 schools that participated in the study, 28 were experimental schools and benefited from the implementation of the PALS program, while 20 schools formed the control group.

The authors used the Collective Efficacy Scale (CES; Goddard, 2001). This 12-item scale assesses the extent to which an educational team believes in its collective ability to positively influence student learning. To measure the amount of problem behavior, present in schools, Sørlie and Torsheim (2011) used two measures based on teacher observations called Problem Behaviour in the School Environment Last Week and Problem Behaviour in the Classroom Last Week, developed by Grey and Sime (1989) and translated into Norwegian by Ogden (1998). The measures consist of 15 and 20 items respectively.

At the school level, collective efficacy and behavior problems showed strong associations as indicated by correlations ranging from 0.70 to 0.78 (+0.70 < r < +0.78). Schools with high collective efficacy reported lower levels of behavior problems. At the school level, the correlation between collective efficacy and observed behavior problems in the classroom exceeded 0.90 (r = +0.90) and was 0.87 (r = +0.87) for behavior problems in common areas. These differences were stable over time.

The data presented in Sørlie and Torsheim’s (2011) longitudinal study confirm that collective efficacy is a significant variable at the school level and likely a stable feature of a school’s culture. Sørlie and Torsheim’s (2011) findings are consistent with the assumptions made by Goddard and colleagues (2000; 2004b) when they established through their empirical research that collective efficacy is a stable school contextual variable that requires substantial effort to change.

That said, intervention programs such as SWPBIS (PALS in Norway) could change this. For example, Sørlie and Torsheim (2011) found that in schools where collective efficacy increased from T1 to T2, teachers consistently reported a lower prevalence of behavior problems over time. In schools with a negative change in collective efficacy beliefs, teachers reported higher rates of behavior problems at T2 compared to T1. However, the authors state that they also found the opposite connection: the increase in the prevalence of behavior problems at the school over time is related to a decrease in perceived collective efficacy. Thus, according to the authors’ empirical research, there is a strong bidirectional predictive relationship between collective efficacy and behavior problems.

Sørlie and Torsheim (2011) conclude that teachers from schools with high collective efficacy implement more consistent positive behavior support practices than teachers from schools with lower collective efficacy. Teachers in more collectively effective schools are also more likely to persist in their efforts to regulate behavior problems and to propose and reinforce a more common set of rules. The reverse is also true: if strong pressure is placed on positive student behavior and consistent responses to rule infractions, as in the implementation of SWPBIS, collective team efficacy may subsequently increase.

Teachers’ collective efficacy contributes decreasing inequalities and is a strong predictor of students’ achievement. Yet little is known – if anything – about how to improve it. SWPBIS, which has demonstrated its effectiveness at improving school climate and decreasing students’ problem behavior, was identified as having the potential to improve teachers’ collective efficacy to a large extent. However, to date, no comparative study has investigated the effectiveness of a program, or of SWPBIS in particular, in improving collective efficacy.

Research Questions and Hypotheses

First research question: Does the implementation of the French version of School-Wide Positive Behavior Interventions and Supports influence the collective efficacy perceived by the members of the educational teams?

It is hypothesized that perceived collective efficacy will increase more in schools where SCP is implemented compared to control schools.

Second research Question: Is teachers perceived collective efficacy related to aspects of school climate?

The hypothesis that perceived collective efficacy is related to school climate is formulated.

Third research Question: To what specific aspects of school climate is perceived collective efficacy related?

Previous research (e.g.: Malinen and Savolainen, 2016) highlights a link between school climate and collective efficacy. A predictive link between collective efficacy and student achievement has also been demonstrated by different authors (Bandura, 1993; Donohoo, 2018; Eells, 2014; Goddard et al., 2000; 2004; Sun et al., 2017). The hypothesis that is therefore posited is that the aspects of school climate that affect student success will be those highlighted by the results of the analyses.

Method and Measurement

Participants

As this is a quasi-experimental research design, each of the four schools in the intervention group is matched with one or two control school(s) selected according to the following criteria: socio-economic status, size, location, educational network where possible, and the options offered (e.g.: language immersion).

It should be noted that three of the four schools in which School-Wide Positive Behavior Interventions and Supports is implemented are schools attended by students from low to very low socioeconomic backgrounds.

The sample is composed of staff members from the four pilot schools in which SCP is implemented. There are three elementary schools and one middle school. In all, this represents an n = 74 at pretest for the intervention schools. The other part of the sample consists of the teachers of the five control schools in which the research team did not intervene at all, apart from the times of data collection (passing questionnaires). This represents an n = 65 at pretest. One hundred and thirty-nine staff members answered the questions in this scale three times: 101 were teachers (73%), 110 were women (78%), 68 were elementary school teachers (49%), and 86 had been teaching for more than 11 yr (62%).

Table 1 details the demographic characteristics of staff members at the participating schools.

TABLE 1

TABLE 1. Demographic characteristics at baseline.

The following table presents the characteristics of the schools participating in the study.

Instruments

Perceived Collective Teacher Efficacy Scale

The Skaalvik and Skaalvik (2007) scale was selected and translated. It consists of seven items (five points Likert scale) that reflect the ease with which the school educational team carries out its behavioral and cognitive tasks.

Here are the items in their original version, then in their French version.

We translated and adapted the Suite using double translation followed by reconciliation and validation by an expert (Grisay, 2003; Harkness, 2003).

After a field test of the scale with about 20 volunteer teachers, four people were contacted again to conduct a cognitive laboratory to ensure the quality of the translation. The research team wanted to verify that the understanding of the concept in French had the same meaning as in the original version of the measure.

Internal consistency analysis of the scale conducted on SAS 9.4. indicated a Cronbach’s alpha of 0.86 on pretest data. Exploratory factor analysis was conducted on MPlus (Múthen and Múthen, 1998-2011) to compute McDonald’s hierarchical omega (Béland et al., 2017; Deng and Chan, 2017; Peters, 2014), with the following formula:

$\varpi = \frac{\left({{\displaystyle \sum }}_{j=1}^J{\lambda }_j\right)2}{\left({{\displaystyle \sum }}_{j=1}^J{\lambda }_j\right)2+ {{\displaystyle \sum }}_{j=1}^J{\psi }_j^2}$

The result obtained is: = +0.84. According to McDonald (1999), this is a good index.

The measures of internal consistency of the perceived collective efficacy scale attests to the internal consistency and reliability of the instrument as used in French speaking Belgium. The scale was added to the end of the school climate questionnaire given annually to members of the educational teams in both the experimental and control schools. This choice was made by the researchers not to multiply the number of questionnaires to be completed by the teams.

To ensure the validity of the pre-test measure, the first administration of the scale took place before any detailed presentation of SCP was made, i.e., at the very beginning of the full team training day devoted to the selection and definition of values. The questionnaire was offered at the same time of year (between the third week of November and the winter break) for the next 2 yr.

School Climate Questionnaire

The Georgia School Climate Survey Suite (La Salle et al., 2021), in its “staff member” version (31 items, four points Likert scale), was translated and adapted using the following method: double translation, reconciliation, and expert validation.

The protocol described for the Perceived Collective Teacher Efficacy scale (field test, cognitive laboratory), was applied to the Climate questionnaire.

The sub-dimensions of the climate questionnaire are staff connectedness: staff perceptions of the degree to which they feel they fit in and are a part of their school (α = 0.80); structure for learning: staff perceptions of the degree to which they feel their colleagues treat students fairly, have high expectations, and set clear rules (α = 0.84); physical environment: staff perceptions of maintenance of school grounds and resources (α = 0.74); peer/adult relations: staff perceptions of how students interact with peers and adults in their school (α = 0.88); and parental involvement: staff perceptions of the degree to which parents are involved in their student’s education (α = 0.83) and school safety: staff perceptions of their own safety at school (removed due to a lack of internal consistency).

Baseline Comparison

Besides school’s characteristics comparability (Table 2), baseline comparability between the intervention and control groups was verified for all sub-dimensions of the school climate questionnaire, as this was the main outcome of the study. The difference between groups at pretest is less than half a standard deviation (Slavin, 2008) for each of the sub-dimensions of the climate questionnaire, with a minimal difference at pretest of ES = +0.02 and a maximal difference of ES = +0.44.

TABLE 2

TABLE 2. Characteristics of participating schools.

Baseline difference between groups with respect to perceived collective efficacy was greater than half a standard deviation on the total sample of teachers (Hedges g = −0.67). When computed by education level according to What Works Clearinghouse standards (2020), analysis shows that intervention and control groups are highly contrasted in primary education (Hedges g = −1.19) and comparable at secondary level (Hedges g = −0.08).

Data collection and software

Data collection to verify the effects of SCP implementation was conducted as follows: the pre-test questionnaire was administrated before starting SCP preparation. The first post-test questionnaire was administrated at the same period 1 yr later, which means 6 mo after the start of the implementation. The second post-test was administrated at the same period 2 yr later or 18 mo after the implementation started. Timeline for data collection can be found in Table 3.

TABLE 3

TABLE 3. Data collection.

To answer research questions 1, 2, and 3, SAS 9.4 software was used: to calculate effect sizes, first; to perform a Stepwise regression analysis, second; and to allow the software to test for a link between the climate questionnaire and perceived collective efficacy.

Results

Question 1: Influence of SCP on Teachers’ Collective Efficacy

The magnitudes of the effect of SCP implementation on perceived collective efficacy calculated first at post-test 1, then at post-test 2, for all staff members across all levels of education are quite large: ES = +0.80 at post-test 1 and ES = +0.71 at post-test 2. The detailed results are shown in Table 4.

TABLE 4

TABLE 4. Teachers’ collective efficacy comparison between baseline, post-test 1 and post-test 2 (all educational levels).

Since comparability was not ensured at pretest between intervention and control groups across all educational levels – there is comparability among secondary staff in terms of sense of collective efficacy, whereas this is not the case in elementary schools – effect sizes were computed separately for elementary and secondary schools. They are presented in Tables 5,6.

TABLE 5

TABLE 5. Teachers’ collective efficacy comparison between baseline, post-test 1 and post-test 2 (at the primary level).

TABLE 6

TABLE 6. Teachers’ collective efficacy comparison between baseline, post-test 1 and post-test 2 (at the secondary level).

The collective efficacy mean score increased at both time in intervention group, while it remained stable in the control group, being already higher at baseline. At the same time, the difference between intervention and control groups at pretest measured by Student’s T became not significant at posttests 1 and 2. The effect sizes of SCP implementation on teachers perceived collective efficacy were +0.90 at post-test 1 and +1.02 at post-test 2 for educational teams in elementary schools. These results must be interpreted with caution considering that the experimental group was less confident in its collective efficacy at baseline.

The situation was different for educational teams in secondary schools. Intervention and control groups were comparable at baseline on demographic and on climate and collective efficacy scales. The effect size of SCP implementation on teachers perceived collective efficacy was positive and in favor of the intervention group at post-test 1 (ES = +0.47) and marginal at posttest 2 (ES = +0.07). In intervention group, mean results increased after 6 mo of implementation, and then remained stable. In control group, there was a slight increase at posttest 1, and another one at posttest 2, with a decrease of the standard deviation.

When mean results are examined, there was an increase in perceived collective efficacy between baseline and post-test 1 at both educational levels. This improvement was maintained at post-test 2, 18 mo after the start of the implementation, for secondary staff. Improvement went further at post-test 2 for primary staff.

Questions 2 and 3: Link Between Collective Efficacy and School Climate

Following the example of Malinen and Savolainen, 2016, the researchers wanted to investigate the possible links between the sense of collective efficacy and school climate. The results of the Stepwise regression analysis performed on SAS 9.4. can be found in Table 7. Model 5 was selected because it has the lowest Mallow relevance index (C(p) = 3.3751) and the highest percentage of variance explained (73.57% of variance explained). Stepwise regression results show that 73.57% of the variance is explained by the variables group (intervention vs. control), gender, structure for learning, physical environment, and peer and adult relationships.

TABLE 7

TABLE 7. Stepwise regression analysis results.

The dimension of the climate questionnaire administered to teachers in the intervention and control groups that explained the greatest percentage of variance in teachers perceived collective efficacy was the dimension “structures for learning.” It explained 63.78% of variance in collective efficacy.

Other elements intervened in the explanation of the variance and notably that of belonging to the intervention group. Membership of the intervention or control group explained 5.23% of the variance.

Discussion

Considering the mean improvement of perceived collective efficacy in the intervention schools, we observe an improvement at both education levels at post-test 1 and post-test 2. Considering the improvement due to the intervention, the extent to which this perception has improved varies depending on the level of education and on the sharpness with which these results are analyzed. As we consider that comparability between groups on all independent (demographic) and dependent (climate and perceived collective efficacy) variables must be present at baseline, then this study concludes that, after the SCP implementation, the secondary school staff perceives an increase in collective efficacy. If some consider that comparability on the independent variables (demographics) and on the main dependent variable (school climate) is an indicator of groups’ comparability at baseline, then both primary and secondary teams in schools implementing SCP perceive an improvement in collective efficacy.

As Kelm and McIntosh (2012) and Sørlie and Torsheim (2011) have demonstrated in their studies, the explanation for these effect sizes of SCP implementation on perceived collective efficacy is the decrease in behavioral problems and time spent on them. Indeed, the harmonization of common practices of positive and coherent discipline management within schools implementing SCP, thanks to all the components of the system, allows educational teams to see a decrease in problematic behaviors and the time spent on them. Through the implementation of SCP components, educational team members perceive greater collective effectiveness.

Based on the results of research conducted by (Ross et al., 2012), the researchers also postulate that the sense of collective efficacy of the educational teams in the four pilot schools in our country where SCP is implemented has increased well because teachers working in low socioeconomic status schools benefit from SCP first, which is the case for the pilot schools our country.

The improvement is maintained at post-test 2, although the changes in leadership in all the intervention schools happened shortly before taking the “climate and collective efficacy of teachers” questionnaires at post-test 2 and may have put the educational teams in significant difficulties in terms of the functioning of the schools. Indeed, these changes in terms of leadership and operational difficulties may have affected the teams’ sense of collective efficacy, since previous research found a link between executive leadership, institutional functioning, and sense of collective efficacy (Goddard et al., 2000; 2001; 2004a, b; Leithwood et al., 2020, Sun and Leithwood, 2017; Tschannen-Moran and Barr, 2004).

As hypothesized in the second research question and as Malinen and Savolainen (2016) demonstrated in previous research, perceived collective efficacy is related to school climate. Stepwise regression analysis linking items from both climate and perceived collective efficacy instruments demonstrates this. Linking sub-dimensions of school climate to perceived collective efficacy explains up greatest variance in perceived collective efficacy. To the extent that school climate and perceived collective efficacy are part of a school’s culture, the link between the two is not surprising. The concept of school climate refers to the quality and characteristics of school life (Cohen et al., 2009; Gage et al., 2016). School climate influences student outcomes in behavior and social skills (Gottfredson et al., 2005; McIntosh et al., 2006; Gage et al., 2016). The Stepwise regression analysis highlighted the subscale “structures for learning” as the part of the climate questionnaire that explained the most variance in collective efficacy. The items constituting this sub-dimension relate to what teachers put in place to support student success (e.g.: “my school promotes success for all students” or “teachers in my school work hard to ensure that all students do well”). The content of these items relates to the efforts made by the educational team to support students’ academic success. To the extent that collective efficacy is defined as teachers’ beliefs about the ability of the entire educational team to positively influence students and their success (see above) and to the extent that academic success is discussed as both a predictor of and an outcome of perceived collective efficacy (Bandura, 1993; Goddard et al., 2000, 2004b; Eells, 2011; Sun et al., 2017; Donohoo, 2018; Donohoo et al., 2018), it is not surprising to find a link here too between collective efficacy and the elements put in place to promote student success and perceived collective efficacy. This dimension “structures for learning” alone explains for the greatest variance in the sense of collective efficacy in the Stepwise model.

This study is subject to a few limitations. Firstly, at the beginning of the research, the questioning of the collective effectiveness of the educational teams with which the researchers would work to implement SCP indicated a deficit of collective effectiveness. Some of the educational teams were dysfunctional. It is worth noting that the research project on the effects of SCP implementation focused on schools in “adjustment mode.” These “underachieving” schools were audited based on a few criteria, one of which concerned the (dys)functioning of the educational team. In other words, it was difficult for the research team to find comparison schools on that criterion. SCP implementation is not intended to solve team cohesion issues. However, thanks to the efforts made by the educational teams to harmonize their practices for the joint management of the various aspects of school discipline – prevention and correction – the collective effectiveness perceived by the teams themselves evolved positively, much more positively than the initial situation would have suggested.

Secondly, the size of the sample on which the research was conducted, and the “pilot” nature of the project do not allow for the generalization of the results observed. If SCP was implemented on a larger scale and the research replicated on a larger sample, the differences observed at baseline between the groups would be reduced. Furthermore, the absence of random components in the construction of the sample and its small size make it impossible to use inferential statistical tools.

Thirdly, the research team did not have access to information from the control schools, apart from that collected through the various questionnaires. It cannot be ruled out that specific situations or elements may have a link with the decrease in collective effectiveness perceived in these educational teams. Investigating the functioning of the schools in the control group would represent an added value for future research to ensure that it is indeed SCP implementation that improves the perceived collective effectiveness and not particular events experienced in the schools in the control group that explain the decrease in this same feeling within these schools.

Our findings give directions for future research. Firstly, schools in the intervention group enroll students from disadvantaged to very disadvantaged backgrounds and are therefore more likely to benefit from SCP implementation. Future research should implement SCP in schools with students from different socioeconomic backgrounds to compare the effects of these demographic characteristics on implementation and to verify that perceived collective efficacy increases more quickly and more in schools with more disadvantaged students.

Secondly, as the Stepwise regression, used to investigate whether a link between collective efficacy and school climate exists, does not allow to analyze mediating variables, SEM analyses could be conducted in future research.

Our findings also has some implications for practice. While the extent to which the perceived improvement in collective efficacy varies, the reasons for this improvement are the same: first, the set of components of SCP and the efforts required from the educational teams to implement them, and second, the socio-economic level of the schools in which SCP has been implemented. Harmonizing practices requires considerable effort, especially considering the pedagogical freedom usually granted to teachers in French speaking Belgium. Getting all members of an educational team to agree on common values, on expected behaviors, on an explicit and active way of teaching them, on a system of reinforcement as well as on a common policy for dealing with inappropriate behaviors is not an easy task, especially since it is not enough to agree in theory, but to apply them in a common and consistent way. Getting all staff members to agree on this common discipline and on its application by all requires considerable effort, which may well correspond to that mentioned by Bandura in his study (1993), and which can lead to a change in perceived collective effectiveness. Changing elements of a school’s culture such as perceived collective efficacy requires significant change and effort.

As Sørlie and Torsheim (2011) have previously demonstrated, by jointly and consistently implementing SCP components, such as expected, and commonly known behaviors taught explicitly, reinforcement of appropriate behaviors, and a common policy for managing inappropriate behaviors applied consistently, the educational team acts on their students’ behavioral issues and time. By aligning behavior management practices to reduce the occurrence of problem behaviors, the school team’s collective effectiveness also increases, if all the key elements of SCP are implemented with fidelity.

It should also be noted that changes in school leadership can have an impact on perceived collective efficacy insofar as the leadership of school principals contributes significantly to the development and maintenance of effective schools through their ability to get their teams to work together by understanding the value of this collaboration and by believing in their ability to overcome, as a team, the obstacles on the way to their students’ success (Bandura, 1993; Goddard et al., 2000; 2001; 2004a, b; Leithwood et al., 2020, Sun and Leithwood, 2017; Tschannen-Moran and Barr, 2004).

Overall, the main purpose of this study was to investigate the issue of perceived collective efficacy improvement through SCP implementation in a quasi-experimental research design. Using the scale translated from Skaalvik and Skaalvik (2007), the research team was able to observe the effect of SCP implementation on perceived collective efficacy in the pilot schools. Using measures collected in the control schools, the researchers found that the implementation of the key elements of SCP improved the collective efficacy perceived by the educational team. Regardless of the issue of group comparability at baseline that has been discussed in this article, the implementation of the various components of the SCP has effects on the perceived collective efficacy of educational team members in the intervention group schools.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

This study was approved by the Ethics Committee of the Faculty of Psychology, Speech Therapy and Education Sciences, University of Liege, Belgium [Project 17-18-70]. Data collection does respect the European Union General Data Protection Regulation (UE 2016/679).

Author Contributions

AB and CD contributed to conception and design of the study. CD organized the database. CM and DD performed the statistical analysis. CD wrote the first draft of the manuscript. AB and CD wrote sections of the manuscript. All authors contributed to manuscript revision, read, and approved the submitted version.

Funding

This research was funded by Administration générale de la Fédération Wallonie-Bruxelles as part of the Pacte pour un Enseignement d’excellence.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Bandura, A. (1993). Perceived Self-efficacy in Cognitive Development and Functioning. Educ. Psychol. 28 (2), 117–148. doi:10.1207/s15326985ep2802_3

CrossRef Full Text | Google Scholar

Béland, S., Cousineau, D., and Loye, N. (2017). Utiliser le coefficient omega de McDonald à la place de l'alpha de Cronbach. mje 52 (3), 791–804. doi:10.7202/1050915ar

CrossRef Full Text | Google Scholar

Cohen, J., Mccabe, E., Michelli, N., and Pickeral, T. (2009). School Climate: Research, Policy, Practice and Teacher Education. Teach. Coll. Rec. 111 (1), 180–213. Available at: https://www.researchgate.net/publication/235420504_School_Climate_Research_Policy_Teacher_Education_and_Practice.

Google Scholar

Deng, L., and Chan, W. (2017). Testing the Difference Between Reliability Coefficients Alpha and Omega. Educ. Psychol. Meas. 77 (2), 185–203. doi:10.1177/0013164416658325

PubMed Abstract | CrossRef Full Text | Google Scholar

Donohoo, J. (2018). Collective Teacher Efficacy Research: Productive Patterns of Behaviour and Other Positive Consequences. J. Educ. Change 19, 323–345. doi:10.1007/s10833-018-9319-2

CrossRef Full Text | Google Scholar

Donohoo, J., Hattie, J., and Eells, R. (2018). The Power of Collective Efficacy. ASCD resources. Available at https://www.ascd.org/el/articles/the-power-of-collective-efficacy

Eells, R. J. (2011). Meta-Analysis of the Relationship Between Collective Teacher Efficacy and Student Achievement. MS dissertation. Chicago (IL): Loyola University of Chicago.

Google Scholar

Gage, N. A., Larson, A., Sugai, G., and Chafouleas, S. M. (2016). Student Perceptions of School Climate as Predictors of Office Discipline Referrals. Am. Educ. Res. J. 53 (10), 492–515. doi:10.3102/0002831216637349

CrossRef Full Text | Google Scholar

Goddard, R. D. (2001). Collective Efficacy: A Neglected Construct in the Study of Schools and Student Achievement. J. Educ. Psychol. 93, 467–476. doi:10.1037/0022-0663.93.3.467

CrossRef Full Text | Google Scholar

Goddard, R. D., Hoy, W. K., and Hoy, A. W. (2004a). Collective Efficacy Beliefs:Theoretical Developments, Empirical Evidence, and Future Directions. Educ. Res. 33, 3–13. doi:10.3102/2F0013189X033003003

CrossRef Full Text | Google Scholar

Goddard, R. D., Hoy, W. K., and Hoy, A. W. (2000). Collective Teacher Efficacy: Its Meaning, Measure, and Impact on Student Achievement. Am. Educ. Res. J. 37 (2), 479–507. doi:10.2307/1163531

CrossRef Full Text | Google Scholar

Goddard, R. D., LoGerfo, L. F., and Hoy, W. K. (2004b). High School Accountability: The role of Perceived Collective Efficacy. Educ. Pol. 18, 403–425. doi:10.1177/0895904804265066

CrossRef Full Text | Google Scholar

Gottfredson, G. D., Gottfredson, D. C., Payne, A. A., and Gottfredson, N. C. (2005). School Climate Predictors of School Disorder: Results from a National Study of Delinquency Prevention in Schools. J. Res. Crime Delinquency 42, 412–444. doi:10.1177/0022427804271931

CrossRef Full Text | Google Scholar

Grey, J., and Sime, N. (1989). “Findings from the National Survey of Teachers in England and Wales,” in Elton Report, Discipline in Schools: Report of the Committee of Enquiry Chaired by Lord Elton. Editor Department of Education and Science and the Welsh office (London: Her Majesty’s Stationary Office), 222–292.

Google Scholar

Grisay, A. (2003). Translation Procedures in OECD/PISA 2000 International Assessment. Lang. Test. 20 (2), 225–240. doi:10.1191/0265532203It254oa10.1191/0265532203lt254oa

CrossRef Full Text | Google Scholar

Harkness, J. (2003). “Questionnaire Translation,” in Cross-Cultural Survey Methods. Editors J. Harkness, F. Van de Vijver, and P. Mohler Wiley, 35–56.

Google Scholar

Kelm, J. L., and McIntosh, K. (2012). Effects of School-wide Positive Behavior Support on Teacher Self-efficacy. Psychol. Schs. 49 (2), 137–147. doi:10.1002/pits.20624

CrossRef Full Text | Google Scholar

Klassen, R. M., Tze, V. M. C., Betts, S. M., and Gordon, K. A. (2011). Teacher Efficacy Research 1998-2009: Signs of Progress or Unfulfilled Promise? Educ. Psychol. Rev. 23, 21–43. doi:10.1007/s10648-010-9141-8

CrossRef Full Text | Google Scholar

La Salle, T. P., Rocha-Neves, J., Jimerson, S., Di Sano, S., Martinsone, B., Majercakova Aobertova, S., et al. (2021). A Multi-National Study Exploring Adolescent Perceptions of School Climate and Mental Health Problems. School Psychol. 36 (3), 155–166. doi:10.1037/spq0000430

Leithwood, K., Sun, J., and Schumacker, R. (2020). How School Leadership Influences Student Learning: A Test of “The Four Paths Model”. Educ. Adm. Q. 56 (4), 570–599. doi:10.1177/0013161x19878772

CrossRef Full Text | Google Scholar

Malinen, O.-P., and Savolainen, H. (2016). The Effect of Perceived School Climate and Teacher Efficacy in Behavior Management on Job Satisfaction and Burnout: A Longitudinal Study. Teach. Teach. Edu. 60, 144–152. doi:10.1016/j.tate.2016.08.012

CrossRef Full Text | Google Scholar

McDonald, R. P. (1999). Test Theory: A Unified Treatment. Mahwah, NJ: Lawrence Erlbaum. doi:10.1007/978-1-349-14280-4

CrossRef Full Text

McIntosh, K., Chard, D. J., Boland, J. B., and Horner, R. H. (2006). Demonstration of Combined Efforts in School-wide Academic and Behavioral Systems and Incidence of Reading and Behavior Challenges in Early Elementary Grades. J. Positive Behav. Interventions 8, 146–154. doi:10.1177/10983007060080030301

CrossRef Full Text | Google Scholar

Múthen, L. K., and Múthen, B. O. (1998-2011). Mplus User’s Guide. 6th Edn. Los Angeles, CA: Múthen & Múthen.

Ogden, T. (1998). Elevatferd og læringsmiljø: Læreres erfaringer med og syn på elevatferd og læringsmiljø i grunnskolen. Oslo: Kirke-, utdannings- og forskningsdepartementet.

Peters, G. J. Y. (2014). The Alpha and the Omega of Scale Reliability and Validity: Why and How to Abandon Cronbach's Alpha and the Route Towards More Comprehensive Assessment of Scale Quality. Eur. Health Psychol. 16 (2), 56–69. doi:10.31234/osf.io/h47fv

CrossRef Full Text | Google Scholar

Ross, S. W., Romer, N., and Horner, R. H. (2012). Teacher Well-Being and the Implementation of School-wide Positive Behavior Interventions and Supports. J. Positive Behav. Interventions 14 (2), 118–128. doi:10.1177/1098300711413820

CrossRef Full Text | Google Scholar

Skaalvik, E. M., and Skaalvik, S. (2007). Dimensions of Teacher Self-efficacy and Relations with Strain Factors, Perceived Collective Teacher Efficacy, and Teacher Burnout. J. Educ. Psychol. 99 (3), 611–625. doi:10.1037/0022-0663.99.3.611

CrossRef Full Text | Google Scholar

Slavin, R. E. (2008). Perspectives on Evidence-Based Research in Education-What Works? Issues in Synthesizing Educational Program Evaluations. Educ. Res. 37 (1), 5–14. doi:10.3102/0013189X08314117

CrossRef Full Text | Google Scholar

Sørlie, M.-A., and Torsheim, T. (2011). Multilevel Analysis of the Relationship Between Teacher Collective Efficacy and Problem Behaviour in School. Sch. Effectiveness Sch. Improvement 22 (2), 175–191. doi:10.1080/09243453.2011.563074

CrossRef Full Text | Google Scholar

Sugai, G., and Horner, R. H. (2009). “Defining and Describing Schoolwide Positive Behavior Support,” in Handbook of Positive Behavior Support. Editors W. Sailor, G. Dunlap, G. Sugai, and R. H. Horner (Germany: Springer), 307–326. doi:10.1007/978-0-387-09632-2_13

CrossRef Full Text | Google Scholar

Sugai, G., and Horner, R. R. (2006). A Promising Approach for Expanding and Sustaining School-wide Positive Behavior Support. Sch. Psychol. Rev. 35 (2), 245–259. doi:10.1080/02796015.2006.12087989

CrossRef Full Text | Google Scholar

Sugai, G., and Horner, R. (2002). The Evolution of Discipline Practices: School-wide Positive Behavior Supports. Child. Fam. Behav. Ther. 24 (1-2), 23–50. doi:10.1300/J019v24n01_03

CrossRef Full Text | Google Scholar

Sugai, G., and Simonsen, B. (2012). Positive Behavioral Interventions and Supports: History, Defining Features, and Misconceptions. Mansfield: Center for PBIS and Center for Positive Behavioral Interventions and Supports.

Sun, J., and Leithwood, K. (2017). Calculating the Power of Alternative Choices by School Leaders for Improving Student Achievement. Sch. Leadersh. Manag., 37, 80–93. doi:10.1080/13632434.2017.12963510.1080/13632434.2017.1293635

CrossRef Full Text | Google Scholar

Sun, J., Zhang, S., and Przybylski, R. (2017). “Conceptualizing the Critical Path Linked by Collective Teacher Efficacy,” in 2017 Annual meeting of the American Educational Research Association, San Antonio, TX, April 27–May 1, 2017 (Washington, DC: San Antonio and AERA).

Google Scholar

Tschannen-Moran, M., and Barr, M. (2004). Fostering Student Learning: The Relationship of Collective Teacher Efficacy and Student Achievement. Leadersh. Pol. Schools 3 (3), 189–209. doi:10.1080/15700760490503706

CrossRef Full Text | Google Scholar

Tschannen-Moran, M., Hoy, A. W., and Hoy, W. K. (1998). Teacher Efficacy: Its Meaning and Measure. Rev. Educ. Res. 68 (2), 202–248. doi:10.3102/00346543068002202

CrossRef Full Text | Google Scholar

What Works Clearinghouse (2020). What Works Clearinghouse Procedures Handbook. Washington, DC: U.S. Department of Education, Institute of Education Sciences. Available at: https://ies.ed.gov/ncee/wwc/handbooks.