Supporting the Installation of Comprehensive, Integrated, Three-Tiered Models of Prevention: Educator Perspectives

We provide results from a district-wide survey of 253 certified educators to determine (a) the degree to which they reported implementing components of their school’s comprehensive, integrated, three-tiered (Ci3T) model of prevention in the current year and (b) the areas in which educators might benefit from continued professional learning. The purpose of this study was to develop a data informed professional learning plan to continue to support implementation at the conclusion of an Institute for Education Sciences (IES) -funded practitioner-researcher partnership grant. At least half of educators reported a high level of implementation for all Tier 1 features. Tier 2 and Tier 3 behavioral and social supports for students were reported as less fully implemented than academic interventions, yet still well-above the scale midpoint. For every component, elementary educators indicated statistically significantly higher levels of implementation relative to secondary educators. Over half of respondents indicated a high level of implementation of eight of 20 research-based practices and supports examined, with a statistically significant relation between ratings of currently implemented practices and the desire for support in four practices: small-group self-determination instruction, peer-mediated support strategies, check-in/check-out, and strategies for internalizing behaviors. In terms of preferred professional development avenues, educators rated in-district during-school workshops, courses for college credit (on-line), and brief “good practice” guides most favorably. We close with a discussion of implications, limitations, and future directions.

Introduction

Educators are charged with the formidable task of meeting the diverse academic, behavioral, and social and emotional well-being needs of all students. Fortunately, state policies and educators continue to meet this challenge by supporting students in all three learning domains, focused on making certain all students have the full set of skills necessary for college and career settings (Every Child Succeeds Act, 2015). For example, 41 states have adopted education standards such as Common Core State Standards (Common Core State Standards Initiative, 2020), and 23 states have learning goals to ensure students develop the “soft skills” needed to navigate school, career, and social settings successfully (Yudin, 2014; Collaborative for Academic, Social, and Emotional Learning [CASEL], 2018). Kansas is one of a few states with comprehensive PK-12 standards for social and emotional learning (Kansas State Department of Education [KSDE], 2019).

Educational leaders have led the way in implementing systemic school improvements through the adoption and installation of a tiered system of supports. Tiered systems offer a framework for meeting school system priorities by organizing and delivering prevention efforts to meet the needs of all students. Such tiered systems include Response to Intervention (RTI; Fuchs et al., 2012) with an academic focus, Positive Behavioral Interventions and Supports (PBIS; Horner and Sugai, 2015) with a behavioral focus, multi-tiered system of supports (MTSS; Averill and Rinaldi, 2013) often including academic and behavioral domains, Interconnected Systems Framework (ISF; Barrett et al., 2013) integrating mental health supports with PBIS, and Comprehensive, Integrated, Three-tiered (Ci3T) models of prevention (Lane et al., 2014) addressing academic, behavioral, and social and emotional wellbeing domains. There is a recent emphasis on building integrated systems to address multiple learning domains (Institute of Education Sciences, 2018). In this descriptive study, we examined professional learning needs of educators in one district implementing an integrated model – Ci3T – to meet their K-12 students’ learning needs across multiple learning domains.

Comprehensive, Integrated, Three-Tiered Model of Prevention

Comprehensive, integrated, three-tiered models are designed uniquely to meet the multiple needs of students across academic, behavioral, and social domains (Lane et al., 2019). In schools with Ci3T models in place, all students have access to educational experiences with research-based practices, strategies, and programs at the Tier 1 level for all three learning domains. When Tier 1 efforts are delivered with fidelity but are insufficient to meet students’ educational needs, additional secondary (Tier 2) and tertiary (Tier 3) interventions or supports are provided. Potential Tier 2 supports include low-intensity strategies (e.g., instructional choice, Royer et al., 2017), Check in/Check out (Hawken et al., 2007), or self-regulated strategy development (SRSD) for writing (Harris and Graham, 2009) or math (Losinski et al., 2021). Tier 3 efforts may be required for students with intensive needs. Examples of Tier 3 efforts include functional assessment-based interventions (Umbreit et al., 2007) or Lindamood Phoneme Sequencing^® (Lindamood and Lindamood, 1998). Using tiered logic, educators collect and monitor data to make decisions effectively and efficiently regarding needed supports.

Data-informed decision-making is a core feature of tiered systems. Data drive shifts in instructional practices across all tiers as well as inform professional learning needs to facilitate high-fidelity Ci3T implementation and sustainability (Fixsen et al., 2005; Lane et al., 2019). Educators collect and analyze student-level data, such as academic and behavior screening, office discipline referrals, attendance, and formative assessment data for instructional planning. To monitor Ci3T implementation, school-site Ci3T Leadership Teams collect social validity (i.e., stakeholder views of goals, procedures, and outcomes; Wolf, 1978) and treatment integrity (i.e., the extent to which an intervention is implemented as intended; Bruhn et al., 2015) data. Collectively, these data inform decisions about Ci3T implementation, professional learning, and the effectiveness of practices implemented to meet students’ needs in all learning domains.

Comprehensive, integrated, three-tiered models of prevention uniquely position educators to meet the diverse learning needs of students (Lane et al., 2019). Ci3T provides a transparent framework supporting general and special educators’ collaboration and delivery of high-quality educational practices across multiple domains. Furthermore, collection and use of data empower educators to review practices continuously and make improvements by adjusting instruction and engaging in targeted professional learning. Given the expectation of educators to meet this challenge, it is essential to investigate their professional learning needs.

Professional Learning and Tiered Systems

Researchers have demonstrated the important role of professional learning in implementing tiered systems. For example, Donnell and Gettinger (2015) found professional learning specifically focusing on tiered systems to be predictive of teacher self-efficacy in implementing such systems. Similarly, educators have identified professional learning as an integral component of the implementation of tiered systems (McIntosh et al., 2013). In fact, insufficient professional learning may lead to confusion and a lack of confidence in implementing tiered systems (Cavendish et al., 2016).

Lane et al. (2015) conducted a statewide survey of Tennessee school administrators to investigate the extent to which components of Ci3T were in place and their needs and desires for professional learning. Results indicated high implementation of many Tier 1 practices across school settings. Authors found a positive relation between reported implementation of tiered system components and desire for professional learning on these practices. We extend these findings by examining Ci3T professional learning needs of certified educators in one Midwestern district implementing Ci3T K-12.

Purpose

We conducted this study to assess the views of certified educators on the implementation of Ci3T following completion of a two-year IES-funded researcher-practitioner partnership grant designed to support the district’s priority of initial installation of Ci3T models district-wide (Fixsen et al., 2005). This study was designed to inform subsequent professional learning plans. To this end, we extended inquiry by Lane et al. (2015) to (a) learn about the degree to which educators reported implementing components of Ci3T and (b) determine the areas in which they might benefit from continued professional learning to support implementation efforts. In addition, we offer this descriptive study as a methodological illustration of a data-based approach to professional learning decisions for other district leaders committed to designing, installing, and sustaining integrated tiered systems. Research questions paralleled those presented by Lane et al. (2015) to examine current school practices (Questions 1 and 2), resources and professional learning needs (Questions 3 through 6), and preferred avenues for professional learning (Questions 7 and 8). Research questions were as follows:

(1) To what extent do educators report implementing features common to Ci3T models?

(2) Are there differences in implementation of these features across elementary and secondary schools?

(3) To what degree do educators report implementing practices consistent with the Ci3T framework and what is their desire for additional professional learning addressing these practices?

(4) What is the relation between current reported implementation and desire for professional learning?

(5) Are there differences in the degree to which educators of elementary or secondary schools implement and desire professional learning to support these educational practices?

(6) What professional learning areas do educators prioritize?

(7) How likely are educators to draw upon various avenues for professional learning about Tier 1, 2, and 3 supports within Ci3T models?

(8) Are there differences between educators of elementary and secondary schools in preferred avenues for professional learning?

Materials and Methods

Participants and Schools

Participants were 253 certificated educators from 21 schools (14 elementary, 4 middle, 2 high school, and 1 college and career center) in a medium-size district involved in an IES partnership grant. Certificated educators refer to teachers who have the state’s teaching credential, also referred to as a teacher license in some states. Most respondents were from elementary schools. Respondents had an average of 15.99 (SD = 10.66; range = 0–45) years of experience in education. See Tables 1A,B for school characteristics.

TABLE 1

Table 1. A. School characteristics.

Procedures

Data collection took place during a 4-week period in spring 2017 during the final few months of the two-year IES partnership grant focused on installing Ci3T models in all district schools. This allowed respondents to reflect on their Ci3T implementation, with elementary educators concluding 3 years of implementation, middle and high school educators concluding 2 years of implementation, and college and career center educators concluding their 1st year. Following university and district approvals, we distributed a link to an informational letter and survey via email to all district faculty and staff using the Qualtrics online survey platform. The information letter explained the purpose of the study was to assess professional learning needs of faculty and staff related to implementation and sustainability of Ci3T, with the intent to examine overall professional development needs for the district and for different school levels (e.g., elementary vs. secondary). Furthermore, the letter explained benefits and risks of the study, that participation was voluntary, and all data would be kept confidential.

After reviewing the information letter, participants who indicated they did not wish to complete the survey received an automatically generated thank you message from Qualtrics for considering the opportunity. Participants who indicated they would like to complete the Ci3T Professional Development Survey were automatically taken to the survey in Qualtrics, with responses submitted directly to researchers. Approximately 1 week later, we distributed an electronic reminder invitation from Qualtrics to those who had not yet responded. We sent a final invitation a few days before the 4-week data collection period closed.

We designed the study to require no more than 20 min of time to complete all items (Lane et al., 2015). Furthermore, we offered online and hard copy versions of the survey and based our follow-up prompting on recommended procedures (Dillman et al., 2008). In addition, educators were informed the information they shared would be used to inform the district’s implementation and sustainability of Ci3T. We received a district list of 1,045 certified staff; however, 143 individuals also served in either part-time administrative or classified roles and were excluded from these analyses. Of the remaining 902 individuals who received the survey, 297 individuals responded to the survey (defined as completing at least one item; 33% of 902) and 253 (28% of 902) agreed to have their responses included in analyses.

Measures

Similar to the Lane et al. (2015) survey, our survey included the following sections, with most of the 136 items using a 5-point, Likert-type scale. One section of the survey, Skills and Behaviors Important for Student Success (20 items), were not included in the current study as they were not related to the research questions.

Implemented Three-Tiered Models in Schools

Respondents rated the extent to which their school was currently implementing 25 common features of Ci3T models of prevention across academic, behavioral, and social domains (see Table 2). See Lane et al. (2015) for details regarding survey construction. Items included Tier 1 features regarding instruction and reinforcement (13 items; selection of a schoolwide social skills curriculum), Tier 2 and 3 supplemental supports (4 items; for academic, behavioral, and social supports), and features regarding monitoring and data-informed decision makings (8 items; including treatment integrity and social validity). Respondents rated each item using a 5-point, Likert-type scale (1 = not at all, 3 = somewhat, 5 = fully). Cronbach’s alphas were 0.93, 0.92, and 0.90, respectively.

TABLE 2

Table 2. Ratings of features of three-tiered models currently being implemented.

Resource and Professional Development Needs

Respondents rated 20 educational practices common to a Ci3T model of prevention at Tier 1, 2, or 3. Item paralleled those developed by Lane et al. (2015), featuring a range of research-based strategies and practices to support academic, behavioral, and social performance. See Table 3 for full listing, some of which included traditional Tier 2 supports (e.g., small-group instruction), low-intensity supports (e.g., self-monitoring), teacher-level strategies (e.g., behavior-specific praise), as well as Tier 3 supports (e.g., 1:1 reading or math instruction, functional behavioral assessments). Respondents rated each item to indicate (a) the extent to which their school was currently implementing the practice (Table 3), and (b) their desire for additional support on implementing each practice (Table 4), with correlations between constructs reported in Table 5. Respondents used 5-point, Likert-type scales to rate implementation (1 = not at all, 3 = somewhat, 5 = fully) and desire for support (e.g., training, coaching, print or web-based resources; 1 = no desire, 3 = some desire, 5 = strong desire). Respondents were also asked to prioritize three areas for professional development in the next school year based on their understanding of the needs of their students, faculty, and staff. Cronbach’s alphas were 0.93 and 0.91, respectively.

TABLE 3

Table 3. Educational practices and supports currently implemented.

TABLE 4

Table 4. Desire for professional development on how to implement educational practices.

TABLE 5

Table 5. Relation between educational practices and supports currently implemented and desire for professional development on how to implement educational practices.

Professional Development Avenues

Participants were given a list of 20 potential avenues for professional development (e.g., state conferences, brief “good practice” guides; see Table 6). Lane et al. (2015) developed this listing from the professional development literature (e.g., Garet et al., 2001; Penuel et al., 2007). Participants used a 5-point, Likert-type scale (1 = very unlikely, 3 = somewhat likely, and 5 = very likely) to rate how likely they would be to draw upon the professional development learning avenues assuming each was actually available. Respondents could write in other avenues. Cronbach’s alpha for this section was 0.88.

TABLE 6

Table 6. Potential avenues for professional development and learning.

Design and Statistical Analysis

We followed the same data analytic plan used by Lane et al. (2015), using descriptive and inferential statistics to answer research questions. We used descriptive statistics to summarize (a) current practices in schools, (b) educational practices currently in place and the desire for professional development in each area, and (c) preferences for professional development avenues. To examine current school practices, we conducted a series of one-way ANOVAs comparing group means, contrasting responses from elementary vs. secondary (middle and high combined; college and career center educators were not included in these contrasts as they were only in their 1st year of implementation; Torff and Sessions, 2008) educators on (a) Tier 1, (b) Tier 2 and 3, and (c) monitoring and decision making. Given unequal cell sizes compared, we used Tukey multiple comparisons (α = 0.05) to examine mean differences. We used this same analysis for all school-level comparisons in subsequent research questions. We computed Pearson correlation coefficients to examine the relation between ratings of currently implemented and desire for professional development for each educational practice.

Results

Implementation of Core Comprehensive, Integrated, Three-Tiered Model Features

We present results for three categories of features: Tier 1, Tiers 2 and 3, and monitoring and data-informed decision making. See Table 2 for item-level data.

Tier 1 Features

Fifty percent or more of respondents reported a high level of implementation (ratings of 4 or 5) for all Tier 1 features. Mean score ratings all exceeded the scale midpoint, with scores ranging from 3.68 (SD = 1.14) for instruction in school-wide behavior expectations (at least once per month) to 4.43 (SD = 0.79) for school-wide expectations for all key settings. Just under 60% of respondents reported this latter feature was fully implemented (rating a 5), yet regular monthly instruction of school-wide expectations was not implemented with the same level of integrity.

Tier 2 and 3 Features

More than 60% of respondents reported a high level of implementation (ratings of 4 or 5) for Tier 2 and Tier 3 support for academic issues. Tier 2 and Tier 3 behavioral or social supports were less fully implemented compared to Tier 1, yet still reported as well above the scale mid-point with respective mean scores of 3.78 (SD = 1.05) and 3.76 (SD = 1.12). Less than 5% of responding educators reported having no Tier 3 behavioral or social supports.

Monitoring and Data-Informed Decision-Making Features

Fifty percent or more of responding educators indicated a high level of implementation (ratings 4 or 5) for all monitoring and data-informed decision-making features. Mean scores all exceeded the scale midpoint, ranging from 3.58 (SD = 1.01) for a method of gathering information from stakeholders on the primary (Tier 1) program to 4.55 (SD = 0.83) for academic screening of all students to benchmark progress (at 3x per year).

Variations in Implementation Across School Levels

Results of a series of one-way ANOVAs contrasting the views of elementary vs. secondary school-level educators on Tier 1, Tiers 2 and 3, and monitoring and data-informed decision-making features suggested for every feature elementary educators reported statistically significantly higher levels of implementation relative to secondary educators (Table 3 for significance testing). All except two mean score comparisons were statistically significant to p < 0.0001: individual classroom management systems in addition to school-wide systems (p = 0.0034) and behavior screening of all students to monitor progress (at 3x per year; p = 0.0017).

Implementation of Practices and Desire for Professional Development

In terms of the current level of implementation of the 20 educational practices and supports examined (see Table 3), 50% or more of the respondents indicated a high level of implementation (ratings of 4 or 5) for eight, with mean scores as follows: small-group reading instruction (M = 4.03, SD = 1.07), behavior intervention plans (BIP; M = 3.65, SD = 0.93), increasing behavior-specific praise (M = 3.85, SD = 0.88), increasing opportunities-to-respond (OTR; M = 3.71, SD = 0.87), incorporating choice and preferred activities into instruction (M = 3.61, SD = 0.86), bullying prevention (M = 3.63, SD = 1.03), courageous conversations (M = 3.72, SD = 0.93), and technology in the classroom (M = 4.20, SD = 0.83). There were several educational practices and supports with implementation averages below the scale midpoint including small-group self-determination instruction, self-monitoring strategy instruction, peer-mediated support strategies, functional behavioral assessments, and strategies for internalizing behaviors.

In terms of the desire for professional development in implementing these 20 educational practices, strategies, and programs (see Table 4), 50% or more of the respondents indicated high desire (ratings of 4 or 5) for training on small-group social skills instruction (M = 3.52, SD = 1.03), self-monitoring strategy instruction (M = 3.64, SD = 0.94), inclusive supports (M = 3.52, SD = 0.95), incorporating choice and preferred activities into instruction (M = 3.69, SD = 0.99), bullying prevention (M = 3.59, SD = 1.02), strategies for internalizing behaviors (M = 3.81, SD = 0.89), de-escalation techniques (M = 3.95, SD = 0.89), and technology in the classroom (M = 3.82, SD = 1.01). All mean scores were above the scale midpoint, suggesting respondents were open to professional development on all educational practices, with strategies for internalizing behaviors, de-escalation techniques, and technology in the classroom as the main priorities.

Relation Between Implementation and Desire for Professional Development

We found a statistically significant correlation between educators’ ratings of currently implemented practices and the desire for support for only four practices: small-group self-determination instruction (r = 0.26, p = 0.0003), peer-mediated support strategies (r = 0.19, p = 0.01), check-in/check-out (CICO; r = 0.17, p = 0.02), and strategies for internalizing behaviors (r = 0.17, p = 0.02). These were four of the five strategies reported as least implemented (Table 5).

Differences in the Views of Elementary and Secondary Educators

We conducted a series of one-way ANOVAs contrasting educators’ ratings of (a) the extent to which they were implementing educational practices and supports that might be implemented as part of Ci3T models and (b) desire for additional support (e.g., training, coaching, print or web-based resources) in addressing each of the 20 practices. In general, the views of elementary vs. secondary educators highly varied in implementation of practices and support. Results indicated statistically significant differences were observed for implementation of small-group social skills instruction, F(1, 187) = 11.77, p = 0.0007, R² = 0.06, small-group reading, F(1, 188) = 181.01, p < 0.0001, R² = 0.49, self-monitoring strategy instruction, F(1, 186) = 14.67, p = 0.0002, R² = 0.07, test-taking strategy instruction, F(1, 187) = 5.95, p = 0.02, R² = 0.03, behavioral contracts, F(1, 187) = 16.58, p < 0.0001, R² = 0.08, peer-mediated support strategies, F(1, 187) = 6.17, p = 0.01, R² = 0.03, behavior intervention plans (BIP), F(1, 186) = 10.65, p = 0.0013, R² = 0.05, providing 1:1 reading or academic instruction, F(1, 186) = 18.57, p < 0.0001, R² = 0.09, increasing behavior-specific praise to students, F(1, 186) = 26.21, p < 0.0001, R² = 0.12, increasing opportunities-to-respond for students, F(1, 187) = 21.45, p < 0.0001, R² = 0.10, check-in/check-out (CICO), F(1, 183) = 26.32, p < 0.0001, R² = 0.13, inclusive supports, F(1, 186) = 22.97, p < 0.0001, R² = 0.11, bullying prevention, F(1, 186) = 21.19, p < 0.0001, R² = 0.10, strategies for internalizing behavior, F(1, 185) = 19.85, p < 0.0001, R² = 0.10, courageous conversations, F(1, 186) = 8.30, p = 0.004, R² = 0.04, and de-escalation techniques F(1, 184) = 21.35, p < 0.0001, R² = 0.10. With the exception of strategies for internalizing behaviors, elementary educators reported a higher level of implementation compared to secondary educators (see Table 3).

In contrast, elementary and secondary school level educators were highly comparable in their desire for professional development on how to implement educational practices and supports related to Ci3T (see Table 4). There were two exceptions where statistical significance was found: functional behavior assessments (FBA), F(1, 184) = 5.80, p = 0.02, R² = 0.03 and behavior intervention plans (BIP), F(1, 184) = 6.27, p = 0.01, R² = 0.03. For both, elementary educators reported a greater desire for professional development relative to secondary educators.

Professional Development Priorities

Educators prioritized their top three areas for professional development in the coming year. As the first priority, approximately 15% of respondents selected de-escalation techniques and 10.42% prioritized technology in the classroom. For the second priority, approximately 11% of respondents prioritized self-monitoring strategy instruction and de-escalation techniques. For the third priority, 13.68% of respondents rated de-escalation techniques, and approximately 11% rated incorporating choice and preferred activities into instruction.

Preferred Professional Development Avenues

When asked about preferences for avenues for professional development to learn about Tier 1, 2, and 3 supports within Ci3T models, nearly half of respondents indicated they would be likely (ratings 4 or 5) to draw upon in-district, during-school workshops (M = 4.06, SD = 1.03; see Table 6). Also, the following avenues were prioritized in descending order: course for college credit (on-line); brief “good practice” guides; teacher study groups, “learning circles”; and teacher collaboratives/networks. In-district, weekend workshops was by far the least preferred avenue (M = 1.81, SD = 0.97).

Differences in the Preferred Avenues Between Elementary and Secondary Educators

Results of a series of one-way ANOVAs indicated elementary and secondary educators reported similar preferences for professional development avenues, assuming all were available. They diverged only on the preferences for webinars, F(1, 185) = 4.19, p = 0.0421, R² = 0.02, with higher ratings at the elementary level (see Table 6).

Discussion

As educational leaders design, implement, evaluate, and sustain integrated tiered systems, professional development plays a key role in supporting educators to achieve these formidable tasks with a high sense of efficacy (Donnell and Gettinger, 2015). In this study, we extended the work of Lane et al. (2015) exploring issues related to achieving high-fidelity implementation of integrated tiered models of prevention. Whereas Lane and colleagues examined administrator views as key decision-makers, we focused on certified educators’ views. Specifically, given educators’ daily actions and decisions influence implementation and ultimately students’ educational experiences, we conducted this study to (a) obtain educators’ views on the extent to which they were implementing core components of Ci3T and (b) determine corresponding professional development and resource needs to support implementation.

This study was conducted in one partner school district as part of an IES-funded grant, (R305H150018). District-university partners used outcomes of this specific study to inform future professional learning for educators in their implementation of Ci3T district-wide. In an era when resources such as personnel time and money are perhaps more precious than ever (Lane et al., 2021a), it is critical to provide focused, high-quality professional development offerings well-aligned to the implementation of the system. By aligning professional development, it is possible to install, sustain, and refine integrated tiered systems such as Ci3T. It is equally important to customize professional development to provide accessible and enjoyable offerings for educators managing multiple responsibilities, professionally and personally (e.g., taking care of their own families). As such, we solicited educators input regarding implementation of core Ci3T features and 20 common practices, as well as their desire and preferred avenues for participating in professional development.

Implementation of Core Comprehensive, Integrated, Three-Tiered Model Features

It was encouraging to find most educators reported implementing all core Ci3T features to a high degree: Tier 1 features focused on instruction and reinforcement, Tier 2 and 3 supplemental supports, as well as monitoring and decision-making features. Of the 25 core items, 14 had ≥70% of educators report high implementation, nine items had 60–70% of educators report high implementation, and only two core features had 50–60% of educators report high implementation.

At Tier 1, teachers reported high implementation of academic (e.g., common curriculum for core academic areas, instruction linked to district and state standards), behavior (e.g., schoolwide PBIS, school-wide expectations for all key settings, a system for students to receive reinforcement for meeting expectations) and social (e.g., school-wide social skills curriculum, monthly [minimum] instruction in the social skills curriculum). These strengths in all three domains are encouraging and unsurprising given Kansas is one of a few states with comprehensive PK-12 standards for social and emotional learning (Kansas State Department of Education [KSDE], 2019). In addition, there is a national emphasis on designing, implementing, and sustaining integrated systems that prioritize “soft skills” to facilitate success not only in school but also later in employment (Yudin, 2014; McIntosh and Goodman, 2016; Institute of Education Sciences, 2018). This may, in part, account for many of the similarities in implementation reported by educators in this study as well as administrators in the Lane et al. (2015) study. For example, educators in the current study and administrator respondents in the Lane et al. (2015) study both indicated instruction in school-wide behavior expectations (at least once per month) was the Tier 1 feature least implemented, although still above the scale midpoint. Given the consistencies between administrators and educators, this may be an excellent priority for professional development. Namely, it may be helpful for those leading implementation and professional development to provide structures such as procedural integrity checklist that include explicit instructional schedules depicting dates for re-teaching school-wide expectations monthly across all key settings.

This study suggested areas for celebration and also refinement at Tier 2 and 3. For example, in just 5 short years since the Lane et al. (2015) survey findings were published, educators in the current sample reported a high level of implementation of both academic and behavioral supports at Tier 2 and 3. The mean level of reported implementation for academic issues at Tier 2 and Tier 3 were nearly identical at 4.04 (SD = 0.94) and 4.01 (SD = 0.95), respectively. While the level of reported implementation was still higher for academic relative to behavioral or social issues, implementation of Tier 2 and Tier 3 behavioral or social supports reported by educators in the current study was substantially higher than mean scores reported by administrators in the Lane et al. (2015) study with mean scores of 2.85 (SD = 1.27) and 2.69 (SD = 1.32), respectively. As we move forward in supporting successful implementation of integrated systems, it will be important to continue to provide high-quality professional development emphasizing the integrated nature of evidenced-based academic, behavioral, and social supports (e.g., integrated lesson planning, Lane et al., 2018). For example, self-monitoring and other behavioral supports (e.g., instructional choice) can be utilized at Tier 2 and 3 to facilitate engagement during instruction and ultimately academic performance (e.g., writing quality; Lane et al., 2011a). To accomplish this charge, it will be critical for educators to have access to multiple sources of data to inform decision making.

Other results suggest strong implementation of monitoring and data-informed decision-making features. It was particularly noteworthy to see two monitoring features, academic and behavior screening of all students 3 times per year, received the highest mean ratings (4.55 and 4.50, respectively), with 70.09% and 67.29% of respondents, respectively, reporting full implementation (rating of 5), perhaps indicative of the value educators place on these data for decision making. Early inquiry by Bruhn et al. (2014) suggested behavior screening were not a regular practice in many schools. One positive outcome of this district-wide commitment to designing, implementing, evaluating, and sustaining Ci3T has been their successful implementation of valid academic (AIMSweb) and behavior (Student Risk Screening Scale - Internalizing and Externalizing; Drummond, 1994; Lane and Menzies, 2009) screeners.

While five of the 24 core Ci3T features in Lane et al. (2015) were rated as more fully implemented in secondary schools by administrators, only one of the 25 core features in the current study was higher for secondary schools as rated by educators: a method of gathering information from stakeholders on primary program. In the current study, most features were more fully implemented by elementary educators. One of the most pronounced differences in implementation for elementary compared to secondary educators was the use of reinforcers (17% of the variance). This distinction was also noted in recent qualitative inquiry conducted as part of this partnership grant, where a theme across secondary teachers suggested they struggled with delivering acknowledgments (e.g., ticket paired with behavior-specific praise) when students met expectations (Lane et al., 2021b). The clear distinction between implementation at elementary compared to secondary schools in the current study was not surprising because the elementary schools had just concluded 3 years of Ci3T implementation whereas the middle and high schools had just concluded 2 years of implementation. Lessons learned from implementation sciences indicate full installation of new practices takes 2–3 years (Fixsen et al., 2005). During each implementation year in this district, Ci3T Leadership Teams attended five Ci3T professional learning sessions to build expertise in analyzing and interpreting treatment integrity, social validity, and student level data. Thus, elementary schools had more time for leadership teams to learn and bring such knowledge back to faculty and staff for improved implementation on core Ci3T features. Yet, despite the complexities of secondary school settings, there are multiple studies documenting the feasibility and effectiveness of integrated systems in middle and high schools over time (e.g., Flannery and Sugai, 2009; Lane et al., 2013).

Common Practices Within Comprehensive, Integrated, Three-Tiered Frameworks

Educators reported a high level of implementation for several of the educational practices presented, with most educators indicating a high implementation of small-group reading instruction, BIPs, increasing behavior-specific praise, increased OTRs, incorporating choice and preferred activities, bullying prevention, courageous conversations, and technology in the classroom. Each area was a topic well-aligned with the goals of the district’s board of education: equity, excellence, and engagement, and supported with multiple professional development offerings lead by district and university partners.

Educational practices and supports with lower levels of implementation included small-group self-determination instruction, self-monitoring strategy instruction, peer-mediated support strategies, FBA, and strategies for internalizing behaviors. This is not surprising given many teachers report not receiving sufficient pre-service instruction on non-academic related competencies. Some university teaching programs feature as little as one class on classroom management and one class on exceptionalities. This is troubling given many teachers with higher numbers of students with disabilities in their class leave the field because they may feel ill-equipped to meet students’ behavioral and social-emotional needs (Gilmour and Wehby, 2019).

Fortunately, results suggested educators were open to professional learning as evidenced by all mean scores above the scale midpoint, including some areas where implementation was reportedly low (e.g., self-monitoring strategy instruction, strategies for internalizing behaviors). There was a positive relation between educators’ ratings of currently implemented practices and desire for support for only four practices: small-group self-determination instruction, peer-mediated support strategies, CICO, and strategies for internalizing behaviors. The relation between implementation and desire for professional development was vastly different for educators in this sample relative to administrators’ views reported by Lane et al. (2015). Whereas administrators reported statistically significant relations for all practices except increasing behavior-specific praise, this was not the case for educators where only four practices – four out of the five least fully implemented – were highly correlated with the desire for professional development. Administrators may recognize the value of additional assistance for educators to implement these strategies and practices effectively and efficiently. In contrast, educators may see potential barriers (e.g., time, multiple demands), making them more cautious as they consider professional development priorities (Lane et al., 2011b).

In terms of differences between elementary and secondary educators, with exception of strategies for internalizing behaviors, elementary educators reported a higher implementation. Yet, elementary and secondary educators were highly comparable in their desire for professional development on how to implement educational practices and supports related to Ci3T, with the exception of FBA and BIP – both of which were more desirable to elementary educators. This desire for support with intensive interventions may be related to elementary educators’ increased contact time with students as students change classes far more frequently in middle and high schools. This may motivate elementary educators to seek support for students with the most challenging behaviors (Umbreit et al., 2007).

Further, results indicated 15% of educators place de-escalation techniques as a top priority followed by 11% of respondents prioritizing self-monitoring and de-escalation as second priorities. Clearly, educators are seeking information on how best to prevent and manage challenging behaviors that impede instruction. This information is essential to district leaders as they develop professional learning plans to support full installation of Ci3T and other integrated tiered systems (Fixsen et al., 2005). The next critical question is: How would educators like to access prioritized professional development?

Preferred Professional Development Avenues

Educators most preferred in-district during-school workshops, course for college credit (on-line), and brief “good practice” guides. Educator-reported likelihood for accessing various avenues of professional development was highly consistent with Lane et al. (2015), including the most likely (in-district, during-school workshops) and least likely (in-district, weekend workshops). Furthermore, elementary and secondary educators reported similar preferences for professional development avenues assuming all were available, with the exception of webinars, which were more favored by the elementary educators.

These ratings are understandable considering the ease of attending professional development during existing work hours compared to adjusting personal priorities for non-work hours. The convenience for educators to attend professional development during the school day must be carefully weighed by districts against the cost of substitute teachers and/or loss of instructional minutes unless professional development days are utilized. The next avenues rated more likely to be utilized by educators may, therefore, be viable options for districts: course for college credit (on-line) and brief “good practice” guides can be accessed independently outside of the instructional day or perhaps during teacher preparatory time. In the case of college credit for professional learning, the added incentive for teachers (e.g., credit toward salary scale increases) may make the logistics of partnering with a local college worth exploring (e.g., Oakes et al., 2020). Information on prioritized avenues is important as those coordinating professional development experiences will want to offer a range of options to facilitate educator access and eventual mastery of strategies, practices, and programs included in their Ci3T framework.

Limitations and Future Directions

We encourage readers to interpret results in light of the following limitations. First, study results constitute self-reported perceptions of educators from one district engaging in district-wide implementation of Ci3T. Although we intentionally collected limited demographic information for respondents to increase the likelihood of completion, it is possible participants provided inflated ratings for reasons related to social desirability. Furthermore, it will be important to be cautious when generalizing findings, given the results are from one district. This study offers a methodological illustration for others interested in focused professional development to promote fidelity of implementation (e.g., assessing district- or state-wide input to inform future professional learning offerings). While the process of engaging in data-informed, professionally learning activities is certainly generalizable, the direct content and focus areas of need and preferences may be district specific. As such, we encourage replication (Coyne et al., 2016), using a similar method with educators in other geographic locales. Furthermore, we encourage observational studies using component checklists to assess implementation coupled with this less direct method (e.g., survey) to more thoroughly assess levels of implementation of components of Ci3T models (Lane et al., 2019).

Second, the survey tool did not include operational definitions of each Ci3T feature listed. As with the Lane et al. (2015), this was intentional to keep the survey length brief and encourage participation. It is likely educators’ individual knowledge and opinions influenced their ratings of implementation and professional development needs. It would be an excellent next step for research teams to conduct more in-depth inquiry into professional learning needs for specific features. For example, what are additional professional learning needs and preferences to be able to design, implement, and evaluate functional assessment-based interventions at Tier 3 (Common et al., 2020)?

Third, as with the Lane et al. (2015) study, we conducted numerous statistical comparisons in this descriptive study. For example, we contrasted mean scores for elementary vs. secondary schools on 20 professional development topics. Given the implications of multiple tests, some results may be spurious. As such, results should be interpreted with caution until replicated in other locales and with schools across a range of implementation phases (e.g., installation, initial implementation, full implementation; Fixsen et al., 2005).

Fourth, it is important to consider the modest response rates. While response rates were similar to averages found in meta-analyses for online surveys (e.g., 34.2% in counseling journals, Poynton et al., 2019; 39.6% in sociometric, psychometric, and public opinion research, Cook et al., 2000), we encourage replication before generalizing outcomes and encourage future studies to also report response rates for comparisons (e.g., elementary vs. secondary) as appropriate. It may be professional learning needs vary as a function of implementation success. For example, survey participants may be those who were more (or less) invested in Ci3T system change efforts with higher (or lower) levels of implementation. Therefore, it also is possible that results of reported implementation of current practices may be over (or under)-inflated and that those who did not respond to the survey have lower (or higher) levels of implementation of the named practices and strategies.

Summary

We hope findings from this descriptive survey study situated in one partner school district in the Midwestern U.S. provides a methodological illustration for using a data-informed approach for professional development. Districts employing a data-informed approach to professional development can better support educators to implement Ci3T core elements and practices with high-fidelity district wide. By better understanding the degree to which elements of integrated tiered systems are in place as planned and specific areas in which school sites may benefit from professional development or resources to support them in such areas, we are hopeful those leading implementation efforts (including professional development) will be able to align services and supports with schools’ actual needs to support students’ academic, behavioral, and social learning. We are also hopeful we have paved a path for future inquiry focused on direct systems of measurement and incorporating stakeholders’ views. We look forward to learning more about how to build a professional learning scope and sequence to assist with effective, efficient, and sustained implementation of integrated tiered systems.

Data Availability Statement

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author/s.

Ethics Statement

The studies involving human participants were reviewed and approved by the Human Subjects Committee - Lawrence, University of Kansas, and the Arizona State University Institutional Review Board. Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.

Author Contributions

WO and KL conceptualized and co-led the study as part of a Researcher Practitioner Partnership grant, as well as co-leading the writing of the manuscript. KL conducted the analysis. DR managed consenting and data collection and contributed to the writing of the method. MB contributed to the data analysis and writing of the Sections “Materials and Methods, Results, and Discussion.” EAC supported data entry and analysis and creation of tables. GA supported data cleaning, table creation and reliability, and editing of manuscript. EDC served as the project coordinator for the partnership including data management. All authors contributed to the article and approved the submitted version.

Funding

This research was supported in part by the Institute of Education Sciences Partnership Grant (R305H150018) and the Office of Special Education Programs personnel preparation grant (H325D160080).

Author Disclaimer

Opinions expressed herein are those of the authors and do not necessarily reflect the position of the U.S. Department of Education, and such endorsements should not be inferred.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Averill, O. H., and Rinaldi, C. (2013). Research Brief: Multi-Tier System of Supports (MTSS). Orlando, FL: University of Central Florida.

Google Scholar

Barrett, S., Eber, L., and Weist, M. (eds) (2013). Advancing Educational Effectiveness: Interconnecting School Mental Health and School-Wide Positive Behavior Support. Available online at: https://www.pbis.org/resource/advancing-education-effectiveness-interconnecting-school-mental-health-and-school-wide-positive-behavior-support

Google Scholar

Bruhn, A. L., Hirsch, S. E., and Lloyd, J. W. (2015). Treatment integrity in school-wide programs: a review of the literature (1993–2012). J. Primary Prevent. 36, 335–349. doi: 10.1007/s10935-015-0400-9

PubMed Abstract | CrossRef Full Text | Google Scholar

Bruhn, A. L., Woods-Groves, S., and Huddle, S. (2014). A preliminary investigation of emotional and behavioral screening practices in K–12 schools. Educ. Treat. Children 37, 611–634. doi: 10.1353/etc.2014.0039

PubMed Abstract | CrossRef Full Text | Google Scholar

Cavendish, W., Harry, B., Menda, A. M., Espinosa, A., and Mahotiere, M. (2016). Implementing response to intervention: challenges of diversity and system change in a high-stakes environment. Teach. College Rec. 118, 1–36. doi: 10.1177/016146811611800505

CrossRef Full Text | Google Scholar

Collaborative for Academic, Social, and Emotional Learning [CASEL] (2018). K-12 Learning Goals for SEL in all 50 States. Available online at: https://casel.org/systemic-implementation/sel-policy-at-the-state-level/

Google Scholar

Common, E. A., Lane, K. L., Oakes, W. P., Schellman, L. E., Shogren, K., Germer, K. A., et al. (2020). Building site-level capacity for functional assessment-based interventions: outcomes of a professional learning series. Behav. Disord. doi: 10.1007/s10864-019-09355-4

CrossRef Full Text | Google Scholar

Common Core State Standards Initiative (2020). Read the Standards. Washington, DC: Common Core State Standards Initiative.

Google Scholar

Cook, C., Heath, F., and Thompson, R. L. (2000). A meta-analysis of response rates in web- or internet-based surveys. Educ. Psychol. Meas. 60, 821–836. doi: 10.1177/00131640021970934

CrossRef Full Text | Google Scholar

Coyne, M. D., Cook, B. G., and Therrien, W. J. (2016). Recommendations for replication research in special education. Remed. Spec. Educ. 37, 244–253. doi: 10.1177/0741932516648463

CrossRef Full Text | Google Scholar

Dillman, D. A., Smyth, J. D., and Christian, L. M. (2008). Internet, Mail, and Mixed-Model Surveys: The Tailored Design Method, 3rd Edn. Hoboken, NJ: John Wiley & Sons.

Google Scholar

Donnell, L. A., and Gettinger, M. (2015). Elementary school teachers’ acceptability of school reform: contribution of belief congruence, self-efficacy, and professional development. Teach. Teach. Educ. 51, 47–57. doi: 10.1016/j.tate.2015.06.003

CrossRef Full Text | Google Scholar

Drummond, T. (1994). The Student Risk Screening Scale (SRSS). Grants Pass, OR: Josephine County Mental Health Program.

Google Scholar

Every Child Succeeds Act (2015). Every Student Succeeds Act of 2015, Pub. L. No. 114-95 §114 Stat. 1177 (2015-2016). Cincinnati, OH: Every Child Succeeds.

Google Scholar

Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M., and Wallace, F. (2005). Implementation Research: A Synthesis of the Literature. Tampa, FL: University of South Florida.

Google Scholar

Flannery, K. B., and Sugai, G. (2009). School–Wide PBIS Implementation in High Schools: Current Practice and Future Directions. London: OSEP National Center.

Google Scholar

Fuchs, L. S., Fuchs, D., and Compton, D. L. (2012). Smart RTI: a next-generation approach to multilevel protection. Except. Children 78, 236–279. doi: 10.1177/001440291207800301

PubMed Abstract | CrossRef Full Text | Google Scholar

Garet, M. S., Porter, A. C., Desimone, L., Birman, B. F., and Yoon, K. S. (2001). What makes professional development effective? Results from a national sample of teachers. Am. Educ. Res. J. 38, 915–945. doi: 10.3102/00028312038004915

CrossRef Full Text | Google Scholar

Gilmour, A. F., and Wehby, J. H. (2019). The association between teaching students with disabilities and teacher turnover. J. Educ. Psychol. 112, 1042–1060. doi: 10.1037/edu0000

CrossRef Full Text | Google Scholar

Harris, K. R., and Graham, S. (2009). Self-regulated strategy development in writing: premises, evolution, and the future. Br. J. Educ. Psychol. 2, 113–135. doi: 10.1348/978185409x422542

CrossRef Full Text | Google Scholar

Hawken, L. S., MacLeod, K. S., and Rawlings, L. (2007). Effects of the behavior education program (BEP) on office discipline referrals of elementary school students. J. Positive Behav. Intervent. 9, 94–101. doi: 10.1177/10983007070090020601

CrossRef Full Text | Google Scholar

Horner, R. H., and Sugai, G. (2015). School-wide PBIS: an example of applied behavior analysis implemented at a scale of social importance. Behav. Anal. Pract. 8, 80–85. doi: 10.1007/s40617-015-0045-4

PubMed Abstract | CrossRef Full Text | Google Scholar

Institute of Education Sciences (2018). Request for Applications: Research Networks Focus on Critical Problems of Policy and Practice in Special Education (CFDA 84.324N). Washington, DC: U.S. Department of Education.

Google Scholar

Kansas State Department of Education [KSDE] (2019). Kansas Education Systems Accreditation (KESA). Available online at: https://www.ksde.org/Agency/Division-of-Learning-Services/Teacher-Licensure-and-Accreditation/K-12-Accreditation-Home/KESA

Google Scholar

Lane, K. L., Carter, E., Jenkins, A., Magill, L., and Germer, K. (2015). Supporting comprehensive, integrated, three-tiered models of prevention in schools: administrators perspectives. J. Posit. Behav. Intervent. 17, 209–222. doi: 10.1177/1098300715578916

CrossRef Full Text | Google Scholar

Lane, K. L., Harris, K., Graham, S., Driscoll, S. A., Sandmel, K., Morphy, P., et al. (2011a). Self-regulated strategy development at tier-2 for second grade students with writing and behavioral difficulties: a randomized control trial. J. Res. Educ. Effect. 4, 332–353. doi: 10.1080/19345747.2011.558987

CrossRef Full Text | Google Scholar

Lane, K. L., Menzies, H., Bruhn, A., and Crnobori, M. (2011b). Managing Challenging Behaviors in Schools: Research-Based Strategies That Work. New York, NY: Guilford Press.

Google Scholar

Lane, K. L., and Menzies, H. M. (2009). Student Risk Screening Scale – Internalizing and Externalizing. Unpublished Rating Scale.

Google Scholar

Lane, K. L., Oakes, W. P., Buckman, M. M., and Lane, K. S. (2018). Supporting school success: engaging lessons to meet students’ multiple needs. Council Childr. Behav. Disord. Newslett. 8, 219–222.

Google Scholar

Lane, K. L., Oakes, W. P., Cantwell, E. D., and Royer, D. J. (2019). Building and Installing Comprehensive, Integrated, Three-Tiered (Ci3T) Models of Prevention: A Practical Guide to Supporting School Success (v1.3). Phoenix: KOI Education.

Google Scholar

Lane, K. L., Oakes, W. P., Carter, E. W., Lambert, W., and Jenkins, A. (2013). Initial evidence for the reliability and validity of the Student Risk Screening Scale for Internalizing and Externalizing Behaviors at the middle school level. Assess. Effect. Intervent. 39, 24–38. doi: 10.1177/1534508413489336

CrossRef Full Text | Google Scholar

Lane, K. L., Oakes, W. P., and Menzies, H. M. (2014). Comprehensive, integrated, three-tiered models of prevention: why does my school—and district—need an integrated approach to meet students’ academic, behavioral, and social needs? Prevent. Sch. Failure 58, 121–128. doi: 10.1080/1045988x.2014.893977

CrossRef Full Text | Google Scholar

Lane, K. L., Oakes, W. P., and Menzies, H. M. (2021a). Considerations for systematic screening PK-12: universal screening for internalizing and externalizing behaviors in the COVID-19 era. Prev. Sch Fail. 65, 275–281. doi: 10.1080/1045988X.2021.1908216

CrossRef Full Text | Google Scholar

Lane, K. L., Oakes, W. P., Royer, D. J., Menzies, H. M., Brunsting, N., Buckman, M. M., et al. (2021b). Secondary teachers’ self-efficacy during initial implementation of comprehensive, integrated, three-tiered models of prevention. J. Posit. Behav. Interv. 23, 232–244. doi: 10.1177/1098300720946628

CrossRef Full Text | Google Scholar

Lindamood, P., and Lindamood, P. (1998). The Lindamood Phoneme Sequencing Program for Reading, Spelling, and Speech. Austin, TX: PRO-ED.

Google Scholar

Losinski, M., Thiele, J., Ennis, R. P., and Shaw, A. (2021). An Investigation of the use of self-regulated strategy development to teach long division to students with or at-risk for emotional disturbance. Educ. Treat. Childr. 44, 169–183. doi: 10.1007/s43494-021-00050-6

CrossRef Full Text | Google Scholar

McIntosh, K., and Goodman, S. (2016). Integrated Multi-Tiered Systems of Support: Blending RTI and PBIS. New York, NY: Guilford Publications.

Google Scholar

McIntosh, K., Mercer, S. H., Hume, A. E., Frank, J. L., Turri, M. G., and Mathews, S. (2013). Factors related to sustained implementation of schoolwide positive behavior support. Except. Children 79, 293–311. doi: 10.1016/j.jsp.2021.02.001

PubMed Abstract | CrossRef Full Text | Google Scholar

Oakes, W. P., Cantwell, E. D., Lane, K. L., Royer, D. J., and Common, E. A. (2020). Examining educators’ views of classroom management and instructional strategies: school-site capacity for supporting students’ behavioral needs. Prevent. Sch. Failure 64, 1–11. doi: 10.1080/1045988x.2018.1523125

CrossRef Full Text | Google Scholar

Penuel, W. R., Fishman, B. J., Yamaguchi, R., and Gallagher, L. P. (2007). What makes professional development effective? Strategies that foster curriculum implementation. Am. Educ. Res. J. 44, 921–958. doi: 10.3102/0002831207308221

CrossRef Full Text | Google Scholar

Poynton, T. A., DeFouw, E. R., and Morizio, L. J. (2019). A systematic review of online response rates in four counseling journals. J. Counsel. Dev. 97, 33–42. doi: 10.1002/jcad.12233

CrossRef Full Text | Google Scholar

Royer, D. J., Lane, K. L., Cantwell, E. D., and Messenger, M. L. (2017). A systematic review of the evidence base for instructional choice in k-12 settings. Behav. Disord. 42, 89–107. doi: 10.1177/0198742916688655

CrossRef Full Text | Google Scholar

Torff, B., and Sessions, D. (2008). Factors associated with teachers’ attitudes about professional development. Teach. Educ. Q. 35, 123–133. doi: 10.1037/t05291-000

CrossRef Full Text | Google Scholar

Umbreit, J., Ferro, J. B., Liaupsin, C. J., and Lane, K. L. (2007). Functional Behavioral Assessment and Function-Based Intervention: An Effective, Practical Approach. Hoboken, NJ: Prentice-Hall.

Google Scholar

Wolf, M. M. (1978). Social validity: the case for subjective measurement or how applied behavior analysis is finding its heart. J. Appl. Behav. Anal. 11, 203–214. doi: 10.1901/jaba.1978.11-203

PubMed Abstract | CrossRef Full Text | Google Scholar

Yudin, M. (2014). PBIS: Providing Opportunity. A Keynote Address Presented at the National PBIS Leadership Forum: PBIS Building Capacity & Partnerships to Enhance Educational Reform. Rosemont, IL: Prentice-Hall.

Google Scholar