Effect of behavior-specific praise on student on-task behavior in career and technical education

Behavior-specific praise (BSP) is a low-intensity strategy used to reinforce positive behaviors, including student academic engagement in school settings. In this study, we leveraged a single training on BSP for a career and technical education (CTE) teacher in a suburban Southeast high school to determine effects on student on-task behavior. Using an A-B-A-B withdrawal design, we observed the teacher’s rate of general praise and BSP statements, and the percentage of 1-min momentary time sampling intervals with on-task behavior of four high school students. The general education CTE teacher increased his rate of BSP, and all four students increased on-task behavior during intervention phases. Social validity questionnaires identified a positive impression from students and teacher about implementing BSP in the classroom. Limitations and future directions are discussed.

Effect of behavior-specific praise on student on-task behavior in career and technical education

A significant contribution to teacher attrition, especially in novice teachers, is disruptive behavior and classroom management concerns (Amitai and Van Houtte, 2022). Within the first 5 years of teaching, about 44% of teachers leave the profession, leading to an increase in underqualified teachers in schools (Gerald, 2019). Teachers report managing student behavior is difficult due to feelings of under-preparedness, poor levels of support by school administration, and competing or inconsistently implemented programs, such as positive behavioral interventions and supports (PBIS) or social and emotional learning (SEL) programs (EAB Global Inc., 2019). Equipping teachers with effective behavior management strategies and providing implementation support through preservice and in-service teacher preparation programs is an important component in building and keeping strong teachers (Flower et al., 2017).

Traditional behavior management approaches, such as reactive or punitive strategies, have been unsuccessful in promoting long-term behavior change (National Collaborating Centre for Mental Health, 2015). This has prompted educators to try other approaches, such as teaching social behaviors and acknowledging appropriate behaviors (Ennis et al., 2019). Among the numerous positive behavior support strategies, behavior-specific praise (BSP) stands out as a compelling means for promoting desirable student behaviors through the behavior analytic principle of positive reinforcement.

BSP is considered a “low-intensity” teacher-delivered strategy, meaning it is easy to implement in terms of preparation time, effort, and cost (Lane et al., 2015). Other examples of low-intensity strategies include active supervision, high-probability request sequences, instructional choice, opportunities to respond, instructional feedback, and pre-correction. Of the low-intensity strategies, BSP is particularly versatile in terms of its ability to be packaged with the other strategies, implemented throughout the day as part of Tier 1 for all students, intensified as a Tier 2 support for a few targeted students, and integrated into Tier 3 interventions to reinforce replacement behavior. BSP is more than feedback or acknowledgment in that it is a more intense response that includes a specific behavioral description component (Ennis et al., 2019). Unlike general praise (e.g., “good job”), BSP provides specific behavioral descriptions, reinforcing desired prosocial and academic behaviors through statements such as, “good job using the wrench safely,” and, “I like the way everyone has their safety goggles on at this station.” Adding specificity helps students to be aware of what they are doing well, increasing the likelihood they (as well as nearby students who hear the praise) will engage in the same behavior in the future.

Based on the Council for Exceptional Children (CEC) (Cook et al., 2014) standards for evidence-based practices in special education, BSP is a potentially evidenced-based practice for increasing on-task behavior and reducing disruptive behavior (Royer et al., 2019). BSP studies demonstrated effectiveness across educational settings, content areas, and participants (Ennis et al., 2019). For the implementation of BSP to be classified as an evidence-based practice however, more studies with three or more participants and 75% of them showing positive results were required per CEC (Cook et al., 2014). Despite the number of studies showing the effectiveness of BSP, too few met this criterion at the time of the systematic literature review by Royer et al. (2019), so further research with an adequate number of participants is needed to determine if BSP can be considered an evidence-based practice across diverse educational contexts.

While there is growing evidence to support the effectiveness of BSP in various general and special education settings (Ennis et al., 2019), few studies explored its impact at the secondary level. O'Handley et al. (2020) explored the use of BSP in middle and high school classrooms and determined delivery of BSP may be an appropriate initial course of intervention when baseline rates of BSP are low and rates of verbal reprimands are high. Yet secondary teachers may be hesitant to use BSP due to inadequate training and misconceptions about the use of BSP with older students (O’Handley et al., 2023). Despite this hesitancy, Haydon and Musti-Rao (2011) found BSP could be successfully implemented by new teachers with 13-year-old students in Grade 7. Though these few studies explored BSP in middle and high school, its application and impact within career and technical education (CTE) environments has yet to be investigated. Maintaining students’ attention and engagement is paramount for achieving learning outcomes in educational environments, especially in the CTE setting. Focused student participation is particularly imperative in the CTE setting, where students are learning and applying technical skills that require heightened precautions to ensure safety (Love and Roy, 2023). In other words, in CTE settings, if students are not on task, they may miss important safety directions and practice opportunities designed to keep them safe.

Purpose

This study aimed to explore the effectiveness of BSP in a high school CTE classroom. Specifically, we sought to increase student on-task behavior by increasing teacher rate of BSP in a CTE classroom following a single professional learning session. Results will contribute to the literature base to help determine if BSP can be categorized as an evidence-based practice for increasing on-task behavior. We focused on a single professional learning session due to the majority of previous BSP studies investigating the impact of on-going coaching and performance feedback on the rate of teacher BSP as the dependent variable (Ennis et al., 2020), whereas we were interested specifically in the effects of teacher-delivered BSP on student on-task behavior as the dependent variable. To that end, our specific research questions were:

1. To what extent will the rate of BSP change for a CTE teacher during automotive class time due to a single training session?

2. To what extent will the rate of on-task behavior for students change due to teacher implementation of BSP in a CTE classroom?

3. How do teacher and student views of the BSP intervention goals, procedures, and outcomes change when comparing pre- to post-implementation?

Method

Setting and participants

The study occurred at the CTE school sharing a wing of the River Valley High School (RVHS; all names and places pseudonyms) building, a suburban school in the southeast United States. The CTE school is a separate school from RVHS with its own administration and a close relationship with RVHS with the CTE programs made available exclusively to the 849 high school students across grades 9–12. The CTE participation for RVHS is 39.5% (National Center for Education Statistics, 2023; State Department of Education, 2024). Demographically, 818 students were White (96.4%), 14 students identified as two or more races (1.6%), ten were Latinx (1.2%), four were Black (0.5%), and three were Asian/Pacific Islander (0.3%). There were 526 (62.0%) students eligible for free- or reduced-price lunch and 79 (9.3%) students received special education services. See Table 1 for additional demographic information. The master schedule consisted of seven 50-min class periods. RVHS implemented PBIS schoolwide, with the broad expectations of always working safely, being respectful, being responsible, and focusing on equipping oneself with certifications for postsecondary readiness. Schoolwide posters were the only evidence available of PBIS implementation.

Table 1

Table 1. School characteristics.

The study took place across two automotive class periods; we observed the last 15 min of second period (not including clean-up time) and the first 15 min of third period (not including the initial transition and orientation). There were 18 students in the second-period class, two with individualized education programs (IEPs; including Mateo), where Mateo and Jack were observed. The third-period class had 15 students, four with IEPs, where Mia and Ethan were observed. The automotive classroom was approximately 17 m × 16 m with a high 9 m ceiling. The main learning space included an area for traditional learning, including a desk and chair for each student. Students had one-to-one Chromebook access and utilized them in the desk area for independent work. Independent work generally included computer-based learning modules where students learned about automotive safety, parts of the car, and how various car parts operate together. Other spaces throughout the room included stations for hands-on learning that presented similarly to an automotive repair workshop, including various parts of cars. Students could take these car parts apart and put them together. The room had two hydraulic car lifts on each side of the student desk area. A tool room and a small teacher workstation were located at the front of the room. See Figure 1 for automotive classroom layouts.

Figure 1

Figure 1. Images of automotive classroom layout.

One high school teacher and four students participated in this study (all names and places are pseudonyms). Mr. Richardson was a 56-year-old White male automotive teacher with more than 20 years of experience working in automotive shops in the community and 3 years of experience teaching at the CTE school (see Table 2 demographics). Mr. Richardson obtained a provisional teaching certificate through the occupation-based route, meaning he fulfilled the state’s requirements of graduating high school, obtaining four successful and appropriate, recent occupational years of experience in the area of certification. In doing so, he was able to be a credentialed teacher in auto technology.

Table 2

Table 2. Characteristics of teacher participant.

Mateo was a 16-year-old Hispanic male in grade nine. He had an IEP and received special education services under the federal eligibility category of intellectual disability (state eligibility category = mild mental disability). Mateo was at moderate risk for externalizing and high risk for internalizing behavior patterns according to the Student Risk Screening Scale—Internalizing and Externalizing (SRSS-IE) (Lane and Menzies, 2009). Jack was a 16-year-old White male in grade 11 who was at a high risk for internalizing on the SRSS-IE. Mia was a 17-year-old female of two or more races who was repeating grade nine. She was at a high risk for both externalizing and internalizing on the SRSS-IE. Ethan was a 16-year-old White male in grade 11 and was at a moderate risk for both externalizing and internalizing on the SRSS-IE.

Measures

Direct observation

We measured the frequency of general and BSP statements by direct observation for 15 min in each of two class periods daily Monday through Friday for 7 weeks. Spring break occurred toward the end of the withdrawal condition for one full week, and some weeks had days where observation did not occur (e.g., data collectors unavailable, teacher absent) but each week had at least three observation days except the final week. The observations occurred at approximately the same time of each class period, about 10 min into each period. We divided the 15-min observations into 1-min intervals for more robust interobserver agreement (IOA) frequency data analysis, recording within each interval the number of general and BSP statements delivered by the teacher. We defined general praise as a positive statement of approval that was more than acknowledgment (e.g., “That is correct,” “Yes”) and does not specify the specific, observable, or effective behavior (Zoder-Martell et al., 2019). Examples of general praise included, “Well done,” and “Good work!” Nonexamples included, “Thank you for a quiet transition, Etta,” and, “Good job having your materials ready.”

BSP was operationally defined as positive statements of approval with a specific behavioral description component (Ennis et al., 2019). Examples included, “Great job using the tool safely,” and, “Thank you for putting your tools away.” Nonexamples included, “Good job, Nathan,” and, “Nice work, Jayden.” Additionally, nonexamples of both general and BSP included reprimands such as, “Get your safety glasses,” “Put your earbuds/headphones away,” and, “Get to work.” Acknowledgments were also non-examples for both categories of praise, such as, “You have the right tool,” and “You did the task.”

We measured student on-task behavior using 1-min momentary time sampling during the same 15 min observation. We observed the four participating students at the end of each interval to determine if they were on- or off-task, marking the respective column on the data collection sheet for the corresponding interval (see Supplementary Figure S2). We defined student on-task behavior as working independently on subject matter or directing their attention to the teacher and/or materials. Examples included engaging in computer modules at their desk, utilizing tools to complete tasks in the engine lab, or recording steps of the task for the group. Nonexamples included having head down on the desk, looking away from materials for more than 3 s, talking off-topic, or using technology for anything other than the teacher-directed task.

The observers included the first author, a doctoral student in special education, and an RVHS special education liaison who worked to support students participating in the special education program in the CTE setting. The first author trained the second observer on direct observation procedures and calibrated the operational definitions applicable to the observation (e.g., on-task, general praise, BSP) with him. The two observers conducted an initial observation and calculated IOA to be 94.3% for all dependent variables, comparing interval-by-interval. This included exact count for frequency of teacher general and BSP within each 1-min interval, and student on- or off-task behavior at the end of each interval. The number of agreements was divided by the sum of agreements and disagreements and multiplied by 100 to obtain a percentage (Cooper et al., 2020). We discussed disagreements and refined operational definitions to improve future IOA. A secondary data collector observed 25.0% of sessions, twice in withdrawal and once in other conditions. We planned to retrain when IOA was <80% for any one observation, but retraining was not required. The mean IOA across conditions for student on-task behavior was 90.4% (range = 87.6–93.1%), BSP was 98.0% (range = 92.0–100.0%), and general praise was 90.2% (range = 64.0–100.0%; see Table 3).

Table 3

Table 3. Procedural fidelity and dependent variable interobserver agreement results by condition.

Social validity

An adapted version of the Children’s Intervention Rating Profile (CIRP; Witt and Elliott, 1985; available at ci3t.org/measures) included seven items rated by the student, asking if the intervention was fair, helpful, acceptable, appropriate, or likely to cause problems with friends. Items were rated on a 6-point Likert-type scale (some were reverse scored) where 1 = strongly disagree and 6 = strongly agree. Example items included, “I think I will like being in this program,” and “I think being in this program will help me do better in school.” Total scores (sum) ranged from 7 to 42, with higher scores indicating higher social validity. Internal consistency of the CIRP in previous studies ranged from 0.75 to 0.89 (Carter, 2007).

An adapted version of the Intervention Rating Profile (IRP-15; Witt and Elliott, 1985; available at ci3t.org/measures) contained 15 questions asking adults whether the intervention was acceptable, fair, appropriate, compatible, and effective. Items were rated on a 6-point Likert-type scale ranging from 1 = strongly disagree to 6 = strongly agree. Example items included, “This intervention would not result in negative side effects for the child,” and “This intervention would be appropriate for a variety of children.” Total scores (sum) ranged from 15 to 90, with higher scores indicating higher social validity. Internal consistency was reported in previous studies to be 0.98 (Common and Lane, 2017).

Procedural Fidelity

We utilized a researcher-created procedural fidelity checklist (see Supplementary Figure S3) across 70% of sessions for all conditions, with a secondary observer completing the same checklist for 25% of all sessions, once or twice per condition (see Table 3). The checklist included five items, with three reflecting observer behaviors (e.g., “Observer did not interact with students”) and two items reflecting teacher behaviors with one item reverse scored during baseline conditions (e.g., “Teacher utilized BSP at least 3x per 15-min observation”). Items were scored as 0 = not implemented, 1 = partially implemented, and 2 = fully implemented. Fidelity scores were summed each time collected, divided by the total points possible (10), and multiplied by 100 to obtain a percentage. Average fidelity across all conditions was 98.8% (range = 80.0–100%) and IOA for procedural fidelity was 100%.

Procedures

We first obtained university IRB and district approvals, then approached the CTE principal for permission and teacher nominations, whom we then screened and obtained informed consent. The principal recommended Mr. Richardson for the study based on his readiness for classroom management support. The lead researcher confirmed he met inclusion criteria: had certification in the current area taught, had over 90.0% attendance, had not previously participated in BSP training, and had low levels of BSP delivery during a 15-min screening observation (fewer than 3 BSP).

Mr. Richardson then completed the IRP-15 and reviewed student data with us for possible inclusion, including completing the SRSS-IE at the request of the research team to help identify potential participants for the study. Student inclusion criteria were (a) moderate or high risk in one of both domains of the SRSS-IE, (b) 75.0% or more attendance, and (c) were off-task at least five times by frequency count during a 15-min screening observation. Four students met these inclusion criteria and we obtained informed parent consent before we approached students and obtained their assent and completion of the CIRP.

During the screening observation, we identified the classroom was very loud due to environmental sounds, such as compressors, fans, and other equipment. Therefore, hearing Mr. Richardson’s interactions with students was not possible. To hear Mr. Richardson, we provided him with a Bluetooth lapel microphone and established a Zoom online call. One or both data collectors joined the Zoom session and were able to hear Mr. Richardson’s interactions through Apple AirPod headphones and record accurate frequency counts for general and BSP statements.

Baseline

When students arrived at class, they were expected to be at their desks by the time the tardy bell rang. After taking attendance, the teacher talked to the students about the plan and format for the class that day. Most days, students were expected to work in the engine lab, where they were to complete a task related to taking a portion of the engine apart. Students who still needed to complete their computer-based modules were to work toward achieving them on their Chromebooks at their desks. Students engaging in the lab setting were expected to obtain safety gear first. Once expectations were confirmed by students’ verbal or visual acknowledgment, students reported to the appropriate area. Of the 50-min class period, students worked for the first 40 min and used the last 10 min to put away materials and prepare for class transition. The teacher circulated throughout the room and provided instruction and feedback to students in both areas. The teacher generally managed off-task behavior with multiple verbal redirections. This class format was the same for both second- and third-period classes.

During the initial baseline phase, Mr. Richardson was unaware of the intervention he was going to receive training for except that it was considered a “low-intensity behavior strategy.” To collect baseline data, the observer entered the room quietly, sat or stood in a discreet location, and did not interact with students. The teacher was directed to keep his current instructional practices during baseline. The observer collected data on the delivery of general and BSP statements via frequency count and student on-task behavior via momentary time sampling, as described in the Measures section.

Intervention training

After 5 days of baseline data collection and visual analysis revealed variable data or countertherapeutic trends, it was decided to train the teacher and begin intervention. The first author trained the teacher on BSP during his planning period in the conference room for 30 min using a training fidelity checklist (see Supplementary Figure S1). The trainer greeted Mr. Richardson with a BSP upon entering to model the practice (i.e., “Thank you for attending this training”), introduced the rationale of BSP with a computer-based slideshow presentation, provided examples of BSP and general praise with time to practice differentiating, and had the teacher generate his own BSP based on a picture prompt. Three checks for understanding were embedded throughout the training, on which Mr. Robinson needed to score at least 90% to continue (M = 100%). The trainer explained the progression of the study, such as to begin using BSP today, how data collectors would continue to observe as in baseline, and how phase changes would occur based on stable teacher and student responding. We planned to retrain the teacher one time if his rate of BSP during either intervention condition was less than 0.2 per min for two consecutive observations, but this was not needed.

To ensure training was delivered as designed and consistent across studies at other districts (see Newton et al., 2024; Royer et al., 2024), we reviewed a training fidelity checklist prior to and after the teacher training. Example items included “agenda presented,” “trainer modeled BSP,” and “trainer administered three checks for understanding,” scored as a binary yes or no. We counted the number of yes items, divided by the total number of items, and multiplied by 100 to get a training fidelity percentage, which was 100%.

Intervention

During intervention conditions, students followed the same procedures as in baseline, completing computer-based automotive learning modules or working directly in the labs with the car parts. Mr. Robinson provided instruction and assistance to students in both contexts but now delivered BSP statements throughout the session for students’ on-task behaviors such as, “Thank you for getting your module completed,” and, “Great job getting your safety glasses first.” The teacher delivered BSP and general praise statements to all students in each class, not just the target students. We did not provide any performance feedback to the teacher.

Withdrawal and reintroduction of BSP

After 5 days of intervention showing less variability in the data and some students showing a therapeutic trend, the decision was made to withdraw the intervention given the approach of state testing and the end of the school year. The first author met face-to-face with Mr. Robinson and instructed him to withdraw the intervention and return to pre-training praise rates. When most students’ on-task behavior and the teacher’s rate of general and BSP stabilized at near-baseline levels after 5 days, the first author instructed the teacher to reintroduce BSP for the final intervention condition, which also lasted 5 days.

Post-intervention

After the final intervention day, the lead researcher showed graphs of student on-task behavior and teacher rates of general and BSP to the teacher and students. The teacher then completed the post-intervention IRP-15 and the four students completed the post-intervention CIRP.

Experimental design and data analytic plan

We used an A-B-A-B withdrawal design to test the effects of a single BSP teacher training session on the rate of teacher-delivered BSP and the subsequent effects of BSP on student on-task behavior. The decision to begin intervention was made based on the stability of baseline data for the independent variables (frequency count of general and BSP) and dependent variable (percentage of intervals of on-task behavior for four students in the high school automotive class during second and third periods) determined with visual analysis of the data in a line graph and taking into consideration counter therapeutic trends (Lane and Gast, 2014). Similarly, the decisions to change conditions for withdrawal and reintroduction of the BSP intervention were made based on data variability and trend determined with visual analysis, balanced with constraints of doing research in an applied school setting such as approaching state testing and the end of the school year.

We analyzed data through descriptive statistics (e.g., mean, range, percentage) for social validity (pre- to post-implementation comparison) and procedural fidelity (see Table 3), and visual analysis of teacher general and BSP and student on-task behavior (level, trend, and stability within and between conditions, plus overlap and determination of functional relation; Lane and Gast, 2014). Last, we calculated the between-case standardized mean difference (BC-SMD) (Valentine et al., 2016) estimate using version 0.7.2 of the online calculator at jepusto.shinyapps.io/scdhlm to determine the BSP intervention’s effect size on student on-task behavior. The BC-SMD is comparable with Cohen’s d effect size for group comparison research and can be interpreted as small (0.20–0.50), medium (0.50–0.80), and large (≥ 0.80) (Fritz et al., 2012).

Results

This study sought to determine the impact of BSP on student on-task behavior in an automotive class in a career and technical education high school. We trained the teacher on BSP with a single 30-min session and used an A-B-A-B withdrawal design over 7 weeks to test the effects of the BSP intervention. Mia was absent from class 1 day due to receiving in-school suspension during B₁ and Ethan was absent 5 days due to illness but was present for at least three observations per condition.

Research question no. 1: teacher participant outcomes

During second period (Figure 2A), general praise averaged near zero in baseline, being present only in Session 1 (M = 0.01/min, range = 0.00–0.07/min) and no BSP statements delivered. General praise increased in implementation to an average 0.24/min (range = 0.00–0.33/min) and BSP immediately and dramatically increased, averaging 0.25/min (range = 0.13–0.33/min). Both general and BSP returned to near baseline levels during withdrawal (general M = 0.03/min, range = 0.00–0.13/min; no BSP statements). When the BSP intervention was reintroduced, general praise increased immediately and averaged 0.17/min (range = 0.00–0.27/min), with BSP similarly increasing to an average of 0.19/min (range = 0.00–0.27/min).

Figure 2

Figure 2. Rate of behavior-specific praise and general praise statements per min during 15-min observation for Mr. Richardson across second period (A) and third period (B) automotive class.

During third period (Figure 2B), general praise averaged near zero in baseline, being present only in Sessions 3 and 4 (M = 0.04/min, range = 0.00–0.13/min), and no BSP statements were delivered. General praise increased in implementation to 0.25/min (range = 0.07–0.40/min), and BSP immediately and dramatically increased, averaging 0.20/min (range = 0.13–0.27/min). Both general and BSP returned to near baseline levels during withdrawal (general M = 0.04/min, range = 0.00–0.07/min; BSP M = 0.01/min, range = 0.00–0.07/min). When the BSP intervention was reintroduced, general praise increased immediately and averaged 0.20/min (all sessions had a rate of 0.20/min), with BSP similarly increasing to an average of 0.17/min (range = 0.07–0.20/min).

Research question no. 2: student participant outcomes

Mateo and Jack participated in the second period class. The average of Mateo’s on-task behavior (Figure 3A) during baseline was 28.0% of intervals (range = 0–53.3%) with a downward trend. In the first intervention phase, his on-task behavior immediately and dramatically increased from 20.0% in baseline to 90.0%, stayed above baseline levels with zero overlap, and averaged 74.0% of intervals (range = 60–90%). During the withdrawal condition, Mateo’s on-task behaviors immediately and dramatically decreased from 86.7% at the end of B₁ to 6.7% and stayed below baseline with an average of 17.3% of intervals (range = 0–53.3%). With the reintroduction of the intervention, Mateo had 2 days of low on-task behavior overlap with withdrawal before dramatically increasing to 93.3% with a downward trend. Mateo’s on-task behavior in B₂ averaged 56.0% (range = 6.7–93.3%), overall demonstrating a functional relation.

Figure 3

Figure 3. Percentage of 1-min momentary time sampling intervals of on-task behavior per 15-min observation across baseline and behavior-specific praise conditions for Mateo (A), Jack (B), Mia (C), and Ethan (D).

Jack’s on-task behaviors (Figure 3B) during baseline had a steep increasing trend with an average of 38.7% of intervals (range = 0–73.3%). During the intervention phase, his on-task behavior increased to an average 79.3% (range 53.3–100%), with an immediate and large jump between the last baseline data point (26.7%) and the first intervention data point (90.0%), which then dropped to 53.3% before a steep trend up to 100% on task. His on-task behaviors decreased when the intervention was withdrawn, to an average of 30.2% (range = 0.00–53.3%) and with a downward trend. When the intervention was reintroduced, Jack’s on-task behavior significantly and immediately increased, with the final four data points all above 90% on task, averaging 92.0% (range = 66.7–100%), also demonstrating a functional relation.

During the third-period class, Mia and Ethan were observed. The average of Mia’s on-task behavior (Figure 3C) during baseline was 58.3% with extreme variability ranging from 0.0–100% of intervals and ending at 0.0%. When the BSP intervention was introduced, her on-task behavior increased only slightly from the previous day’s 0.0%, then jumped up to average 68.9% for the next three observations, and then dipped back down to 6.7%, with an average of 44.0% of intervals (range = 6.7–73.3%) across the condition. During the withdrawal condition, Mia’s on-task behaviors decreased to below baseline with high variability with an average of 33.3% (range = 0–93.3%). With the reintroduction of the intervention, Mia’s on-task behaviors increased to an average of 76.0% of intervals (range = 26.7–100%).

Ethan’s on-task behavior (Figure 3D) during baseline averaged 68.9% of intervals (range = 46.7–93.3%). During the intervention phase, there was a lot of variability in his on-task behavior, which decreased to 53.3% (range 6.7–93.3%). His on-task behaviors decreased when the intervention was withdrawn (M = 37.2%, range = 15.4–60.0%). When the intervention was reintroduced, Ethan’s on-task behaviors increased to an average of 58.1% of intervals (range = 13.3–92.3%), with his final two sessions being high at an average of 89.5%. While both Mia and Ethan demonstrated some positive changes in on-task behavior, results were too variable and inconsistent to establish a functional relation between the introduction of BSP and effects on on-task behavior.

The BC-SMD estimate for student on-task behavior was 0.89 with a standard error 0.40 and 95% CI of (0.08, 1.71). This represents a large effect of BSP on student on-task behavior.

Research question no. 3: social validity

Mr. Richardson’s pre-intervention IRP-15 was 79 out of 90, which increased by four at post-intervention to 83, indicating intervention expectations were exceeded. The social validity ratings for the CIRP also revealed positive results. Student pre-intervention scores ranged from 25 to 40.83 out of 42 (M = 32.21) and at post-intervention ranged from 30 to 40 (M = 36.50), an average increase of 4.29, indicating the BSP intervention exceeded student expectations. Mateo omitted one question (“There are better ways to teach me”) on the pre-intervention CIRP, making his score of 35 out of a possible 36. To more accurately compare the pre- and post- intervention CIRP across all students, we substituted Mateo’s average 5.83 for his missing item, making his pre-intervention score 40.83 out of 42. With this mean substitution for the missing data, his post-intervention score of 40 reflected a slight decrease of 0.83. See Table 4 for individual adult and student social validity results.

Table 4

Table 4. Social validity results by participant and condition.

Discussion

This study explored the effectiveness of BSP in a high school CTE classroom to increase student on-task behavior. Our research questions first examined treatment integrity to ensure the BSP intervention was put in place by the teacher, then examined student on-task behavior outcomes, and how the teacher and students viewed the BSP intervention goals, procedures, and outcomes (social validity). For Mr. Richardson’s social validity, before beginning this study he communicated it was difficult to manage students and keep them on task. After the study concluded and student graphs were shared, Mr. Richardson’s IRP-15 positive increased score gives the impression he found the low-intensity BSP strategy intervention helpful in supporting on-task behavior. Interestingly, Mr. Richardson met the procedural fidelity indicator of three BSP per 15-min observation (0.2/min) during 4 out of 5 intervention sessions (85%) by delivering them in a cluster within the last five intervals of data collection toward the end of each class period. BSP should ideally be immediate and contingent (Ennis et al., 2018), so it is plausible that on-task behavior might have been even higher if BSP was delivered evenly throughout the class periods while students were working. With the teacher delaying BSP for work done earlier in the class and generally positive trends in student data (though variable for Mia and Ethan), we can see that BSP can still be effective, even if delayed. Mr. Richardson communicated to the researcher he believed this intervention was easy to implement as it was something he believed he was already doing, even though the baseline data indicated only general praise was present.

All four students displayed increased on-task behavior and three out of four scored highly positive impressions on the CIRP, similar to Mr. Robinson. Only one student’s CIRP score went down by about one point, but it was high to begin and remained high, showing expectations of the BSP strategy were met or exceeded by students. Though students engaged in different activities within and across observation sessions, such as computer-based learning modules or engine lab work, there was not an identifiable difference in student on-task behavior across various classroom activities. It appeared the effectiveness of the BSP strategy will depend more on how it is implemented by the teacher as well as student characteristics (e.g., does the student find attention reinforcing) and less on what classroom tasks are assigned.

In this study BSP appeared to work best for Mateo and Jack where a functional relation was demonstrated between introduction of BSP and changes in student on-task behavior. Changes in on-task behavior for Mia and Ethan were more variable and did not always have immediacy of change when BSP was introduced or withdrawn. While this could be because of differences in class period (Mateo and Jack were in a different period), anecdotal observation by data collectors indicated minimal if any differences in instructional delivery or classroom procedures between periods, and Mr. Robinson’s rate of BSP was very similar in each (i.e., slightly higher in second period for B₁ and slightly lower in B₂). Differences in student responding might therefore be explained by Mia and Ethan’s off-task behavior having function other than positive reinforcement: attention and thus BSP given to replacement on-task behavior might not have been reinforcing. We did not conduct a functional analysis for the target students to know what the function of their off-task behavior was, so even though Mia and Ethan’s behavior screener indicated similar levels of risk on the externalizing and internalizing subscales of the SRSS-IE compared to Mateo and Jack, the function of their off-task behavior might have been negative reinforcement: attention or positive reinforcement: tangible/activity (Umbreit et al., 2024). Meaning, BSP statements for on-task behavior, which were intended to reinforce on-task behavior, might have been a mismatch to the actual function of Mia and Ethan’s behavior and a different low-intensity strategy might have been more reinforcing. If BSP attention was actually, potentially, reinforcing for Mia and Ethan, perhaps it needed to be immediate and directly delivered for it to influence on-task behavior more consistently compared to the generally positive but variable results observed. The overall average effect size of 0.89, a large effect size, shows BSP has the potential to be an effective strategy for educators but effects may vary across students, as this study showed. Other factors, such as attendance (Eklund et al., 2022), past trauma exposure (Fondren et al., 2020), function of behaviors (Cooper et al., 2020), and fidelity of implementation (Gage et al., 2020) may impact results for various students. Further, educators should use data to analyze students’ behaviors, such as conducting a functional behavior assessment (FBA) and gathering student input on reinforcers, to then select the most effective strategies (Umbreit et al., 2024).

Behavior-specific praise is a low-intensity strategy, meaning it is free, quick, and easy to implement to support student behavior (Ennis et al., 2018; Lane et al., 2015). Therefore, the value of this intervention is noteworthy for teachers who seek to promote positive and productive behaviors in the classroom without needing extra materials or additional planning time. With minimal general praise and BSP statements during baseline, surprisingly small amounts of BSP can increase student on-task behavior (cf. Newton et al., 2024). While the strategy may not work for all students, results of this BSP study further support its potential as a low-intensity and high-yield strategy.

In addition to using BSP to increase student on-task behavior, there are benefits for teachers. The toll of negative student behavior can have a physical and/or emotional effect on a teacher’s job satisfaction (Buckman and Pittman, 2021). Shifting the classroom culture to be more positively focused (reducing student behavior challenges) may have an inverse effect on teachers (increasing classroom self-efficacy and job satisfaction). Shifting away from reactive and punishment-based approaches to classroom management toward a proactive prevention-based approach utilizing high-rates of BSP can cultivate a more positive environment because teachers’ priorities shift to noticing positive and on-task behaviors (while ignoring minor off-task behavior) and hearing peers receive BSP encourages all students to self-reflect on their behavior choices (Newton et al., 2024).

Mr. Richardson spent more than 20 years working in automotive shops in the community and entered the classroom setting about 3 years ago to share his skills and knowledge with students. Yet his teacher preparation experience through an alternate pathway may not have prepared him with classroom management strategies or how to prevent challenging behavior. Before the training, Mr. Richardson either tolerated off-task behavior or verbally redirected students. In anecdotal conversation, he expressed frustration with students’ lack of engagement, indicating the need for more behavior management strategies. Although being an automotive teacher represents a unique role in education, Mr. Richardson is not alone in his alternate route to licensure. The traditional route to teacher licensure generally consists of education and training in a university undergraduate teacher preparation program. Alternate routes generally refer to other training programs that result in teacher licensure, such as university programs for post-baccalaureate students (non-degree, licensure only) or master’s degree students (graduate degree programs with licensure embedded), non-university-based programs, Teach for America,¹ iteach,² and AmeriCorps³ Inspired Teacher Certification Program. From 2012 to 2020, about 23% of special education teachers obtained their teacher certification through nontraditional means (Day et al., 2024). Teachers like Mr. Richardson following an alternative route to teacher certification may benefit from learning low-intensity strategies like BSP to support academic engagement and prevent challenging behavior, especially because training can be delivered in short sessions (e.g., our 30-min professional learning). Such trainings can be provided by district or school staff at faculty meetings to supplement what was provided during teacher preparation coursework. More studies such as this one are needed, however, to determine if Mr. Richardson’s experiences with delivering BSPs may be improved upon and generalizable to other alternate certification teachers and CTE teachers in other areas, such as electricity, nursing, or agriculture.

Limitations and future opportunities

It is important to unpack the limitations that may influence the interpretation and generalizability of our findings in analyzing this study. First, during the baseline phase, the teacher demonstrated nearly nonexistent rates of both general praise and BSP statements. Subsequently, during the intervention phases, the teacher demonstrated increased rates of delivery of both types of praise statements. Therefore, it is difficult to determine whether the increases in student on-task behavior were a result of the delivery of general praise statements, BSP statements, or a combination of the two. Future researchers might design a study to control for general praise remaining the same across conditions, or employ a comparison design (e.g., A-B-C-B-C) to introduce higher rates of general praise (B condition) compared to BSP (C condition).

Second, Mr. Richardson’s role as an automotive teacher at a CTE school is unique in the field of education, meaning most schools do not have many, if any, of this type of teacher on staff. Therefore, there may be unique factors to automotive CTE teachers and classrooms we are not aware of that could influence study results, as well as the generalizability of this study to any context other than automotive teachers in a CTE school. Replication studies are needed to determine how generalizable this BSP intervention is to automotive and other CTE teachers. Replications should include other teacher and student participants, settings (i.e., rural, urban), and grades (i.e., middle school, elementary school).

The third limitation is that we did not consider the function of students’ behaviors or the dosage of the BSP intervention per student. As evidenced by the variation across students’ on-task behavior percentage of intervals, it can be inferred that BSP impacts students differently depending on (a) if attention is reinforcing to their on-task behavior and perhaps (b) how much attention it takes to impact behavior. Despite meeting the minimum expectation for BSP dosage, Mr. Robinson often delivered approximately two BSPs in a cluster toward the end of the class (minimum was three BSP statements for each 15-min observation to achieve 0.2/min). Future research may include a functional analysis or pre-intervention survey of students to analyze what they find reinforcing to then target the delivery and dosage of the intervention more intentionally (Hanratty and Hanley, 2021). If students do not find attention reinforcing, a BSP intervention likely will not result in desired shifts in dependent variables and interventionists should consider other low-intensity strategies such as instructional choice (Royer et al., 2017).

A fourth possible limitation could be the presence of the Hawthorne effect, where behavior changes simply in the presence of observers (Ledford and Gast, 2024). During a pre-baseline observation, it was noted that due to the environmental brown noise, the teacher’s verbal interactions were difficult for the observers to hear. Therefore, the teacher agreed to wear a small lapel microphone that allowed the observers to hear his interactions through headphones. The students were observant of this change and had curiosities around the microphone. The target students, however, did not comment about the microphone at any point during the study. Additionally, the teacher’s behaviors might have been impacted by the observers, though they always entered the classroom quietly and did not interact or engage with the students or teacher.

A final limitation to note is that we were unable to collect any maintenance or follow-up data to determine if Mr. Richardson continued implementing BSP (continued use a sign of social validity) and if student on-task behavior continued to improve in both periods. Given the study occurred at the end of spring semester and additional scheduling barriers, maintenance data could not be collected. Future researchers should endeavor to start classroom studies sooner in the academic year and include a plan for collecting maintenance or follow-up data.

Conclusion

This study adds support to the literature on the positive impact of BSP delivery, with understand that each student may respond differently to the strategy. Mr. Richardson increased his rate of BSP after a single 30-min training session and all four students increased their on-task behavior (some with high variability) during intervention phases. Future studies might consider providing performance feedback as well to help increase teacher BSP rates even more, such as by coaching the teacher to provide BSP throughout class periods immediately when desired behaviors are observed (versus delaying BSP deliver to the end of the period). By empowering teachers with low-intensity strategies such as BSP, teachers can make their repertoire of effective practices more robust, increasing classroom self-efficacy and strengthening themselves as professionals. As a result of teacher delivery of low-intensity strategies, students can experience a more positive learning environment and receive higher rates of reinforcement for the positive academic and prosocial behaviors they demonstrate.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving humans were approved by University of Louisville Institutional Review Board. The studies were conducted in accordance with the local legislation and institutional requirements. Written informed consent for participation in this study was provided by the participants’ legal guardians/next of kin. Written informed consent was obtained from the individual(s) for the publication of any potentially identifiable images or data included in this article.

Author contributions

JH: Writing – original draft, Writing – review & editing. DR: Writing – original draft, Writing – review & editing. NN: Writing – review & editing. AP: Writing – review & editing.

Funding

The author(s) declare that financial support was received for the research, authorship, and/or publication of this article. This research was funded in part by Project SPIDERS (School-University Partnerships Influencing aDvocacy and Engaging Rural Special educators), a U.S. Department of Education Office of Special Education Programs grant (H325D210042), and the Gheens Community Engagement Mini Grant Program at University of Louisville.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The handling editor MR declared a past co-authorship with the author DR.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/feduc.2024.1441739/full#supplementary-material

Footnotes

1. ^teachforamerica.org

2. ^iteach.net

3. ^americorps.gov

References

Amitai, A., and Van Houtte, M. (2022). Being pushed out of the career: former teachers’ reasons for leaving the profession. Teach. Teach. Educ. 110, 103540–103512. doi: 10.1016/j.tate.2021.103540

Crossref Full Text | Google Scholar

Buckman, D. G., and Pittman, J. T. (2021). Student discipline and teacher job satisfaction: a dual district analysis. Georgia Educ. Res. 18, 24–51. doi: 10.20429/ger.2021.180202

Crossref Full Text | Google Scholar

Carter, S. L. (2007). Review of recent treatment acceptability research. Educ. Train. Dev. Disabil. 42, 301–316.

Google Scholar

Common, E. A., and Lane, K. L. (2017). “Social validity assessment” in Applied behavior analysis advanced guidebook: A manual for professional practice. ed. J. K. Luiselli (Elsevier Academic Press), 73–92.

Google Scholar

Cook, B., Buysse, V., Klinger, J., Landrum, T., McWilliam, R., Tankersley, M., et al. (2014). Council for Exceptional Children: standards for evidenced-based practices in special education. Teach. Except. Child. 46, 206–212. doi: 10.1177/0040059914531389

Crossref Full Text | Google Scholar

Cooper, J. O., Heron, T. E., and Heward, W. L. (2020). Applied behavior analysis. 3rd Edn: Pearson.

Google Scholar

Day, J., Nagro, S. A., and Mason-Williams, L. (2024). The nationwide trends and preparation requirements of alternative route programs in special education. Teach. Educ. Spec. Educ. 47, 93–109. doi: 10.1177/08884064231172525

Crossref Full Text | Google Scholar

EAB Global Inc. (2019). Breaking bad behavior: the rise of classroom disruptions in early grades and how district are responding. Education Advisory Board.

Google Scholar

Eklund, K., Burns, M. K., Oyen, K., DeMarchena, S., and McCollom, E. M. (2022). Addressing chronic absenteeism in schools: a meta-analysis of evidence-based interventions. Sch. Psychol. Rev. 51, 95–111. doi: 10.1080/2372966X.2020.1789436

Crossref Full Text | Google Scholar

Ennis, R. P., Royer, D. J., Lane, K. L., and Dunlap, K. D. (2019). Behavior-specific praise in pre-K–12 settings: mapping the 50-year knowledge base. Behav. Disord. 45, 131–147. doi: 10.1177/0198742919843075

Crossref Full Text | Google Scholar

Ennis, R. P., Royer, D. J., Lane, K. L., and Dunlap, K. D. (2020). The impact of coaching on teacher-delivered behavior-specific praise in pre-K-12 settings: a systematic review. Behav. Disord. 45, 148–166. doi: 10.1177/0198742919839221

Crossref Full Text | Google Scholar

Ennis, R. P., Royer, D. J., Lane, K. L., Menzies, H. M., Oakes, W. P., and Schellman, L. E. (2018). Behavior-specific praise: an effective, efficient, low-intensity strategy to support student success. Beyond Behav. 27, 134–139. doi: 10.1177/1074295618798587

Crossref Full Text | Google Scholar

Flower, A., McKenna, J. W., and Haring, C. D. (2017). Behavior and classroom management: are teacher preparation programs really preparing our teachers? Prev. Sch. Fail. 61, 163–169. doi: 10.1080/1045988X.2016.1231109

Crossref Full Text | Google Scholar

Fondren, K., Lawson, M., Speidel, R., McDonnell, C. G., and Valentino, K. (2020). Buffering the effects of childhood trauma within the school setting: a systematic review of trauma-informed and trauma-responsive interventions among trauma-affected youth. Child Youth Serv. Rev. 109, 104691–104618. doi: 10.1016/j.childyouth.2019.104691

Crossref Full Text | Google Scholar

Fritz, C. O., Morris, P. E., and Richler, J. J. (2012). Effect size estimates: current use, calculations, and interpretation. J. Exp. Psychol. Gen. 141, 2–18. doi: 10.1037/a0024338

Crossref Full Text | Google Scholar

Gage, N., MacSuga-Gage, A., and Detrich, R. (2020). Fidelity of implementation in educational research and practice. Available at: https://www.winginstitute.org/systems-program-fidelity (Accessed November 15, 2024).

Google Scholar

Gerald, V. (2019). Teacher retention: A growing problem. National School Boards Association.

Google Scholar

Hanratty, L. A., and Hanley, G. P. (2021). A preference analysis of reinforcer variation and choice. J. Appl. Behav. Anal. 54, 1062–1074. doi: 10.1002/jaba.835

PubMed Abstract | Crossref Full Text | Google Scholar

Haydon, T., and Musti-Rao, S. (2011). Effective use of behavior-specific praise: a middle school case study. Beyond Behav. 20, 31–39.

Google Scholar

Lane, J. D., and Gast, D. L. (2014). Visual analysis in single case experimental design studies: brief review and guidelines. Neuropsychol. Rehabil. 24, 445–463. doi: 10.1080/09602011.2013.815636

PubMed Abstract | Crossref Full Text | Google Scholar

Lane, K. L., and Menzies, H. M. (2009). Student risk screening scale – internalizing and externalizing (SRSS-IE). Available at: https://www.ci3t.org/screening

Google Scholar

Lane, K. L., Menzies, H. M., Ennis, R. P., and Oakes, W. P. (2015). Supporting behavior for school success: A step-by-step guide to key strategies. Guilford.

Google Scholar

Ledford, J. R., and Gast, D. L. (2024). Single case research methodology: Applications in special education and behavioral sciences. 4th Edn. Routledge.

Google Scholar

Love, T. S., and Roy, K. R. (2023). Critical safety considerations to support CTE. Techniques 98, 32–35.

Google Scholar

National Center for Education Statistics. (2023). CDD data files common core of data (CDD). Available at: https://nces.ed.gov/ccd/schoolsearch/

Google Scholar

National Collaborating Centre for Mental Health (2015). Challenging behaviour and learning disabilities: prevention and interventions for people with learning disabilities whose behaviour challenges : National Institute for Health and Care Excellence.

Google Scholar

Newton, N. A., Royer, D. J., Hogg, J., and Priddy, A. (2024). Behavior-specific praise: enhancing on-task behavior in a rural elementary school. Front. Educ. 9:1445523. doi: 10.3389/feduc.2024.1445523

Crossref Full Text | Google Scholar

O’Handley, R. D., Olmi, D. J., Dufrene, B. A., Radley, K. C., and Tingstrom, D. H. (2023). The effects of different rates of behavior-specific praise in secondary classrooms. J. Posit. Behav. Interv. 25, 118–130. doi: 10.1177/10983007221091330

Crossref Full Text | Google Scholar

O'Handley, R. D., Olmi, D. J., Dufrene, B. A., Tingstrom, D. H., and Whipple, H. (2020). The effects of behavior-specific praise and public posting in secondary classrooms. Psychol. Sch. 57, 1097–1115. doi: 10.1002/pits.22375

Crossref Full Text | Google Scholar

Royer, D. J., Lane, K. L., Cantwell, E. D., and Messenger, M. L. (2017). A systematic review of the evidence base for instructional choice in k-12 settings. Behav. Disord. 42, 89–107. doi: 10.1177/0198742916688655

Crossref Full Text | Google Scholar

Royer, D. J., Lane, K. L., Dunlap, K. D., and Ennis, R. P. (2019). A systematic review of teacher-delivered behavior-specific praise on K–12 student performance. Remedial Spec. Educ. 40, 112–128. doi: 10.1177/0741932517751054

Crossref Full Text | Google Scholar

Royer, D. J., Priddy, A., Hogg, J. A., and Newton, N. A. (2024). Using behavior-specific praise in a high school setting. Manuscript in review.

Google Scholar

State Department of Education. (2024). State school report card. Available at: https://www.[blinded]schoolreportcard.com/home?year=2023

Google Scholar

Umbreit, J., Ferro, J. B., Lane, K. L., and Liaupsin, C. J. (2024). Functional assessment-based intervention: Effective individualized support for students. Guilford.

Google Scholar

Valentine, J. C., Tanner-Smith, E. E., Pustejovsky, J. E., and Lau, T. S. (2016). Between-case standardized mean difference effect sizes for single-case designs: a primer and tutorial using the scdhlm web application. Campbell Syst. Rev. 12, 1–31. doi: 10.4073/cmdp.2016.1

Crossref Full Text | Google Scholar

Witt, J. C., and Elliott, S. N. (1985). “Acceptability of classroom intervention strategies” in Advances in school psychology. ed. T. R. Kratochwill (Routledge), 251–288.

Google Scholar

Zoder-Martell, K. A., Floress, M. T., Bernas, R. S., Dufrene, B. A., and Foulks, S. L. (2019). Training teachers to increase behavior-specific praise: a meta-analysis. J. Appl. Sch. Psychol. 35, 309–338. doi: 10.1080/15377903.2019.1587802

Crossref Full Text | Google Scholar