Teacher Practices, Peer Dynamics, and Academic Enablers: A Pilot Study Exploring Direct and Indirect Effects Among Children at Risk for ADHD and Their Classmates

Owens, Julie Sarno; Qi, Hongyuan; Himawan, Lina K.; Lee, Mary; Mikami, Amori Yee

doi:10.3389/feduc.2020.609451

ORIGINAL RESEARCH article

Front. Educ., 13 January 2021

Sec. Educational Psychology

Volume 5 - 2020 | https://doi.org/10.3389/feduc.2020.609451

This article is part of the Research Topic The Role of Teachers in Students’ Social Inclusion in the Classroom View all 16 articles

Teacher Practices, Peer Dynamics, and Academic Enablers: A Pilot Study Exploring Direct and Indirect Effects Among Children at Risk for ADHD and Their Classmates

$\nJulie Sarno Owens$ Julie Sarno Owens¹^*

¹Department of Psychology, Ohio University, Athens, OH, United States
²Department of Psychology, University of British Columbia, Vancouver, BC, Canada

Establishing a positive peer climate in elementary school classrooms is an important goal for educators because peer dynamics are thought to affect academic learning. Thus, it is important to (a) understand the relationship between children's peer dynamics and academic functioning, and (b) identify teacher practices that influence both peer processes and academic outcomes. In this pilot study, we explored whether specific teacher strategies that promote positive behaviors in children and positive peer dynamics influence children's better academic enablers, as well as whether they do so indirectly via improving peer sociometric ratings. Such teacher strategies may be particularly relevant for supporting children who demonstrate impairment in both social and academic domains, such as children at risk for attention-deficit/hyperactivity disorder (ADHD). Thus, we also examined whether these relationships differ for children with elevated ADHD symptoms and peer problems (i.e., target students), relative to classmates (i.e., non-target students). Participants were 194 children in the classrooms of 12 teachers (grades K-4) who participated in an open-trial pilot study of the school-based version of the Making Socially Accepting Inclusive Classrooms (MOSAIC) program. In the fall and spring of a school year, we assessed children's sociometric ratings received from peers, and academic enabler skills as rated by teachers. Throughout one academic year, we obtained assessments of teachers' use of MOSAIC strategies (observed and self-reported). Results showed that, after accounting for fall academic enablers, the teacher strategy of CARE time (involving one-on-one interaction with the student to build the teacher-student relationship) was positively associated with spring academic enablers. However, findings did not support the hypothesized indirect effect of peer sociometric ratings on the relationship between teacher strategy use and academic enablers, or the moderated indirect effect by target student status. Implications for future research and classroom interventions are discussed.

Introduction

Establishing a positive peer climate in elementary school classrooms is an important goal for educators, as peer dynamics are theorized to relate to academic performance (Wentzel, 2017; Sette et al., 2020). Specifically, positive peer dynamics create a social context in the classroom that may foster the growth of academic enablers, which are cognitions, attitudes, and behaviors that facilitate and predict student academic achievement on grades and test scores, educational attainment, and future employment (e.g., DiPerna et al., 2002; Borghans et al., 2008; Farrington et al., 2012). Key academic enablers for elementary school students are motivation (e.g., academic interest and persistence), engagement (e.g., attention and participation), and effective communication in an academic context (working effectively in groups, listening to others; DiPerna and Elliott, 2002).

Being poorly regarded by classroom peers, as evidenced by sociometric measures, may interfere with children's development of academic enablers (Buhs and Ladd, 2001), as is evident among students at risk for attention-deficit/hyperactivity disorder (ADHD). This population of students has problems in both academic and social functioning, and furthermore, there is evidence that peer problems create risk for their subsequent academic failure beyond that conferred by initial academic impairment (Mikami and Hinshaw, 2006; Gardner and Gerdes, 2015). Given the dynamic relationships between ADHD behaviors, peer sociometrics, and academic enablers within the ecology of the classroom, it is important to identify teacher practices that can influence student behavior and peer dynamics as a way to facilitate the development of academic enablers and ultimately, academic success. In the current open trial pilot study, we examine the extent to which teacher practices in the Making Socially Accepting Inclusive Classrooms (MOSAIC) program directly influence children's academic enablers, and whether they do so indirectly via peer sociometric ratings. Because teacher practices may have unique influences on children with ADHD and social problems (Mikami et al., 2013a), we also examine whether these relationships differ for target students selected for elevated ADHD symptoms and peer problems vs. their classmates. A primary goal of the current pilot study was to identify specific teacher practices that have the most promise for future study.

The Making Socially Accepting Inclusive Classrooms (MOSAIC) Program

Although evidence-based classroom interventions for elementary school children with ADHD improve parent and teacher ratings of children's social and academic competencies (Pfiffner et al., 2016), there are no interventions that successfully improve peers' sociometric judgments of these children (Hoza et al., 2005). We argue that this is because such interventions focus solely on behavior management and the deficient skills of the target student and fail to account for critical peer influences (Mikami et al., 2010). The MOSAIC program was designed to address the limitations of existing attempts to alter peers' sociometric judgments of students with or at risk for ADHD. Namely, the program includes behavior management strategies to promote positive behavior among students with ADHD, as well as strategies to dismantle negative peer dynamics (e.g., reputational biases, exclusionary behavior, devaluation of students who behave differently from others). Teachers in MOSAIC are trained to incorporate the strategies into their day-to-day activities to encourage students' positive social behaviors, increase positive peer dynamics, and foster positive teacher-student relationships. Key MOSAIC strategies include: reviewing and reinforcing expectations for positive classroom behavior; reviewing and reinforcing expectations for inclusiveness among peers; brief one-on-one quality time between the teacher and student (referred to as CARE time; see Method); discreet corrections for inappropriate behavior, and teacher statements that specifically highlight personal strengths of individual students (see Mikami et al., 2020 for details).

The efficacy of the MOSAIC program was initially examined in a 2-week summer day camp with 24 children with ADHD (i.e., the target group) and 113 typically developing children in Grade 1 through Grade 3 (Mikami et al., 2013a,b). Findings showed that, relative to children receiving behavioral management only, children in the MOSAIC group received more favorable peer sociometric nominations and liking ratings, and received more positive messages written by peers in memory books. Although positive effects of MOSAIC were observed in typically developing children, the effects were stronger for target students, indicating the presence of moderation. Given the proof of concept demonstrated by the summer program finding, we collaboratively developed (with teachers) the school-based version of MOSAIC (Mikami et al., 2020). In our initial open trial pilot study (without a control group), we assessed the associations between specific MOSAIC strategies and peer sociometric ratings in spring of the school year, after statistical control of sociometric ratings in fall of the school year (Mikami et al., 2020). Our findings suggest that specific teacher practices (i.e., reviewing expectations for appropriate behavior, reinforcing expectations for inclusiveness, highlight personal strengths, and use of CARE time) in the MOSAIC program may influence better peer sociometrics, and that the effect of some of these practices may be moderated by target student status (see Moderating Role of ADHD Status section below). To date, however, direct effects of these same teacher practices on academic enablers have not been tested. Nor has the potential for indirect effects through the influence of these same teacher practices on peer sociometric ratings been studied. In the current pilot study, we explore these possibilities with the aim of identifying teacher practices that may relate to the complex interplay between student peer dynamics and academic enablers.

MOSAIC Practices and Peer Sociometrics

The practices included in the MOSAIC program are designed to shift peers' sociometric judgments by both increasing deficient skills of children who are poorly regarded by peers, and by altering peer group process. Behavioral theory and evidence-based classroom management strategies (Epstein et al., 2008; Simonsen et al., 2008) suggest that teachers can improve children's off-putting behavior by creating, reviewing, and reinforcing expectations for appropriate behavior. However, given that improving disruptive behavior may be insufficient for shifting peers' sociometric judgments (Mikami and Normand, 2015), teachers can potentially also use effective classroom management strategies to affect peer dynamics. For example, teachers can create, review, and reinforce expectations for peer respect and inclusiveness. One study found that by declaring “you can't say you can't play” as a classroom rule, or including language in a classroom charter about respectful treatment of others, teachers were able to shape more favorable class-wide peer sociometric ratings (Harrist and Bradley, 2003; Bacete et al., 2019).

Teachers may also influence peer dynamics via indirect methods, wherein teachers act as an “invisible hand” and guide peers to have more positive sociometric judgments of a classmate without explicitly instructing them to this end (Farmer et al., 2011). These indirect methods are thought to influence peer sociometrics via modeling. Specifically, in early elementary grades children make sociometrics judgments about their peers which are partly influenced by perceptions of how their teacher evaluates those same peers (Chang et al., 2007; Brey and Shutts, 2018). For example, when teachers give personalized, positive attention to a student, or highlight a positive attribute about a student, it may implicitly communicate to others that the student has desirable characteristics and is likable. Similarly, positive teacher-student interactions may offer a model for how students should treat one-another and send cues to peers that a given student has value. Support for these indirect influences comes from studies showing that a more positive teacher-student relationship predicts peers having better sociometric judgments of that student (Hughes et al., 2001; Hughes and Kwok, 2006). Additional support for these indirect influences comes from longitudinal investigations that show that teachers' personal liking of certain students predicts increases in favorable peer sociometric judgments of those students over time, with subsequent benefits for the recipients' academic functioning (Hughes and Chen, 2011; Sette et al., 2020). These ideas are also supported by our recent study showing associations between specific MOSAIC strategies and better peer sociometric ratings in spring of the school year, after statistical control of sociometric ratings in fall of the school year (Mikami et al., 2020).

In summary, there is emerging evidence that teacher practices can influence classroom peer dynamics. With this pilot study, we aim to advance the literature by examining the extent to which MOSAIC strategies predict children's better academic enablers at the end of the year, and whether they do so indirectly via improvements in peer sociometric ratings.

MOSAIC Practices and Academic Enablers

Direct Effects

Academic enablers are malleable and influenced by multiple factors within the dynamic classroom ecology, including teacher practices (Greenwood et al., 2002; Lekwa et al., 2019). Some of these teacher practices (effective classroom management and strategies to build student-teache relationships) are included in the MOSAIC program. For example, teachers may facilitate academic enabler skills through the use of effective classroom management practices. In the MOSAIC program, teachers are encouraged to establish and reinforce student behaviors that align with classroom expectations, and consistently use mild, discreet consequences for behaviors that violate classroom expectations, as the use of these strategies is associated with greater student academic task engagement and fewer disruptive behaviors (see Simonsen et al., 2008 for review). These links may exist, in part, because when teachers review expectations before an activity, it reminds students how to participate and communicate successfully. Similarly, when teachers reinforce those expectations during activities, it facilitates student on-task behavior and persistence (Jenkins et al., 2015).

In addition, teachers' use of strategies that promote their interpersonal closeness with and support of students may help reduce student disruptive behavior and increase academic engagement (Simonsen et al., 2008). This is consistent with empirical evidence finding that positive teacher-student relationships were associated with children's academic success concurrently and prospectively (Hamre and Pianta, 2001; Roorda et al., 2017). In the MOSAIC program, teachers are encouraged to call positive attention to students' strengths, or take a personal interest in students through spending one-on-one quality time with them. These strategies likely help students feel more welcome in the teacher's classroom and more motivated and supported to approach challenging tasks. In other words, supportive teacher behaviors may create a safe environment for children to take the intellectual risks needed for academic learning to occur (Roorda et al., 2011). Given this body of literature, we hypothesized that MOSAIC practices may have a direct effect on improvement in students' academic enablers.

Indirect Effects via Peer Dynamics

Peer dynamics may also have a significant influence on academic enablers (Buhs and Ladd, 2001; Ladd and Burgess, 2001). One commonly used metric of peer dynamics is the sociometric judgments that children receive from their classroom peers, indicating the extent to which a child is positively vs. negatively regarded by the peer group (e.g., liked or disliked; Coie et al., 1982). Positive sociometric judgments not only afford children supportive interpersonal interactions that are conducive for social and academic growth, but also may lead children to feel a sense of safety and belonging, thereby reinforcing their academic engagement and aspirations (Robinson and Mueller, 2014; Wentzel, 2017). In contrast, negative sociometric judgments are associated with children's lower participation in classroom activities and higher rates of off-task disruptive behaviors, both of which negatively impact academic achievement for the affected student and the classroom as a whole (Robinson and Mueller, 2014; Wentzel, 2017). Moreover, such children may develop low self-esteem and low expectations for social success (Sandstrom et al., 2017), which may diminish motivation to initiate social interactions and to pursue their academic goals (Boivin and Hymel, 1997; Wentzel, 2017). Indeed, it is well-established that negative sociometric judgments, especially when they occur year after year, undermine children's motivation to spend time on academic tasks (e.g., due to fear of being mocked and marginalized), and subsequently result in children's withdrawal from classroom participation (Buhs et al., 2006; Ladd et al., 2008; Iyer et al., 2010). Further, recent evidence suggests that classroom interventions focused on relationship building are capable of changing peer networks, as well as students' subsequent academic performance (DeLay et al., 2016), perhaps via the indirect effects of peer dynamics. Collectively, these findings highlight the relevance of peer dynamics for students' academic enabler skills, and coupled with the previously described direct effects of MOSAIC practices on sociometric ratings, provide the rationale for our examination of the relationship between MOSAIC practices and academic enablers via peer sociometrics.

The Moderating Role of ADHD Status

In our first evaluation of the school-based version of MOSAIC, we explored whether the associations between MOSAIC teacher practices and sociometric ratings differed among target students (students selected for being at risk for ADHD and peer problems) relative to non-target students (Mikami et al., 2020). Although we found evidence for the moderating effects of target student status for the outcome of sociometric ratings, the pattern was inconsistent across teacher practices (Mikami et al., 2020). Namely, we found that teacher use of reinforcing expectations for behavior and discreet corrections were associated with improved sociometric ratings for target children but not non-target students (Mikami et al., 2020). In contrast, teacher use of highlighting positive attributes and spending one-on-one quality time (CARE time) were useful for all children, and had accentuated benefits for non-target students (Mikami et al., 2020). In the current study, we also explore whether target students might differentially benefit from teacher practices to enhance academic enablers, directly and through the indirect effects of improved sociometric ratings. However, there is potential for these associations to be either stronger or weaker for target relative to non-target students.

On one hand, given that children at risk for ADHD show pronounced deficits in academic enablers and sociometrics (Hoza et al., 2005; Loe and Feldman, 2007), there may be more room for them to benefit from teacher practices that address these outcomes. Indeed, classroom behavioral management strategies represent an evidence-based intervention for this population (Epstein et al., 2008; Owens et al., 2020b). Thus, these strategies may act as a buffer between child deficits and classroom outcomes. In addition, children with ADHD often have strained relationships with their teachers (Greene et al., 2002). Therefore, teacher practices, such as promoting classroom inclusiveness, highlighting students' strengths, and spending one-on-one positive time talking with the student may help target students feel a sense of belonging in the classroom. Such feelings may increase target children's motivation to engage in classroom activities and to persist in difficult tasks. Thus, target students may show accentuated effects from these teacher practices relative to non-target students. In fact, in the summer program pilot, positive effects of MOSAIC were observed in typically developing children; however, the effects were stronger for target students, indicating the presence of moderation.

On the other hand, the substantial deficits in academic enablers and sociometrics shown by children at risk for ADHD may lead this group to not experience as much benefit from these teacher practices as do typical children. That is, children at risk for ADHD may have more entrenched negative reputations among peers and teachers for having poor behaviors and academic skills. Subtle teacher practices may be insufficient to change these negative peer sociometric judgments, or deficient academic enablers, in target children compared to in non-target children. This may be why, to date, evidence-based treatments for ADHD have had limited impact on changing peer sociometrics (Evans et al., 2018).

The Current Study

Using the dataset from the open trial pilot study of the school-based version of MOSAIC program (Mikami et al., 2020), the current analyses examine: whether teachers' use of MOSAIC strategies have direct effects on students' academic enablers (Aim 1), the extent to which any relationships between MOSAIC strategies and academic enablers might operate via the indirect effect of sociometric ratings (Aim 2), and whether the above direct and indirect effects are moderated by target student status (i.e., among students at risk for ADHD, relative to their classmates; Aim 3). See Figure 1 for a conceptual model of these relationships. We hypothesized that teachers' use of MOSAIC strategies would predict improvements in children's academic enablers at the end of the school year, while controlling for academic enablers in the fall, at the whole class level (Hypothesis 1). We expected to find indirect effects of peer sociometric ratings (Hypothesis 2) on this relationship. That is, we hypothesized that the use of MOSAIC strategies would predict children receiving more positive sociometric ratings, which in turn would lead to higher scores in teachers' ratings of their academic enablers. Lastly, in all of the above relationships, we explored the potential moderating role of target student status. Given that moderation by target student status was found for some but not all teacher strategies in our previous studies (Mikami et al., 2013a,b, 2020), and that the direction of the moderation was inconsistent, we did not make directional hypotheses.

FIGURE 1

Figure 1. Diagram for the conceptual model. Aim 1: Examines the direct effects of teacher strategies on academic enablers. Aim 2: Examines the indirect effects of spring sociometric ratings on the relationship between teacher strategies and academic enablers. Aim 3: Examines whether the above direct and indirect effects are moderated by target student status.

Materials and Methods

Participants

Participants were 12 elementary school teachers (K-4) and 194 students in their classrooms in Southern British Columbia, Canada (6 classrooms) and Central and Southeast Ohio, United States (6 classrooms), in the 2017–2018 school year. Of all students in the 12 classrooms, 82% of their parents provided consent at the Canada site and 70% at the United States site (range: 56–95% across the 12 classrooms). See Table 1 for teacher and student demographic information.

TABLE 1

Table 1. Demographic information for teachers and students.

Procedure

All procedures were approved at both sites by the associated university research ethics boards and school district administrators. School districts at each site were chosen based on interest from districts, variability in student demographic characteristics, and proximity to the university site. Teachers were recruited at staff meetings and by principals sharing project information with staff. Consenting teachers then shared project information with parents of all students in their classrooms (consent forms were translated into additional languages based on school request). Children whose parents provided consent and who provided assent participated in the study. All participating teachers received intensive coaching during the 2017–2018 school year in the use of the MOSAIC program.

Target Student Selection

At the beginning of the school year, teachers selected between three to five of the consented students in their classroom to serve as target students for the MOSAIC intervention (i.e., to receive a higher dose of the intervention). Target students were chosen based on having elevated ADHD symptoms and peer problems. Specifically, teachers completed a measure of children's ADHD symptoms (ADHD Rating Scale−5; DuPaul et al., 2016) and of children's peer problems (Dishion Social Acceptance Scale; Dishion and Kavanagh, 2003) for all consented students in the classroom. Additionally, their parents completed a brief measure of hyperactivity/inattention and of peer problems (Hyperactivity/Inattention subscale and Peer Problems subscale of the Strengths and Difficulties Questionnaire; Goodman, 1997). In each classroom, children were rank-ordered by teacher-rated ADHD symptoms, then teacher-rated peer problems. Based on this ranking, teachers selected the top three to five children who had both high ADHD symptoms and peer problems. If there was a tie for the top children, parent ratings were examined. This process resulted in the selection of 51 target students across the 12 classrooms. See Table 1 for target and non-target student demographic information, and Table 2 for their group differences in ADHD symptoms, academic enablers, and peer problems.

TABLE 2

Table 2. Descriptive statistics on study measures for target and non-target students.

Coaching Teacher Use of MOSAIC

In the spring before the 2017–2018 school year, recruited teachers attended a 2-h borientation on study procedures and MOSAIC strategies. At this orientation, teachers were given a manual describing the intervention and were encouraged to review it over the summer. During the 2017–2018 school year, teachers were asked to deliver all MOSAIC strategies to the whole class (Tier 1) with an emphasis on using the strategies more frequently with target students (Tier 2). Teachers received coaching on MOSAIC strategies from a consultant, who was a research team member. One of the goals of the pilot study was to reduce the number of strategies, thus, teachers were encouraged to use and provide feedback on several MOSAIC strategies. The current study focuses on seven strategies as they have the strongest psychometric properties and they are consistent with the strategies examined in Mikami et al. (2020). See Mikami et al. (2020) for a more detailed discussion on selection of these strategies.

Throughout the school year, teachers were observed by their consultants or by other research team members twice per month for approximately 40 min each time. After each observation, consultants emailed feedback to teachers on their use of MOSAIC strategies. Teachers met with a consultant twice per month (about 45 min each meeting) to discuss the teachers' the observation data to encourage the teacher's use of strategies with the whole class, and in a higher dose with target students.

The MOSAIC consultants were a postdoctoral fellow and two graduate students in clinical or school psychology, and two research associates (at the BA level) with experience working in schools. Consultants received a full-day training and weekly supervision. The team of study staff who served as observers consisted of the consultants, other graduate students and research associates (who were not consultants assigned to the teacher), and undergraduate research assistants. The study staff who interviewed children were research associates and undergraduate research assistants.

Measures

Throughout the school year, teachers' use of the specific MOSAIC practices was assessed through observations approximately twice per month, and through self-report surveys once per month where teachers reported the extent to which they used each MOSAIC strategy on the day prior to receiving the survey. Approximately one month after the start of school (to allow teachers and students time to know one another; fall assessment), and one month before the end of school (spring assessment), students participated in a peer sociometric interview and teachers completed a questionnaire measure of students' academic enablers. See Table 2 for the descriptive statistics of our measures.

Observed Teacher Practices

Across the school year, teachers were observed an average of 29.3 times (SD = 6.9, range = 19–37) for 40 min each time (broken down into five, 8 min blocks). On average, 39.8% of observations were completed by the consultant while the rest were completed by other research team members. As there were no significant differences in rates of MOSAIC practices observed by consultants vs. other research team members, observations from both types of raters were used in this study. Additionally, two coders together completed 30.2% of observations and these observations were used to calculate inter-rater reliability using inter-class correlation coefficients (below 0.40 = poor, 0.40–0.59 = fair, 0.60–0.74 = good, 0.75 and above = excellent; Cicchetti, 1994) for continuous variables. The seven strategies described below were considered the key strategies of MOSAIC.

Reviewing Expectations for Behavior

This strategy involved a teacher reminding students of what behaviors are expected before the activity occurs (e.g., before transitioning to independent work the teacher reminds students to use voice level 0). The purpose of this strategy was to encourage children to display appropriate behavior. Any time a teacher reviewed expectations for general behavior that was not inclusiveness (inclusiveness was tallied differently, see below), it was counted in this category (ICC = 0.99, excellent).

Reinforcing Expectations for Behavior

This strategy involved a teacher reinforcing appropriate behavior by calling attention to the behavior using specific praise or a reward (e.g., the teacher tells a student “Great job waiting your turn to speak!”). The purpose of this strategy was also to encourage appropriate behavior. Any instance of a teacher reinforcing positive behavior was counted in this category, unless specific to inclusiveness (ICC = 0.99, excellent).

Reviewing Expectations for Inclusiveness

Similar to reviewing expectations for behavior, this strategy involved a teacher reminding students of expected behaviors related to inclusiveness, before the activity begins (e.g., a teacher reminds the class before small group work that all members should be allowed to contribute). The purpose of this strategy was to encourage students to exhibit behavior that creates positive peer dynamics in the classroom environment. Any instance of the teacher reviewing expectations for inclusive behavior before an activity was counted here (ICC = 0.96, excellent).

Reinforcing Expectations for Inclusiveness

This strategy involved the teacher calling attention to a student exhibiting inclusive and prosocial behavior (e.g., a teacher tells a student “Thank you for helping your classmate clean up that mess!”). The purpose of this strategy was also to encourage student behavior that creates positive peer dynamics. Any instance of the teacher reinforcing inclusive or prosocial behavior was counted here (ICC = 0.97, excellent).

Highlighting Positive Attributes

This strategy involved a teacher calling peers' attention to persistent, positive qualities of a child that were related to the child's talent or character, and not to behavioral compliance (e.g., a teacher points out that a child is great at solving puzzles or telling jokes). The purpose of this strategy was to promote the idea that every child in the classroom is valued by the teacher, therefore creating more positive peer dynamics. Any instance of the teacher calling attention to a positive steadfast quality of a child in the presence of other children was counted here (ICC = 0.86, excellent).

CARE Time

This strategy involved the teacher providing a short amount of one-on-one quality time (3–5 min) to a student, where the teacher takes a positive interest in the student and what the student likes to do. Teachers were encouraged to provide time that was Child-centered, Affirms the child, during which the teacher Reflects the child's feelings and behavior, and the teacher Enjoys the child (i.e., CARE time). This technique was adapted to be used with elementary school students from one typically used with preschoolers (e.g., Banking Time; Driscoll and Pianta, 2010). The purpose of this strategy was to increase the teachers' personal liking of the student as well as to show that each child is valued, therefore increasing positive peer dynamics. The number of minutes that teachers conducted CARE time during the observation was counted (ICC = 0.99, excellent).

Discreet Corrections

This strategy involved the teacher providing corrective feedback on child misbehavior in a discreet manner when possible (e.g., a teacher calls a child aside and uses a low voice to inform the child to raise a hand to speak instead of blurting out). The purpose of discreet corrections is not to hide the corrections from peers (as they likely are aware of the teacher's intentions) but rather, to convey that the child should not be shamed for the behavior and that the teacher still respects and cares for the child. Each instance of the teacher correcting a child's behavior discreetly was counted here (ICC = 0.99, excellent).

Self-Reported Teacher Practices

In addition to observation of the above described MOSAIC strategies, teachers also completed nine surveys over the course of the academic year about their use of these same strategies. Teachers reported whether they used or did not use the strategies on the last full school day. To decrease teacher burden, only half of the seven strategies were assessed for each survey (i.e., each strategy was rated between four and five times across the nine total surveys sent out). Of the 12 teachers in the study, seven completed all of the surveys and five completed all but one of the surveys. We calculated a proportion score reflecting the number of surveys in which the teacher reported using that strategy divided by the number of surveys in which the teacher was asked about that strategy.

Peer Sociometric Ratings

In the fall and spring, a sociometric procedure (Coie et al., 1982) was conducted with consented children. Research team members interviewed each consented child individually, in private, and provided a visual of the names and pictures of all consented children in the classroom to aid in recall. Children were asked to rate how much they liked each classmate on a scale from 1 to 5 (1 = dislike a lot, 5 = like a lot), while the research assistant checked for children's comprehension and recorded children's answers. Children were also provided a visual of a face that ranged from frowning to smiling to correspond with the ratings. This procedure has strong test-retest reliability over a 6-month period (Wasik, 1987). The average sociometric rating received from peers was computed for each child.

Academic Enablers

In the fall and spring, teachers rated children's academic enablers using three subscales (Motivation, Engagement, and Interpersonal Skills) of the Academic Competence Evaluation Scales–Short Form (ASF; Anthony and DiPerna, 2018). The ASF has strong psychometric properties including high internal consistency, and convergent and discriminant validity (Anthony and DiPerna, 2018; Owens et al., 2020a). Engagement (3 items) captures active class participation, Motivation (5 items) measures persistence on challenging academic tasks, and Interpersonal Skills (5 items) reflects appropriate communication skills in an academic context. For each item, teachers rated the frequency the child exhibits the academic enabler behavior on a scale from 1 to 5 (1 = never, 5 = almost always). The mean of all items was calculated for each child to produce a total academic enablers score. Internal consistency of the 13 items was excellent in both fall (α = 0.94) and spring (α = 0.95).

Data Reduction

Reports of MOSAIC strategy use from observations and teacher self-report were moderately positively correlated (range of 0.31–0.63). To reduce the number of analyses, a composite of observations and teacher self-report was created for each strategy. First, the strategy was converted to a z-score. Second, for each strategy, an average of the z-scores of the observations and teacher self-report was calculated. For the strategy of reinforcing expectations for behavior, teachers reported that they used this strategy 100% of the time so the observation z-score was used for this strategy.

Data Analytic Strategy

Of the 194 students, all had complete data on the MOSAIC strategies their teacher used, and their target status. However, one child was missing fall scores on academic enablers, seven were missing spring scores on academic enablers, 10 were missing sociometric ratings in the fall, and nine were missing sociometric ratings in the spring, leaving 178 children with complete data across all variables. Missing data were mostly due to a child joining the class after the fall measures were completed or leaving before the spring measures were completed. There were no significant differences between the children with vs. without complete data on target status, gender, race (dichotomized as White vs. non-White), or academic enablers at fall or spring. However, children with complete data received higher sociometric ratings in fall, t₍₁₈₂₎ = 2.45, p = 0.015, and in spring, t₍₁₈₃₎ = 2.79, p = 0.006, and were younger in age, t₍₁₈₉₎ = 2.34, p = 0.017, compared to those with missing data. Missing data were handled using Full Information Maximum Likelihood (FIML) in all models.

The diagram of the model fitted for Aims 1, 2, and 3 is presented in Figure 1. We created seven models, one for each teacher MOSAIC strategy (reviewing expectations for behavior, reinforcing expectations for behavior, reviewing expectations for inclusiveness, reinforcing expectations for inclusiveness, highlighting positive attributes, CARE time, and discreet corrections). To test Aim 1, we examined the direct effect of the MOSAIC strategy on children's spring academic enablers, after statistical control of fall academic enablers and target student status (dummy coded as 0 = non-target, 1 = target). To test Aim 2, we examined the indirect effect of the MOSAIC strategy on spring academic enablers via spring sociometric ratings, after statistical control of fall academic enablers, fall sociometric ratings, and target student status. Finally, to test Aim 3 we added interaction effects to explore whether the direct effect of the MOSAIC strategy on academic enablers differed as a function of children's target vs. non-target status, and whether the indirect effect of the MOSAIC strategy on academic enablers via sociometric ratings differed as a function of children's target vs. non-target status (i.e., moderated mediation).

There is a nested structure within the data, where students (Level 1) are nested within classrooms (Level 2). We first tested Aim 1 using Multilevel Model (MLM) model in SAS for Windows Version 9.4, where we accounted for this nested structure. Then, to test Aims 2 and 3 we attempted to fit a Multilevel Structural Equation Model (MSEM) to the data using Mplus Version 8.4. However, MSEMs are difficult to fit and our models ran into a non-identification problem because we had more parameters to be estimated than the number of clusters. Therefore, Aims 2 and 3 were analyzed with Structural Equation Model (SEM) with Mplus Version 8.4 which does not account for the nested structure. The indirect effect in the mediation (Aim 2) and moderated mediation (Aim 3) analyses were produced using bootstrapping with 10000 subsamples drawn. All variables were grand-mean centered, in line with recommendations from Enders and Tofighi (2007) for analyses testing the main effects of a Level 2 predictor on a Level 1 outcome (Aim 1), and because the tests of Aims 2 and 3 did not incorporate nesting.

Results

Descriptive Statistics

On average, students received high sociometric ratings (M = 4.01 in fall, 3.87 in spring; possible range of scores = 1–5) and high ratings of academic enablers (M = 3.61 in fall, 3.90 in spring; possible range of scores = 1–5). Sociometric ratings in the fall and spring were negatively skewed (skewness = −0.81 in fall, −0.68 in spring). All variables were examined for outliers using absolute deviation around the median technique (Leys et al., 2013). The variables of fall and spring sociometric ratings, and spring academic enablers, had outliers at the lower end as determined by this method (seven for fall sociometric ratings, nine for spring sociometric ratings, and one for spring academic enablers). However, the outlier cases were examined and they did not appear to result from a data entry or calculation error. Additionally, the majority of the outliers were target student cases. Considering that classrooms have a wide range of students and target students (selected for elevated ADHD symptoms and peer problems) are likely to fall on the lower end of the class distribution in social and academic functioning, these cases were included. The teacher variables representing teachers' use of CARE time, highlighting positive attributes, and reinforcing expectations for behavior were positively skewed, whereas discreet corrections was negatively skewed. As there were only 12 teachers, these teacher variables were not assessed for the presence of outliers.

All student variables were significantly and positively correlated at the bivariate level (p < 0.01; r = 0.52–0.78). The following MOSAIC strategies were significantly and positively correlated (p < 0.05): CARE time with highlighting positive attributes (r = 0.75), reinforcing expectations for behavior with discreet corrections (r = 0.67), and reinforcing expectations for inclusiveness with reviewing expectations for behavior (r = 0.71). However, as there were only 12 teachers in this sample, a lack of significant correlation between other MOSAIC strategies should be interpreted cautiously.

Direct Effects of MOSAIC Strategies on Academic Enablers

For Aim 1, results showed that teachers' use of CARE time (β = 0.19, p = 0.029) was positively associated with spring academic enablers, after controlling for fall academic enablers and target student status (see Table 3). The MOSAIC strategies were z-scored, and the outcome of academic enablers reflected the average (from 1 to 5) of the ratings of each academic enabler item. Therefore, the beta weights indicate that a 1 SD increase in teachers' use of CARE time was associated with increases of 0.19 in spring academic enabler mean scores. The other teacher practices were not significantly associated with spring academic enablers (all ps > 0.10), after accounting for fall academic enablers.

TABLE 3

Table 3. Summary of analyses for Aims 1 and 2.

Indirect Effects of MOSAIC Strategies on Academic Enablers via Sociometric Ratings

As seen in Table 3, there were no significant indirect effects between teacher strategy use and spring academic enablers, as mediated by spring sociometric ratings (Aim 2). In these mediational models, the direct effects of reviewing expectations for behavior (β = 0.14, p = 0.007), highlighting positive attributes (β = 0.09, p = 0.046), CARE time (β = 0.19, p < 0.001), and discreet corrections (β = −0.10, p = 0.015) to spring academic enablers were significant. Because the analyses of direct effects in Aim 1 account for nesting, we base our interpretation of direct effects of teacher practices on academic enablers on the results from Aim 1 rather than Aim 2.

Moderation by Target Status

Table 4 contains the results testing Aim 3. In the full model (see Figure 1), target student status was not found to moderate the direct effects of MOSAIC strategies on spring academic enablers, nor the indirect effect of MOSAIC strategies on spring academic enablers, via spring sociometric ratings. As a replication of our previous findings, target status moderated the direct effects of some strategies on sociometric ratings (see Mikami et al., 2020 for discussion of those results). Similar to as in Aim 2, the direct effects of reviewing expectations for behavior (β = 0.16, p = 0.004), CARE time (β = 0.17, p = 0.002), and discreet corrections (β = −0.12, p = 0.018) to spring academic enablers were significant.

TABLE 4

Table 4. Summary of analyses for Aim 3.

Discussion

The current pilot study explored specific teacher practices that may be associated with improvement in children's academic enablers, whether any such associations operate via the indirect effect of better sociometric ratings, and the extent to which the findings may be similar for children at risk for ADHD relative to their classmates. These teacher practices were suggested in previous work (Mikami et al., 2020) to be associated with children receiving better sociometric ratings from their classroom peers. With the current analyses we attempted to determine whether the suggested benefits of any of these teacher practices might also extend to academic outcomes. Our goal was to better understand the complex associations between teacher practices, classroom peer dynamics, and student academic functioning, and also to identify potentially unique or previously undiscovered teacher practices that have the promise of shaping both peer dynamics and academic enablers. In partial support of our hypotheses, teacher use of the MOSAIC strategy of CARE time was positively associated with children having better academic enablers in spring, after accounting for fall levels of academic enablers. However, we did not find support for the hypothesized indirect effects of sociometric ratings on academic enablers, or for moderation by target student status on any of the above relationships. Below we interpret our pattern of findings and discuss the implications of the lack of significant findings for future research.

Teacher Practices and Academic Enablers

Greater teacher use of CARE time was associated with children having better academic enablers in spring after adjusting for fall enablers. The positive association for CARE time is consistent with the Banking Time intervention literature (Driscoll and Pianta, 2010) that shows that teacher use of this strategy is associated with teacher-reported improvements in student task persistence, engagement, and participation. In our previous analyses (Mikami et al., 2020), we found that greater use of CARE time was also significantly associated with students receiving higher peer sociometric ratings. Collectively, these findings, coupled with the literature, suggest that spending one-on-one time where the teacher shows interest in students may have positive impacts on student academic, behavioral, and social functioning. Given its impact on multiple domains of student functioning, teacher use of this practice could be considered high priority in pre-service and in-service professional development training and consultation, and research on potential mechanisms of action should continue.

Interestingly, the other teacher strategies (i.e., highlighting positive attributes, reviewing and reinforcing expectations for behavior, reviewing and reinforcing expectations for inclusiveness, and discreet corrections) were not significantly associated with improvement in academic enablers. In our previous analysis of these data (Mikami et al., 2020), we found the greater teacher use of highlighting positive attributes, reviewing expectations for behavior, and reinforcing expectations for inclusiveness, were significantly associated with students receiving higher peer sociometric ratings. These strategies are also thought to enhance student motivation, participation, and engagement through fostering positive teacher-student relationships and encouraging adaptive classroom behaviors. For example, reviewing expectations for behavior is an evidence-based classroom management strategy (Simonsen et al., 2008), and is particularly useful for students at risk for ADHD (Epstein et al., 2008). Moreover, highlighting students' strengths could make students feel more welcome in the teacher's classroom and more motivated and supported to persist in challenging tasks. However, in the current study, the use of these strategies was not positively associated with spring academic enablers as expected. We have considered multiple explanations for this null finding. First, other practices, such as teacher instructional strategies (e.g., critical thinking and problem-solving tasks, variety in academic work) have a powerful impact on academic enablers (Greenwood et al., 2002; Lekwa et al., 2019). It is possible that MOSAIC teachers were also using instructional strategies (that we did not measure) that directly targeted academic enablers, and these strategies exerted a stronger influence than did the MOSAIC strategies. In this context, the unique impact of MOSAIC strategies may not be detectable. Second, academic enablers skills were fairly high in the fall, leaving little room for improvement.

Another possibility is that our measurement approach may have limited our ability to detect the intended effects. Namely, we isolated strategies at the micro-level to enhance our ability to reliably measure each teacher strategy; however, these micro-level strategies may be necessary but insufficient to alter, or to predict change in, academic enablers¹. For example, recent studies have found that (a) teachers' appropriate response to student rule violations (e.g., verbal or non-verbal behaviors accompanied by appropriate tone, affect, intensity, and pitch) was more predictive of rates of student disruptive behavior than was the use of effective instructions or reinforcing expectations (Owens et al., 2018), and (b) that the relationship between use of appropriate response to rule violations and lower disruptive behavior became stronger over time for both target students and other students (Owens et al., 2020b). Thus, although reinforcing rules, which is a MOSAIC strategy, is a critical component of classroom management, this practice may only be sufficient in addressing student behavior and enablers when coupled with appropriate responses to rule violation. In the MOSAIC trial, we did not measure appropriate responses to rule violations as described above; if we had, perhaps we would have found a greater association between teacher practices and academic enablers.

Lastly, it is possible that we did not have the sample size to detect the impact of these strategies on academic enablers. Namely, even among the effects that were significant, the magnitude of these effects is rather modest. Thus, in a dynamic classroom context where multiple factors and their interactions are predicting student outcomes (Kyriakides et al., 2013; Steinbrenner and Watson, 2015), and because we statistically controlled for fall academic enabler skills in data analyses, MOSAIC strategies may only account for a small proportion of variance in spring academic enablers.

Indirect Effects of Peer Dynamics

We did not find support for the hypothesized indirect effects of sociometric ratings on the relationship between teacher strategies and academic enablers. In the context of the lack of findings, it is prudent to consider limitations in research design and measurement, as well as possible modification to the theory of change. With regard to design and measurement, there are many lessons learned from this pilot project that can inform future research. First, we are assessing multiple complex constructs within a dynamic ecological system. Thus, in order to detect the effects of interest, future researchers may need a significantly larger sample of teachers and more distinct and more frequent measurement periods (e.g., enablers measured in the fall and spring, sociometric ratings assessed in the winter, and rates of teacher strategy use prior to each of these time periods). We had a sample of 12 teachers which may have prevented the MSEMs from converging. A larger sample of teachers and a more distinct temporal sequence of the predictor, mediator, and outcome variables would likely allow for model converge and tests of mediation effects. Similarly, although there is evidence that peer dynamics can affect academic achievement within one school year (e.g., DeLay et al., 2016; Mikami et al., 2017), it may take longer than a year to detect the indirect effects of peer dynamics on a pathway between teacher practices to academic enablers.

Second, as described above, future researchers should consider the level of measurement of teacher practices. It is possible that capturing both a “global” indicator of effective teaching, in addition to micro-level indicators of strategy use, would allow for greater detection of effects of teacher behavior on student outcomes. For example, a highly effective teacher may use both specific MOSAIC strategies and other general strategies (e.g., an overall sensitive approach to understanding and incorporating students' unique emotional and learning needs into instructional practice; an organized classroom that runs like a well-oiled and productive machine) that together, contribute to students' social and academic functioning. Yet the measurement approach used in this study could not capture this global effectiveness factor. In future studies, researchers should consider assessing both micro-skills and more comprehensive indicators of classroom success, such as those assessed by the Classroom Assessment Scoring System (CLASS; Pianta et al., 2008).

Third, it is important to consider the overlap that may exist among the constructs assessed in this study. For example, one of the key academic enablers is interpersonal skills, which is the ability to communicate effectively and cooperate with peers on academic tasks. There is likely shared variance between this academic enabler and sociometric ratings. This shared variance might have contributed to the challenges we experienced when fitting the MSEM models. Thus, in future studies, researchers may need to consider examining indicators of academic performance that have less conceptual overlap with the variables assessing peer dynamics. Lastly, it is possible that the hypothesized pathways in this study are best detected early in the student's educational career (e.g., Kindergarten) before social reputations become intractable.

As we are learning more about the outcomes of the program (Mikami et al. under review), it may be fruitful to consider prioritizing fewer strategies (e.g., CARE time) and increase the use of strategies designed to shift peer sociometric judgments directly rather than indirectly via teacher modeling. Strategies that directly impact peer relationships include strategic seating, peer tutoring, cooperative learning tasks, use of direct peer compliments, and peer problem-solving skills (e.g., Van den Berg and Stoltz, 2018). Although some of these are included in the MOSAIC program (e.g., cooperative learning tasks, peer compliments), they are introduced to teachers later in the year (i.e., in the third of three phases, so as not to overwhelm teachers with multiple strategies and to account for the developmental progression of teacher and peer relationships over the course of the year) and were not prioritized with teachers in the current study. Future iterations of the MOSAIC program could examine the utility of applying these strategies earlier in the year to enhance their dose and potency.

Moderation by Target Status

The associations between teacher strategies and academic enablers were not moderated by target student status. This was the case for both the direct effects of teacher practices on academic enablers, and the indirect effects of sociometric ratings on the relationship between teacher strategies and academic enablers. These findings suggest that CARE time may be effective for all students' academic enablers as a direct effect, possibly rendering them valuable strategies that teachers can apply universally to all students in the classroom. On the other hand, the indirect effects of sociometric ratings on improvements in academic enablers did not seem to appear, regardless of students' target or non-target status. Future research with larger and more homogeneous samples (e.g., students who meet diagnostic criteria for ADHD and who have significant deficits in academic functioning) may be warranted to detect if there are differential benefits for subgroups of students in the classroom.

Strengths and Limitations

Study strengths include (a) use of a short-term longitudinal design across one school year, (b) a multi-informant, multi-method approach to measurement of both teacher practices and student outcomes to obtain good separation of method variance, (c) the controlling of fall academic enabler scores when predicting spring academic enabler scores, and (d) a two-site study that enhanced diversity of the sample and generalizability of our findings.

However, there are several limitations that are consistent with the pilot nature of the current study, including a small sample of teachers and the assessment of a limited number of teacher practices. The lack of assessment of the amount of teacher strategies specifically directed toward target vs. non-target students is another limitation. Had we found a moderating effect, we could not be certain if the effects were a function of target student status (i.e., ADHD symptoms and peer problems), or occurred because these students received higher doses of the MOSAIC practices. This limitation is being addressed in our current randomized clinical trial of the MOSAIC program.

In addition, our testing of Aims 2 and 3 did not account for the nested structure of students in classrooms. We understand that not accounting for nesting may provide biased estimates of standard errors, because it is ignoring interdependence among participants within a cluster. Of concern, it is more likely that the standard errors will be underestimated; that is, one may find a significant result without nesting that would be non-significant once nesting occurred (Osborne, 2000). In the current study, we find it notable that the hypotheses for Aims 2 and 3 were not supported in the non-nested data analyses. Therefore, we suspect that they would also not be supported if we had tested them using a nested structure.

Other limitations are that the target students were at risk for ADHD but we do not know if results can be extrapolated to children with confirmed ADHD diagnoses in the general education classroom. Lastly, we only measured three academic enablers (i.e., we did not include a measure of the academic enabler of study skills); different patterns may be detected with other academic enablers.

Conclusions and Implications

Our results suggest that specific teacher strategies designed to influence relationships (i.e., CARE time) hold some promise for improving student academic enablers from fall to spring. Although the magnitude of this effect may be small, this strategy should be considered a priority in both research and practice arenas, as this study and others collectively suggest the utility of this strategy on children's social, behavioral, and academic outcomes (Driscoll and Pianta, 2010; Mikami et al., 2020). Our findings also highlight the challenges of documenting the impact of teacher practices on social and academic outcomes, as well as the mechanisms through which these practices are operating to produce changes. It is recommended that researchers examine these relationships by recruiting a larger sample of teachers; comprehensively assessing teacher use of strategies (at micro- and macro-levels) and their use directed toward target vs. non-target students; and including multiple measurement periods (perhaps over multi-year periods) to establish the temporal sequence for mediation analyses. Given that multiple strategies may be required to develop a comprehensive approach that improves children's classroom functioning, it is recommended that teachers work collaboratively with behavioral consultants or school psychologists to determine the best combination for each classroom based on student needs and characteristics.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved by Institutional Review Board, Ohio University, Institutional Review Board, University of British Columbia, and reviews boards in all participating school districts. Written informed consent to participate in this study was provided by the participants' legal guardian/next of kin.

Author Contributions

ML contributed to the conceptualization of the paper. HQ contributed to writing the Introduction and Discussion sections and was responsible for the references section. LH conducted all statistical analyses, prepared all tables and figures, and contributed to the writing of the results section. All authors contributed to the article and approved the submitted version.

Funding

The research was supported by the Institute of Education Sciences, U.S. Department of Education (R324A160053). The opinions expressed are those of the authors and do not represent views of the Institute or the U.S. Department of Education.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Footnotes

1. ^We note that we also explored a composite variable representing each teacher's use of all MOSAIC strategies. However, none of the results changed when using this variable.

References

Anthony, C. J., and DiPerna, J. C. (2018). Piloting a short form of the academic competence evaluation scales. School Mental Health 10, 314–321. doi: 10.1007/s12310-018-9254-7