Comparing the Effectiveness of Evidence-Based Parenting Programs on Families of children with and without Special Educational Needs: Short-term and Long-term Gains

Totsika, Vasiliki; Mandair, Sashvinder; Lindsay, Geoff

doi:10.3389/feduc.2017.00007

ORIGINAL RESEARCH article

Front. Educ., 10 March 2017

Sec. Special Educational Needs

Volume 2 - 2017 | https://doi.org/10.3389/feduc.2017.00007

Comparing the Effectiveness of Evidence-Based Parenting Programs on Families of children with and without Special Educational Needs: Short-term and Long-term Gains

$\r\nVasiliki Totsika*$ Vasiliki Totsika¹*

Sashvinder Mandair² $Geoff Lindsay\r\n$ Geoff Lindsay³

¹Centre for Educational Development, Appraisal and Research (CEDAR) and Centre for Education Studies (CES), University of Warwick, Coventry, UK
²Leadership Development Programme, Teach First, London, UK
³Centre for Educational Development, Appraisal and Research (CEDAR), University of Warwick, Coventry, UK

Parents of children with a disability or special educational needs (SEN) have three available options when accessing parenting programs: (a) to access a parenting program that has been adapted for use by families with a child with disability, (b) to access a disability-specific parenting program, or (c) to access a parenting program developed for typically developing children. The aim of the present study was to examine whether accessing evidence-based parenting programs (EBPPs) developed for typically developed children (option c) could benefit families of children with SEN, and whether benefits could be maintained when program delivery takes place as part of sustained service implementation. Using data from an effectiveness trial, we found that there was no evidence of differential effectiveness: i.e., families of children with SEN experienced similar gains to families whose child did not have SEN with respect to child behavior problems, parenting style, and parental mental well-being. Using data from services’ sustained implementation, our findings indicated that gains during the implementation phase were of similar magnitude to gains during the research trial: following EBPPs, families of children with SEN experienced small to moderate improvements in behavior problems, and moderate to large improvements in parenting and parental mental well-being across the two phases. One year later, gains were significantly maintained in families who had accessed EBPPs as part of the research trial. While the study is not proposing that EBPPs developed for typically developing children are a replacement for disability-adapted or disability-specific parenting programs, there was a pragmatic need to evaluate the effectiveness of EBPPs that are in practice accessed by families with a child with SEN. Overall, families of children with SEN can benefit from EBPPs similarly to families whose child does not have SEN, and the gains are significant and substantial even when EBPPs are offered as part of regular service provision. Longer term maintenance of gains (1 year) in service-led implementation of EBPPs likely requires more input.

Introduction

A number of parenting programs have been developed to address child behavior problems and improve child well-being through improvements in parenting, increased knowledge, and understanding of child behavior and appropriate use of disciplining. Parenting programs can be adjusted depending on the severity of child behavior problems (Bunting, 2004), and they are often delivered in a small group format in community settings.

The efficacy of many parenting programs has been evaluated using gold standard randomized control trials (RCTs). A number of systematic reviews and meta-analyses have summarized the RCT findings. Furlong et al. (2012) reviewed the evidence on the effectiveness of behavioral and cognitive-behavioral group-based parenting programs and identified 13 studies (10 RCTs and 3 quasi-randomized studies). Findings indicated significant improvements for child conduct problems, evaluated either through parental self-report or independent assessment (Dretzke et al., 2009; Furlong et al., 2012). Parenting programs are also effective in improving parenting skills (more positive parenting, less negative parenting practices) and maternal mental health (Barlow et al., 2002; Furlong et al., 2012). Improvements in parenting skills, especially positive parenting, mediate the improvements seen in child behavior problems (Gardner et al., 2006).

Further to establishing the efficacy of parenting programs, research is now also available on their effectiveness. In the UK, the Department for Education funded a pathfinder program in 18 Local Authorities (LAs) (Lindsay et al., 2008) followed by a national roll out of evidence-based parenting programs (EBPPs) across all English LAs as well as a research evaluation of the effectiveness of both the pathfinder (Lindsay et al., 2011) and national roll out (Lindsay and Strand, 2013). Findings for that trial supported the effectiveness of EBPPs such as Triple P and Incredible Years in improving child behavior problems, improving parenting skills, and parental mental well-being when rolled out across the population on a large scale (Lindsay et al., 2011; Lindsay and Strand, 2013).

Evidence-based parenting programs are typically not developed for children with a disability or special educational needs (SEN); rather, these programs have been developed to target behavior problems in typically developing children. However, children with a disability, especially developmental disabilities, often present significantly higher levels of behavior problems than typically developing children (Dekker et al., 2002; Emerson and Einfeld, 2010; Totsika et al., 2011a,b). Therefore, there is great need among parents to access effective tools that help them address behavior problems. For this, a small number of programs have been specifically developed for parents of children with disabilities. For example, positive parenting (Rollings and Hames, 2009) has been developed for parents of children with intellectual disability. However, to date, there is no evidence of its efficacy from controlled evaluations.

Another option for these families is to access EBPPs that have been adapted for use with parents of children with a disability. The Incredible Years program (Webster-Stratton, 2001) has been adapted and piloted with parents of children with developmental delay (McIntyre, 2008). Triple P (Sanders, 1999) has been adapted to be used with parents whose children have various disabilities and is known as Stepping Stones Triple P (SSTP; Sanders et al., 2004); SSTP has been evaluated in several RCTs and a recent meta-analysis summarized available evidence from nine RCTs and three uncontrolled studies (Tellegen and Sanders, 2013). Findings indicated that immediately after SSTP, there were significant improvements in child problem behaviors and improvements in parenting skills among parents of children with various developmental disabilities. In fact, SSTP is at the moment the only disability-adapted parenting program that has established efficacy through repeated RCTs.

Therefore, the evidence base regarding the efficacy of disability-specific or disability-adapted parenting programs is very limited (c.f., Petrenko, 2013), while there is no evidence on their effectiveness, i.e., whether they remain effective when rolled out on a large scale across the population. However, large-scale roll out of disability-specific EBPPs is an essential component of any public health approach to addressing the high levels of behavior problems across children with disabilities (Mazzucchelli and Sanders, 2011). The lack of a robust evidence base for most disability-specific parenting programs and the fact that they are addressing a small(er) part of the population may explain why disability-specific parenting programs are not available as a standard early intervention provision for this population. Combined with cost-efficiency savings that service providers have had to deliver in the post 2008 economic crisis era, disability-specific programs remain largely inaccessible to the majority of parents whose child has a disability.

For these reasons, the aim of the present study was to examine the effectiveness of non-disability-specific EBPPs in improving outcomes for families with a child with a disability. The aim was driven by a realistic need to examine effectiveness in a part of the population that has high levels of need that are addressed by non-specialist, albeit evidence-based programs.

To address this aim, we accessed data from the evaluation of the national roll out of EBPPs conducted by Lindsay et al. (2011), where we identified parents whose child had SEN. In the UK, SEN is the formal term for identifying disabilities in children and young people, where disabilities have implications for their educational, health, and social care needs (Department for Education, 2016). To access the label of SEN, there is no requirement of a preexisting clinical diagnosis, although one (or several) may be available. The route to accessing the label of SEN in the UK is dual: SEN is recognized either at school for those with less severe needs or SEN identification follows a standardized process of assessment by a multi-disciplinary team that is based within the local area’s statutory services (Department for Education, 2015). In 2016, approximately 14% of children in England were identified as having SEN (Department for Education, 2016).

Using data from the evaluation of the national roll out (Lindsay et al., 2011), our first research question aimed to understand whether EBPPs were as effective for parents with a child with SEN as they were for parents whose children did not have SEN. A narrative review of available evidence suggested that it is likely that parenting programs developed for the general population are effective for children with developmental disabilities (Petrenko, 2013), but the review was not systematic and the suggestion was not empirically tested. A recent meta-analysis supported the effectiveness of parenting programs for children with developmental disabilities but considered both programs that had been adapted for disabilities and programs that had not been adapted (Skotarczak and Lee, 2015). On the basis of the available evidence, we hypothesized that EBPPs would be equally effective across these two groups.

Our second research question focused on understanding whether effectiveness levels for these families can be maintained when they are receiving EBPPs as part of regular service provision. Regular service provision is the provision of parenting programs as made available by public services in the context of their standard provision to the population they serve (in the same way, educational services are a regular service provision to school-aged children). Under these circumstances, the provision of a service is led by an organization without researcher involvement. The effective implementation of evidence-based practices in real-world settings is one of the core challenges of implementation science (Franks and Schroeder, 2013). When transferring EBPPs from “research trials” to real-world practice, the change in conditions between research and practice might lead to a change in the level of expected outcomes (Durlak and DuPre, 2008). While much of implementation research is focused on the process of implementation, monitoring outcome data from sustained implementation of EBPPs is key to ensuring successful service provision and the sustainability of a program (Franks and Schroeder, 2013). Ultimately, however, being able to demonstrate whether effectiveness of EBPPs can be maintained during sustained implementation under real-world conditions, where there will always be some deviation from research conditions, is a pre-requisite for establishing the potential of EBPPs to be used as a public health approach with families whose children have SEN. To address our second research question, we used data from the sustained implementation of EBPPs after the end of the trial in a number of LAs. We compared outcomes from families of children with SEN between the research trial and the sustained implementation. While we could locate no previous research to guide our hypothesis, we anticipated that changes following EBPPs delivered during a research trial would be of similar magnitude to changes following EBPPs delivered during regular service provision.

Materials and Methods

Study Design

Data were derived from two phases. During the Trial Phase (Phase 1), EBPPs were rolled out across all 152 LAs in England as part of the Parenting Early Intervention Program (PEIP 2008–2011; Department for Education). A number of LAs (N = 43) were selected as representative of all English areas to evaluate the effectiveness of the roll out (Lindsay and Strand, 2013). To address the first research question, we identified parents from the Trial Phase (Phase 1) whose children had SEN and compared them to parents whose children did not have SEN.

During the Sustained Implementation Phase (Phase 2), four LAs opted to continue providing parenting programs following the end of the national roll out (2011–2015). They also continued to collect evaluation data as part of service monitoring. To address our second research question, we identified parents whose children had SEN during Phase 2 and compared them to parents of children with SEN from Phase 1.

Participants

During Phase 1, a total of 708 parents among the overall group of 6,143 parents identified their child as having SEN, indicating that 12% of families had a child with SEN (Table 1). The majority of parents across both groups were female (83 and 85% in the SEN and non-SEN group, respectively), biological parents (95 and 97% respectively) and White British (86 and 80%), and rented their house (78 and 65%). Within the SEN group, about a third of parents had no educational qualifications (32%) and half of them were single parents (49%), compared with 23 and 43% respectively for the non-SEN group. The children of parents in the SEN group were slightly older than the non-SEN group (mean 9.7 years, SD 3.11) compared with the non-SEN group (mean 8.5, SD 3.9) more likely to be boys (75 vs 60%), and more likely to be eligible for free school meals (FSM): 68 vs 46%, a means tested benefit available for low income families.

TABLE 1

Table 1. Descriptive statistics for parent and child background measures for Phase 1.

During Phase 2, 453 of the total 3,673 parents from the four LAs identified their child as having SEN (12% of the total sample in this phase). Table 2 presents the social and demographic characteristics of these parents and children (N = 453) compared to a group of 445 parents of a child with SEN who had received the same parenting programs in the four LAs during Phase 1 (see Procedure). The majority of parents in Phase 2 were female (80%), White British (84%), and biological parents (95%), and rented their house (68%). Just over a third of the parents were single parents (37%), and 23% had no educational qualifications. Children with SEN at Phase 2 were mainly boys (75%), had a mean age of 9.02 years (SD 3.7), and 53% were eligible for FSM.

TABLE 2

Table 2. Descriptive statistics for the two special educational needs (SEN) groups across Phases 1 and 2.

Measures

Social and Demographic Characteristics

Before the start of the parenting programs, parents provided information on their child’s SEN status, their age and gender, and relationship to child. Parents reported on their own gender, single parent status, housing status, ethnicity, and educational qualification levels. To create a composite measure of socioeconomic deprivation, we combined data from housing status (own home vs rent home), educational qualifications [no educational qualifications vs Level 1 + qualifications (Level 1 qualifications is the lowest level of educational qualification that can be obtained at the end of compulsory education at age 16; it demonstrates basic knowledge and skills)], FSM entitlement, and single parent status. Scores ranged from 0 to 4, with higher scores indicating higher socioeconomic disadvantage.

Child Behavior Problems

The Strengths and Difficulties Questionnaire (SDQ: Goodman, 1997) is a widely used scale measuring behavior difficulties in children aged between 2 and 17 years. The parent report version includes 25 items rated on a 3-point scale: “Not true; Somewhat true; Certainly true.” The SDQ comprises five subscales, four measuring behavioral difficulties: emotional symptoms, conduct problems, hyperactivity, peer problems, which combine to produce a total difficulties scale; and a fifth scale, Prosocial, which measures positive social behavior. Each scale is scored from 0 to 10, total difficulties 0–40, with higher scores indicating more behavioral problems. In the present study, we report findings on conduct problems, hyperactivity, emotional symptoms, and total difficulties, as the behavior problems most closely targeted by parenting programs and most relevant from a parental perspective. For the SEN groups in Phases 1 and 2, internal consistency estimates (Cronbach’s alpha) were as follows: emotional symptoms 0.69 and 0.72, respectively, conduct problems 0.65 and 0.63, hyperactivity 0.71 and 0.71, and total difficulties 0.80 and 0.79.

Parental Mental Health

The Warwick Edinburgh Mental Well-Being Scale (WEMWBS; Tennant et al., 2007) was developed to provide a unidimensional measure of subjective mental well-being. The WEMWBS includes 14 items rated on a 5-point scale ranging from “all the time” to “none of the time.” A total score of mental well-being is obtained by summing all items, range 14–70 with higher scores indicating more positive mental well-being. High internal consistency was found in both Phase 1 and 2, alpha 0.92 and 0.93, respectively.

Parenting Style

The Parenting Scale-Adolescent (Irvine et al., 1999) is a 13-item scale that was shortened from an original 30-item measure of parenting styles of parents of pre-school children (Arnold et al., 1993). Six items measure parental laxness, or parents’ inconsistency of parenting, and six measure parental over-reactivity, or parents’ harsh parenting style. The last item assesses monitoring and is used when estimating a total parenting style score. Items are scored on a 7-point Likert scale, with higher scores indicating less effective parenting styles. The present study focused on laxness and over-reactivity: the stability and good psychometric properties of these two factors have been demonstrated across a broad age range of children (Karazsia et al., 2008). These scales had satisfactory internal consistency: alphas for the Phase 1 and Phase 2 SEN groups were 0.71 and 0.79 for laxness and 0.70 and 0.78 for over-reactivity, respectively.

Parenting Programs

During Phase 1, eight programs were selected by the UK Department for Children, Schools and Families (DCSF), following accreditation by the National Academy for Parenting Practitioners (NAPP),¹ which reviewed parenting programs available in the UK on the basis of their evidence base, staff training, content and delivery methods as well as target audience, and produced an overall rating to aid the commissioning of parenting programs. The programs were: Triple P (Sanders, 1999); Strengthening Families 10-14 (SFP 10-14; Molgaard et al., 2000); Strengthening Families Strengthening Communities (SFSC; Race Equality Foundation, 2016); Incredible Years (Webster-Stratton, 2001); Families and Schools Together (FAST; Conduct Problems Prevention Research Group, 1992); STOP (Ministry of Parenting, 2015); Parent Power and Parent Plus (Sharry and Fitzpatrick, 1997). During Phase 2, three of these programs were made available: STOP (12%), Triple P (83%), and Incredible Years (5%).

Parenting programs were delivered in a group format in community settings by trained facilitators. The core aim of all programs was to improve and encourage positive parenting by educating parents on positive parenting and effective behavior management approaches. Each parenting program was manualized with detailed instructions for the facilitators and supervisors regarding administration of each program. Various methods of training were used including DVD modeling and role play to encourage discussion amongst parents and facilitators.

Procedure

Parents were recruited to parenting programs locally through various routes because of concerns about their children’s behavior (schools, health service, social services, and self-referral). Program participants completed a pre-intervention questionnaire with demographic data and outcome measures. Post-intervention data were collected at the end of the last parenting group session. Parents were provided with paper questionnaires distributed by the program facilitators. Once collected, pre and post data were passed on to the research team. One year after the date of completion of the pre-questionnaire, participants were posted a follow-up questionnaire. Questionnaires were posted directly from the research center as program facilitators were no longer in contact with participants. Pre-paid envelopes were provided for participants to return their questionnaire. Evaluation processes were identical during both phases of the study. However, while the aim of the evaluation during Phase 1 was to establish the effectiveness of the roll out, Phase 2 data were used by program providers to monitor their own work and report to their individual LAs or parent organization.

This study was carried out in accordance with the recommendations of the Humanities and Social Sciences Research Ethics Committee at the University of Warwick with written informed consent from all participants (Eth App 45/07-08 and Eth App 122/14-15). All participants provided written informed consent in accordance with the Declaration of Helsinki.

Approach to Analysis

To address the first research question, we identified parents in Phase 1 who indicated that their child had SEN and compared their outcomes to parents (from the same phase) whose child did not have SEN. The aim was to examine whether changes in child behavior problems, parenting, and parental well-being were similar between the two groups. We conducted ANCOVAs, with Group as the between-subjects factor (SEN vs non-SEN) and Time as the within-subjects factor (pre, post, follow-up). We controlled for the effects of socioeconomic disadvantage on parent outcomes and the effects of both socioeconomic disadvantage and gender on child outcomes. In line with our hypothesis, we anticipated that there would be no significant Group × Time interaction, but that both groups of parents would experience significant improvements over time.

To address the second research question, we identified parents in Phase 2 who indicated their child had SEN. We compared their outcome data to parents with children with SEN from Phase 1 who had received the same three parenting programs: Triple P, Incredible Years, or STOP. The aim was to examine whether the level of change experienced by parents with a child with SEN during Phase 1 was at similar levels to changes experienced by parents of children with SEN who received parenting groups as part of regular service provision (Phase 2). Comparisons were conducted using ANCOVAs for each of the main outcomes, with Group as the between-subjects factor (SEN in Phase 2 vs SEN in Phase 1) and Time as the within-subjects factor (pre, post, follow-up). We controlled for the effects of deprivation and child gender on child outcomes, and the effect of deprivation on parent outcomes. In line with the hypothesis, we anticipated that there would be no significant interaction between Group and Time, but that both groups would experience significant improvements in outcomes pre–post, which would be maintained at 1-year follow-up.

Finally, effect sizes were calculated using adjusted means (from the ANCOVAs) to estimate the magnitude of change in the main outcomes in the SEN groups at Phases 1 and 2. Effect sizes were estimated as standardized mean differences (d) accounting for pre-intervention scores and can be interpreted using Cohen’s guidelines (1988) as small (0.20), medium (0.50), and large (0.80).

Post and follow-up data were available from approximately one third of participants in each sample who had provided pre-course data. This “attrition” rate for data was very similar across groups and study phases: from 1,650 (30%) of 5,435 participants in the Phase 1 non-SEN group; from 222 (31%) of the 708 participants in the Phase 1 SEN group; from 133 (30%) of the 445 participants in the Phase 1 SEN group for Research Question 1, 1,650 (30%) of the 5,435 non-SEN group and 223 (31%) of the 708 in the SEN group during Phase 1; and for Research Question 2, 139 (30%) of the 445 Phase 1 SEN group and 150 (33%) of the 453 participants in the Phase 2 SEN group. Data were missing either because participants completed the courses but did not complete the questionnaires, or because they did not complete the courses, or they were not provided with questionnaires due to program providers’ administrative error (Lindsay and Strand, 2013).

Results

Phase 1: Trial Phase

To address the first research question, we compared outcomes between parents with a child with SEN to parents of a child without SEN during Phase 1. Table 3 presents the means (SDs) for the parent measures (upper part) and child behavior problems (lower part).

TABLE 3

Table 3. Parent and child outcomes between pre-course, post-course, and follow-up for the non-special educational needs (SEN) and SEN groups in Phase 1.

Parent Outcomes

There were no significant interactions between group and time for parenting style after controlling for deprivation: laxness F(2, 1822) = 1.10, p = 0.331 and over-reactivity F(2, 1818) = 0.51, p = 0.601. There was a main effect of group on laxness F(1, 911) = 7.89, p = 0.005, whereby parents of children with SEN scored higher for laxness. There was no main effect of group on over-reactivity F(1, 909) = 2.04, p = 0.153. There was a main effect of time on both laxness [F(2, 1822) = 46.41, p < 0.001] and over-reactivity [F(2, 1818) = 85.49, p < 0.001]. Follow-up Bonferroni comparisons indicated both parenting styles significantly improved from pre to post-intervention (all p < 0.001) and from pre to follow-up (all p < 0.001) suggesting a significant change and maintenance of gains in parenting.

There was no significant interaction between group and mental well-being scores, after controlling for SES [F(2, 1840) = 0.46, p = 0.630]. There was no main effect of group F(1, 920) = 0.29, p = 0.589, but there was a significant main effect of time [F(2, 1840) = 54.93, p < 0.001]. Follow-up pairwise comparisons indicated mental well-being scores significantly improved from pre- to post-intervention (p < 0.001) and from pre- to follow-up (p < 0.001). These results indicate significant gains on all three parent measures immediately following attendance at a parenting program and successful maintenance of skills over 1 year.

Child Behavior

There were no significant interactions between group and time for any of the child outcomes, after controlling for SES and child gender: conduct problems F(2, 1698) = 0.80, p = 0.450; emotional symptoms F(2, 1714) = 1.18, p = 0.307; hyperactivity, F(2, 1704) = 0.13, p = 0.880; and total difficulties F(2, 1648) = 0.81, p = 0.444. There were significant main effects of group for each child outcome, whereby children with SEN scored significantly higher on the SDQ. There were significant main effects of time on each child outcome. Follow-up Bonferonni pairwise comparisons indicated that there was a significant improvement in all scores from pre to post-intervention (all p < 0.001) and from pre to 1-year follow-up (all p < 0.001), indicating significant gains immediately after the parenting course and successful maintenance of gains in behavior problems over the following year.

In summary, the hypothesis of no significant interaction between group and time was confirmed suggesting that parents of children with SEN benefited from parenting programs at similar levels to gains experienced by parents whose children did not have SEN. The significant group difference in SDQ scores suggested children with SEN presented higher levels of behavior problems throughout the trial. All outcomes had a significant main effect for time indicating that child behavior problems, parenting laxness, parenting over-reactivity, and parental mental well-being significantly improved over time.

Phase 2: Sustained Implementation Phase

To address our second research question, we compared parents with a child with SEN from Phase 2 to parents with a child with SEN who had received the same parenting programs during Phase 1. Table 4 presents the means (and SDs) on parenting measures (upper part) and child behavior problems (lower part).

TABLE 4

Table 4. Parent and child outcomes between pre-course, post-course, and follow-up for the special educational needs (SEN) groups in Phase 1 and Phase 2.

Parent Outcomes

There was no significant interaction between group and parental laxness [F(2, 174) = 0.76, p = 0.470] and over-reactivity [F(2, 158) = 2.89, p = 0.062]. There was no main effect of group on laxness [F(1, 87) = 0.11, p = 0.746], but a significant group effect was present for over-reactivity [F(1, 79) = 4.90, p = 0.030], indicating that the SEN group from Phase 1 scored higher on over-reactivity compared to the SEN group at Phase 2 (Table 4). There was a significant main effect of time on laxness [F(2, 174) = 4.36, p = 0.014]. Follow-up pairwise comparisons indicated laxness significantly improved from pre-intervention to post-intervention (p < 0.001) and from pre-intervention to follow-up (p < 0.004), indicating successful maintenance of parenting style gains. There was no significant main effect of time for over-reactivity [F(2, 158) = 2.92, p = 0.060], indicating over-reactivity did not significantly improve over time.

There was no significant interaction between group and time for mental well-being [F(2, 198) = 2.46, p > 0.089]. There was no main effect of group [F(1, 99) = 1.56, p = 0.215] but the main effect of time was significant [F(2, 198) = 13.50, p < 0.001]. Follow-up pairwise comparisons indicated that mental well-being scores had significantly improved from pre to post-intervention (p < 0.001) and from pre to follow-up (p < 0.001).

In summary, the lack of significant interactions between group and time during the sustained implementation phase confirmed the hypothesis that parents of children with SEN would benefit from parenting programs offered as part of regular service provision similar to parents who attended programs offered during a research trial. With the exception of parental over-reactivity, child behavior problems, parenting laxness, and mental well-being significantly improved over time across phases.

Child Behavior

There were no significant interactions between group and time for any child outcomes, after controlling for deprivation and child gender: conduct problems F(2, 174) = 1.60, p = 0.205; emotional symptoms F(2, 180) = 0.34, p = 0.708; hyperactivity F(2, 188) = 1.79, p = 0.170; and total difficulties F(2, 170) = 1.05, p = 0.352. The main effect of group was also not significant throughout models suggesting the levels of behavioral and emotional problems between the two groups were similar. There were significant main effects of time on each child outcome (Table 4). Follow-up pairwise comparisons suggested that there was a significant improvement in all scores from pre to post-intervention (all p < 0.001) and from pre-intervention to 1-year follow-up (all p < 0.01), apart from emotional symptoms from pre-intervention to 1-year follow-up (p > 0.05). This suggests significant gains after the intervention for all behavior problems and significant maintenance over 1 year for all behavior problems except emotional symptoms.

Magnitude of Change in Parent and Child Outcomes across Phases

To examine the magnitude of change in the SEN group during Phase 2 (N = 453), we used the adjusted means from the ANCOVA (pre–post and pre-follow-up) to estimate effect sizes. We compared this group to the SEN group from Phase 1, both the full SEN group (N = 708) and the smaller subgroup that had received the same three parenting programs (N = 445) [see Figures 1 and 2 (Table A1 in the Appendix provides all effect sizes)]. Effect sizes were estimated to represent improvement: e.g., reduction in challenging behaviors associated with a negative sign for the effect size and increase in mental well-being associated with a positive sign for the effect size. Where the effect size is reported within | |, we refer to its absolute value disregarding the sign.

FIGURE 1

Figure 1. Effect sizes of parent outcomes between pre and post (upper chart) and between pre and follow-up (lower chart) for Phase 1 and Phase 2.

FIGURE 2

Figure 2. Effect sizes of child outcomes between pre and post (upper chart) and between pre and follow-up (lower chart) for Phase 1 and Phase 2.

Figure 1 presents the effect sizes demonstrating change in parent outcomes between pre and post (upper part) and pre and follow-up (lower part). The upper part of Figure 2 indicates moderate to large gains in parenting and parental mental well-being during Phase 1 both when we consider the overall SEN group (N = 708, effect sizes ranging from |0.62| to 0.89), and the smaller SEN group that received three of the parenting courses (N = 445, effect sizes ranging from |0.65| to −1.02). During Phase 2, the magnitude of change was moderate with effect sizes ranging from |0.45| to 0.76. All effect sizes were significant (their 95% confidence intervals did not cross 0), and they were not significantly different between them (their 95% confidence intervals overlapped). Therefore, moderate gains in parenting and parental well-being experienced by parents who received parent training as part of regular service provision were significant, and of similar magnitude to gains reported by parents who attended courses offered during the effectiveness trial.

The lower part of Figure 1 focuses on parent outcomes pre to follow-up. These effect sizes indicate how well improvements were maintained over the period of 1 year. Effect sizes during the trial phase were small to moderate (ranging from |0.42| to |0.63|) and significant, suggesting that maintenance of moderate gains in parenting and parental well-being was achieved when parenting courses were delivered during research trial. However, the effect sizes during the sustained implementation phase were small (from |0.21| to |−0.35|) and non-significant. Therefore, when parenting courses were delivered as part of regular service provision, the small gains in parenting and mental well-being after 1 year were not significant.

Figure 2 presents the effect sizes demonstrating change in child behavior between pre and post (upper part) and pre and follow-up (lower part). The upper part of Figure 1 indicates small to moderate change across behavior problems in Phase 1, i.e., the overall SEN group (N = 708, effect sizes ranging from −0.49 to −0.33), and the smaller SEN group that received three of the parenting courses (N = 445, effect sizes ranging from −0.61 to −0.46). During the sustained implementation phase (Phase 2), the magnitude of change was comparable, with effect sizes ranging from −0.54 to −0.32. Effect sizes across all groups were significant (their 95% confidence intervals did not cross 0), suggesting all SEN groups experienced significant small to moderate improvements in child behavior problems. Importantly, effect sizes were not significantly different between groups (as suggested by overlapping confidence intervals), indicating that the magnitude of change pre to post was similar across groups.

The second part of Figure 2 focuses on change between pre-course and follow-up. In Phase 1, small gains were maintained for behavior problems in the overall SEN group (N = 708): effect sizes ranged from −0.41 to −0.27. Likewise, moderate to small gains in behavior problems were maintained in the smaller SEN group from Phase 1 (N = 445): effect sizes ranged from −0.53 to −0.21. Effect sizes in the trial phase were all significant. However, the effect sizes from the sustained implementation phase (Phase 2) ranged from −0.06 to −0.20, indicating no change or very small change. The latter was non-significant, suggesting that over the course of 1 year gains in behavior problems were not maintained when parenting courses had been provided as part of regular service provision.

In sum, immediate pre–post gains in children’s behavior problems, parenting skills, and parental mental well-being were of comparable magnitude when parents of children with SEN received the course during a research trial or as part of regular service provision. One-year maintenance of these gains, however, was only achieved when courses were delivered in the context of a research trial.

Discussion

The present study was designed to address two questions: whether EBPPs not developed for children with SEN or a disability were as effective for parents of children with SEN as they were for parents of children without SEN; and whether their effectiveness could be maintained when they were delivered as part of regular service provision.

In response to the first research question, we compared families with a child with SEN to families with a child without SEN who received EBPPs as part of an effectiveness trial (Phase 1). Children with SEN were reported as having significantly higher levels of behavior and emotional problems compared to the other children. This supported previous research (Dekker et al., 2002; Emerson and Einfeld, 2010; Totsika et al., 2011a,b). However, significant improvements were similar across the two groups, as demonstrated by the lack of significant Group × Time interactions, combined with significant time effects in the ANCOVAs. Our findings support evidence that EBPPs are effective for children with SEN (Beresford, 2009; Petrenko, 2013; Skotarczak and Lee, 2015), but extend significantly the available evidence by indicating that EBPPs that have not been developed or adapted for children with disabilities are associated with significant changes in these families, similarly to other families. In fact, weighted effect sizes for behavior problems for children with SEN (Table A1 in Appendix) during the trial phase were |0.49| for conduct problems, an effect size very similar to that reported by Skotarczak and Lee (2015) who examined the effect of the disability-adapted SSTP on disruptive behaviors of children with developmental disabilities (0.48: Skotarczak and Lee, 2015).

One possibility for this similarity may be the fact that the “active components” of parenting programs are similar whether programs are developed for children with or without SEN or a disability. The content that is related to improvements in positive parenting and reductions in negative parenting (that mediate the effectiveness of EBPPs on child outcomes; Gardner et al., 2006) may be very similar across programs as it tends to be based on social learning theory and behavioral principles related to behavior management (Kaminski et al., 2008). While future research could directly compare the effectiveness of non-disability-specific EBPPs with adapted or disability-specific EBPPs for families with a child with SEN (Beresford, 2009), our study findings suggest that the needs of these two different groups of families can be met equally well by one type of EBPPs. The suggestion for this may be that parent teaching is equally effective for all groups of parents, and that parent learning translates into similar effects for children’s behavior problems. Brookman-Frazee et al. (2006) argued that the presentation of behavior problems is very similar across children with clinical disorders and developmental disorders either because of multi-morbidities or because of similar phenotypic expressions of behavior problems.

Interestingly, similar gains were experienced for parenting style, despite evidence of a differential parenting approach by parents with a child with disability (less discipline use but more negative relationship; Totsika et al., 2014). Similar gains were evident for parental mental well-being, an area that has received substantial research attention in families of children with disabilities. Evidence to date suggests that mental health problems (depression, anxiety) tend to be higher in mothers of children with developmental disabilities (Singer, 2006), but positive dimensions of mental health (positive mental health, life satisfaction) do not appear to be significantly different between mothers of children with disabilities and those without (Totsika et al., 2011a,b; Hastings, 2016). The current findings supported this lack of differences in positive mental well-being between groups and demonstrated similar improvements in well-being following EBPPs.

Findings for the second research question suggested that the overall effectiveness of EBPPs for families of children with SEN was maintained during service-led implementation. In particular, the ANCOVAs demonstrated that significant gains over time across phases (trial and sustained implementation phase) for SDQ scores, parenting laxness, and parental mental well-being. However, the effect sizes estimated using the ANCOVA-adjusted means added richer information to this question: pre to post gains in child behavior problems were of small to moderate magnitude and of similar strength across both phases. Pre to post gains in parenting skills and parental mental well-being were of moderate to large magnitude, and again, of similar strength across the two phases.

The move from research-controlled delivery to a service-controlled delivery is associated with a change in the process of delivery, particularly a change in fidelity, and that is considered crucial in the determination of effectiveness (Fixsen et al., 2005; Eames et al., 2009). In the present study, the comparison of effectiveness suggested that gains were of similar magnitude across the two phases whatever changes in delivery may have taken place over the 5 years of sustained implementation of EBPPs. The ultimate goal of implementation of EBPPs is to lead to demonstrable improvements in socially significant outcomes (Ogden and Fixsen, 2014). Demonstrating that effectiveness of EBPPs can be maintained when they are delivered as part of service provision carries an important message for policy-related decisions regarding the promotion of EBPPs as a public health approach for families of children with disabilities (Mazzucchelli and Sanders, 2011).

Our knowledge regarding the longer term effects of EBPPs comes mainly from RCTs. Long-term effects in RCTs where children do not have a developmental disability or SEN have been summarized in meta-analyses as small for both child behavior and parenting (e.g., effect sizes 0.21 and 0.25 for child and parent outcomes, respectively; Lundhal et al., 2006). For children with a developmental disability, available meta-analyses have not examined long-term maintenance (c.f., Tellegen and Sanders, 2013). However, evidence of maintenance of disability-adapted EBPPs is available from single trials of SSTP. Medium-term maintenance of SSTP effects (3 months after training) was achieved for child behavior problems, parenting style (over-reactivity), and parenting confidence (Sofronoff et al., 2011), but longer term maintenance (1 year) was achieved only for child disruptive behavior not other behavior problems, parenting, or parental confidence (Plant and Sanders, 2007). In our study, longer term maintenance (1 year) was evident for both child and parent outcomes for families with a child with SEN in the trial phase. Effect sizes (pre to follow-up) suggested small gains for child behavior problems (|0.27| to |0.41|) and small to moderate gains for parent outcomes (|0.42| to |0.63|). However, 1 year maintenance was not achieved in the sustained implementation phase: when families received EBPPs as part of regular service provision effect sizes for child behavior problems were near 0 or small (ranging from 0.06 to 0.20) and small for parent outcomes (range from 0.35 to 0.21). Program delivery during sustained implementation may have been associated with more variable gains as overall effect sizes tended to be smaller with wider confidence intervals; although they were not significantly different pre–post, the difference became more pronounced 1 year later when effects dissipated for the sustained implementation phase. Taken together with the available SSTP evidence (Plant and Sanders, 2007), there are two possibilities: either EBPPs which are not disability specific do not maintain their gains in the long-term when their delivery is service led or in families where the child has a disability longer term maintenance is not likely. The practical implication in either case is that families with a child with SEN are likely to require more regular input from services to maintain reduced levels of child behavior problems and improved parenting outcomes.

It is important to note that all findings need to be considered in light of two significant limitations in the present study. The first is that identification of SEN relied exclusively on parental report, and while the study SEN prevalence was largely in line with national prevalence, the lack of further ascertainment or detailed measurement precluded examining effectiveness for specific disabilities. The second limitation is the high levels of data missing at post and follow-up. High levels of missing data are a frequent phenomenon in interventions for families of children with developmental disabilities (up to 87% in a summary of 19 studies by McIntyre, 2013). The high levels of missingness did not render approaches such as multiple imputation appropriate, and we focused the analysis on available data only. The implication of this is that the relevance of the results is limited only to families who provided data only.

Conclusion

Early intervention, with a substantial parent training component, is currently an important component of policy and practice in the UK. The Early Intervention Foundation, a “what works” center for synthesis and implementation,² and the 10-year A Better Start program³ are flagships of the current emphasis on early intervention. So, in most English areas, parents will be able to access one or more types of EBPPs, but these are not likely to be disability specific. The present study provides important information for practice as it indicates that EBPPs not developed for families of children with SEN will be as effective for these families as they are for families whose children do not have a disability. However, practitioners need to be mindful that significant gains are probably not maintained in the longer term and these families will likely benefit from more regular provision or ongoing monitoring. EBPPs that have been developed without consideration of child SEN are no replacement for disability-specific or disability-adapted programs. However, disability-specific programs are not as widely available either because they are not yet evidence based or because services have not prioritized their provision. While researchers and policy-makers should work to address these issues, families with a child with SEN could access non-disability-specific EBPPs in the knowledge they will benefit from those as much as other families.

Author Contributions

Substantial contributions to the conception or design of the work; or the acquisition, analysis, or interpretation of data for the work; drafting the work or revising it critically for important intellectual content; final approval of the version to be published; agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved (VT, SM, and GL).

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Footnotes

References

Arnold, D. S., O’Leary, S. G., Wolff, L. S., and Acker, M. M. (1993). The parenting scale: a measure of dysfunctional parenting in discipline situations. Psychol. Assess. 5, 137–144. doi:10.1037/1040-3590.5.2.137