- 1Janssen Pharmaceuticals Inc., Horsham, PA, United States
- 2Department of Health and Rehabilitation Sciences, Temple University College of Public Health, Philadelphia, PA, United States
- 3Piller Child Development, LLC, Phoenix, AZ, United States
Objective: The Parent Effort Scale (PES) is a parent report questionnaire designed to quantify the level of effort required of caregivers to assist their children in developmentally appropriate home- and community-based activities. This manuscript describes the psychometric evaluation of the PES.
Method: Data collected from 304 parents of children ages 2–7 years (167 parents of a children with autism spectrum disorder and 137 parents of neurotypical children) were factor analyzed, calibrated using item response theory, and evaluated for construct validity.
Results: The final PES scales are reliable and valid measures of the level of parental effort required to assist children in dressing, personal hygiene, sleep, socialization at home, participation in community events, and access to healthcare. A total score reflects overall parental effort.
Conclusion: The PES can be used to plan and evaluate the effectiveness of interventions that aim to help parents enhance children's participation opportunities and thus, support their cognitive and social development.
Introduction
Participation in home and community contexts is essential for children's cognitive, social, and behavioral skill development, relationships, and health-related quality of life (1–4). Preschool-aged children with autism spectrum disorder (ASD) participate in fewer self-care, community mobility, leisure, social interaction, domestic chore, and education activities than their neurotypical peers (5). For these children, participation opportunities are restricted by sensory sensitivities and social, communication, and behavioral difficulties, which are often elicited or exacerbated by task demands and triggers due to a mismatch between the child and their environment (5–7).
Parents of young children, especially those with ASD and other neurodevelopmental conditions, use a variety of strategies to enable their child's successful participation in home- and community-based activities (8, 9). Strategies include strict adherence to set routines, constant supervision, use of repeated verbal and physical prompts to support task completion, reward or punishment systems, modifying the environment to limit exposure to aversive stimuli, removing children from challenging situations, and comforting children when they become distressed (9). These strategies are designed to improve child-environment fit. However, children's behavior challenges may continue despite parents' use of multiple caregiving strategies. Persistent behavioral challenges may erode parents' confidence and thwart future attempts to facilitate participation, further limiting children's opportunities for social learning as well as child and family quality of life (10).
Parents of children with ASD describe having to “pick their battles” when considering how to balance children's participation with their capacity to manage problem behaviors (10). They may abandon or delay task demands or participation attempts if problem behaviors outweigh their energy and ability to manage them (10). Pfeiffer et al. observed a decision-making process wherein parents of children with ASD considered the amount of effort needed to support their child's participation in relation to the meaningfulness of the activity (11). Parents restricted children's participation in non-essential or non-meaningful activities when they deemed it to be too “effortful” (11). Although parent effort significantly influences the quality of family life and opportunities for children to engage in activities that are essential for their development, the concept is under-represented in existing measurement tools. A measure of parent effort could be used to identify caregivers who are either expending very high (unsustainable) levels of effort or avoiding activities with excessive effort demands. Providers could use the measure to target skill-building interventions or environmental modifications that improve environmental fit and reduce parent effort demands, ultimately enhancing children's participation opportunities.
The Parent Effort Scale (PES) measures the amount of parent effort required to enable children's participation in common developmentally appropriate home- and community-based activities. Instrument content was informed by 34 parent/caregiver interviews regarding caregiver strategies for enhancing children's participation at home and in the community (11). All participants were full time caregivers of children ages 3–7 years with ASD. Initial versions of the PES items were generated based on interview results and thereafter, items were refined based on cognitive interviews conducted with parents of children with ASD (12). Here, we describe further refinement and validation of the PES through psychometric evaluation.
Methods
All study procedures were approved by the Institutional Review Board at Temple University.
Participants
Participants were 304 parents of children ages 2 years to 7 years (M = 4.2, SD = 0.9; Table 1). Participants were purposively sampled to represent parents of children with ASD (n = 167) and neuro-typical development (n = 137). Parents were recruited by collaborating schools and via ASD support groups and social media sites.
Measures
Child/Family Characteristics
Parents provided sociodemographic information including child age, gender, race/ethnicity, family income, residential community type (urban, suburban, small town, rural), and the primary language spoken at home.
ASD Symptoms
Parents completed the Gilliam Autism Rating Scale–3rd edition (GARS-3), a reliable and valid norm-referenced informant-report screener for ASD (13). A total Autism Index score characterizes children's restricted/repetitive behavior, social interaction, social community, and emotional response for all children, plus cognitive style and maladaptive speech for non-verbal children. The GARS-3 Autism Index score > 70 was used to classify children into ASD and neuro-typical subgroups.
Parent Effort Scale
Parents completed 34 PES items, each describing a home- or community-based activity (Table 2). For each, parents are asked “how much effort is it for YOU to help your child participate in this activity?” All items use a 5-point Likert scale with lower score indicating less parental effort: (1) none, (2) a little, (3) some, (4) a lot, (5) too much to participate.
Fifteen PES items assessed the level of parental effort required to enable children to participate in four types of home-based activities: dressing (three items: e.g., putting on socks and shoes), hygiene/self-care (six items: e.g., bathing, toileting), sleeping (two items: falling and staying asleep), and social/play activities (4 items: mealtime with family, play with other children). The 19 remaining PES items assessed the level of effort required for parents to help their children participate in three types of community activities: social activities (five items, e.g., parties for another child, meals at home of family or friends), community outings (12 items, e.g., movies/theaters, library activity, using a public restroom), and receipt of healthcare services (two items: dentist and doctor appointments/procedures).
Caregiver Strain Questionnaire
A randomly selected subset of 135 parents (49 parents of children with ASD, 86 parents of neurotypical children) completed the Caregiver Strain Questionnaire (CGSQ). CGSQ responders and non-responders were alike on all demographic characteristics. The CGSQ overall strain score reflects objective (e.g., disruption of personal time, financial strain, effect on work), internalized (e.g., worry, guilt, and unhappiness), and externalized (e.g., negative feelings directed toward the child, relating poorly with child) experiences of caregiver burden (14). The CGSQ has strong reliability and validity evidence for families of children with ASD (15).
Procedures
All measures were administered to parents online using Qualtrics Survey Software. A randomly selected subset of participants (n = 126) completed the PES again approximately two weeks after initial administration to enable evaluation of the measure's test-retest reliability.
Data Analysis
Response category frequencies and test-retest reliability (intraclass correlation coefficient; ICC) were calculated for each item. Chi square tests of independence were conducted to compare item response category distributions between the ASD and neurotypical subgroups. We expected to observe shared variance within subsets of PES item that reflected similar or related activities (e.g., dressing). We also expected parental effort to be correlated across activity subtypes, reflecting a higher-order effort dimension. To test these assumptions, we fit a 2-level confirmatory factor analysis (CFA) model to the data using the lavaan package in R (16). The model specified that each PES item contributed to one of six factors (dressing, hygiene/self-care, sleeping, home-based social/play activities, community-based activities, or healthcare), which themselves contributed to a higher-order total effort factor. We evaluated model fit for the full sample using three indices: Comparative Fit Index (CFI ≥ 0.95), Tucker–Lewis fit index (TLI ≥ 0.95), and Root Mean Square Error of Approximation (RMSEA ≤ 0.08) (17). Assuming adequate model fit, items with loadings ≥ 0.70 were thought to contribute to the measurement of parent effort (18, 19).
Next, we fit separate unidimensional graded response models to PES item response data for each scale to estimate item response theory (IRT) discrimination and difficulty parameters (20). The discrimination statistic (a) measures the degree to which an item differentiates respondents by their level of the latent trait (e.g., parental effort for dressing). The IRT model also produces threshold parameters (b). For a measure with five response categories, the IRT model produces four item threshold statistics (b1-b4). Each b parameter indicates the level of effort required of parents who endorse a specific response to a particular item (e.g., “some effort is required to assist children in putting on socks and shoes”).
Items were selected for inclusion in the final PES scales based on factor analytic and IRT analyses. Internal consistency and test-retest reliability of the final PES scales were evaluated using Cronbach's α and ICCs, respectively. Once scale composition was finalized, we calculated PES scale scores for each respondent using Bayesian Expected A Posteriori (EAP) estimation (21), which uses an individual's pattern of responses and IRT parameters to estimate standardized theta scores. The theta scores for subscales were linearly transformed to standardized scores with a mean of 10 and a standard deviation of 3 as follows: (Θ*3)+10. We calculated a total PES scoring by summing scale scores and transforming the sum score to standardized T-scores (M = 50, SD = 10). We compared PES scores between the ASD and neurotypical subgroups using independent 2-group t-tests.
To evaluate construct validity of the PES, we tested for expected score differences between ASD and neuro-typical groups using generalized lineal models. For children with ASD, we used Pearson correlation to evaluate associations between PES scores and ASD symptoms as measured by the GARS Autism Index. We hypothesized that parents of children with ASD would report higher levels of effort and that ASD symptom severity would be positively associated with parental effort. Finally, we explored associations between parent effort and caregiver strain by estimating correlations between PES scores and the CSQ Global Strain score. We expected to observe moderate associations, because parent effort and caregiver strain are related, but distinct constructs.
Results
Overall, the sample included a higher proportion of wealthier families than exist in the general US population. The majority of families (73%) had annual incomes >$60,000 and 59% earned more than $80,000 annually. Families of children with ASD and those with neurotypical children did not differ on any socio-demographic characteristic, except child gender (ASD: 76% boys, NT: 53% boys). Additional sociodemographic information is shown in Table 1.
PES item descriptive statistics are shown in Table 2. Compared to parents of neurotypical children, parents of children with ASD reported significantly higher effort levels on all PES items. For the ASD subgroup, the most effortful activities included dental appointments, attending religious services, and going to the movies or theater. Parents of neurotypical children rated hair washing and bathing as the most effortful activities. For both subgroups, playing with toys or objects was the least effortful activity for parents.
In an initial version of the two-level CFA model, 4 of the 34 items had low factor loadings (<0.70). These items assessed the parental effort required for hair care, nail care, swimming, and going to an amusement park. These items were removed. We addressed local dependency for a single item pair (child party, parent party) by removing the later item. The two-level CFA model was an excellent fit for the remaining 29 items (CFI = 0.97, TLI = 0.96, RMSEA = 0.08). Items that contributed to each subscale were internally consistent. Cronbach's alpha statistics for the subscales ranged from 0.81 to 0.95 (Table 3). Test-retest reliability was moderate or good for all PES items (ICCs: 0.51–0.82) and scales (ICCs: 0.61–0.86) (22).
Discrimination and threshold parameters from the graded response models are shown in Table 4. In the current sample, each PES item discriminated among the varying levels of effort needed to support children's activity participation (M = 2.66, SD = 0.73, range = 1.50–3.96). For all items, category thresholds were ordered as expected. The probability of endorsing higher response categories increased as the level of parental effort increased. Item-person maps for each PES scale show that in the current sample, items assessed a wide range of parental effort levels, but the items provided more information about children for whom participation requires moderate to high levels of parental effort (Figures 1–6).
Parents of children with ASD had significantly higher PES subscale and total scale scores than parents of neurotypical children (Table 3; Figure 7). ASD symptom severity, as measured by the GARS Autism Index, was positively associated with all PES subscale scores (range: 0.27−0.49) and PES Total (0.60) (Table 5). PES subscale scores are moderately correlated with the CSQ Total: Global Strain score (range: 0.49–0.69; Table 5).
![www.frontiersin.org](https://www.frontiersin.org/files/Articles/780302/fresc-03-780302-HTML/image_m/fresc-03-780302-g007.jpg)
Figure 7. Parent effort scale distributions—total parent effort: children with ASD vs. NT children. F(1, 281) = 268.8, p < 0.0001; Black bars = children with ASD, white bars = NT children.
Discussion
The purpose of this study was to refine and evaluate the psychometric properties of the PES, a caregiver-report measure of the parent effort needed to enable children's participation in home- and community-based activities. Similar to the International Classification of Functioning, Disability and Health (ICF) construct of participation, the PES defines participation as involvement in a life situation (23). At this level, the child is involved in the activity; however, the PES measures the environmental factor of parental effort required to support participation in that activity rather than the capacity of the child to complete the task (23). The PES measures parental effort in six domains: (1) dressing, (2) hygiene/self-care, (3) sleep, (4) social activities in the home, (5) social activities in the community, and (6) healthcare. Subscale scores are combined to generate a total score. CFA modeling provided support for the tool's structural validity after four items were removed. These included two personal hygiene items: “hair care” and “nail care.” Two additional items, “swimming” and “going to an amusement park” were removed because they failed to contribute to the measurement of social events in the community, perhaps because they are seasonal in nature. Another item, “family parties at another's home” was removed because it provided nearly identical information as “parties for another child in the community/at another's home” in this sample. The final PES scales were found to have adequate internal consistency and test-retest reliability. The final PES subscales and total score are reliable and precise.
As expected, parents of children with ASD reported expending significantly more effort than parents of neurotypical children and ASD symptom severity was positively associated with effort levels. For children with ASD, language delays (24), sensory sensitivities (25), sleep differences (26), and behavioral rigidity and maladaptive behaviors (27) may increase parent effort demands. When parents of children with ASD perceive that they are unable to manage these challenging behaviors, they may experience internalized stigma and further restrict their child's participation opportunities for fear of others' judgement (28). For parents of children with ASD, effort demands are greater for community-based activities than for activities done at home (e.g., dressing, hygiene, sleep, and home-based social activities). Due to this, parents must often weigh the cost-benefit of their child's participation in home and community activities (29). Family with children with ASD may participate in fewer activities outside the home because the gap in child-environment fit is greater. Community contexts are less predictable and environmental modifications to support children's participation are more challenging to implement (10).
Parent effort and caregiver strain were found to be moderately correlated, indicating that they are related, but distinct constructs. Whereas parental effort describes the amount of perceived work required to complete caregiving tasks, strain describes a perceived burden of more responsibilities and demands that result in negative psychological consequences (14). Parents who expend high levels of effort across multiple activity domains for prolonged periods of time will likely experience strain (30). In contrast, parents who expend moderate (and gradually less) effort to facilitate their child's participation in a one or a small number of domains may view the effort as rewarding and a source of pride (31).
By gauging the level of parent effort within and across activity domains and over time, the PES can be used to target and evaluate the effectiveness of interventions that aim to help parents enhance their child's participation and thereby, support children's cognitive and social development. PES subscales can be used to create a “parent effort profile” that distinguishes activity domains that require minimal parent effort (strengths) from those that demand significant effort (challenges). Providers could work with families to identify child or parent skills (or skill deficits) and environmental conditions that reduce parent effort requirements for some domains and increase effort demand for others. In this way, PES results can inform the design of interventions that ease parental effort and thus, increase opportunities for children to participate. When administered repeatedly, the PES could be used to gauge the effectiveness of these interventions.
Limitations of this study include participant homogeneity and sampling bias. The sample over represents wealthier, Caucasian, and suburban families whose participation opportunities and preferences may differ from those of less well-resourced, urban, or rural families and those with different cultural backgrounds. For example, families in this study may have more opportunities to participate in costly community leisure activities and less need for public transportation. The PES should be further validated in larger and more diverse samples, including parents of children with other neurodevelopmental and chronic health conditions that influence parenting effort and children's participation.
Data Availability Statement
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.
Ethics Statement
The studies involving human participants were reviewed and approved by Temple University IRB. The patients/participants provided their written informed consent to participate in this study.
Author Contributions
All authors contributed to the conception and design, acquisition of data, or analysis and interpretation of data. They all drafted and revised the manuscript's important intellectual content.
Conflict of Interest
KB is employed at Janssen Pharmaceuticals Inc., and AP owns and is employed by Piller Child Development, LLC.
The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher's Note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
1. Swanson J, Raab M, Dunst CJ. Strengthening family capacity to provide young children everyday natural learning opportunities. J Early Child Res. (2011) 9:66–80. doi: 10.1177/1476718X10368588
2. King G, Lawm M, King S, Rosenbaum P, Kertoy MK, Young NL, et al. Conceptual model of the factors affecting the recreation and leisure participation of children with disabilities. Phys Occup Ther Pediatr. (2003) 23:63–90. doi: 10.1080/J006v23n01_05
3. Law M. Participation in the occupations of everyday life. Am J Occup Ther. (2002) 56:640–9. doi: 10.5014/ajot.56.6.640
4. Lawlor MC. The significance of being occupied: the social construction of childhood occupations. Am J Occup Ther. (2003) 57:424–34. doi: 10.5014/ajot.57.4.424
5. LaVesser P, Berg C. Participation patterns in preschool children with an autism spectrum disorder. OTJR Occup Particip Health. (2011) 31:33–9. doi: 10.3928/15394492-20100823-01
6. Kirby AV, Williams KL, Watson LR, Sideris J, Bulluck J, Baranek GT. Sensory features and family functioning in families of children with autism and developmental disabilities: longitudinal associations. Am J Occup Ther. (2019) 73:7302205040p1. doi: 10.5014/ajot.2018.027391
7. Kirby AV, White TJ, Baranek GT. Caregiver strain and sensory features in children with autism spectrum disorder and other developmental disabilities. Am J Intellect Dev Disabil. (2015) 120:32–45. doi: 10.1352/1944-7558-120.1.32
8. Schaaf RC, Toth-Cohen S, Johnson SL, Outten G, Benevides TW. The everyday routines of families of children with autism: examining the impact of sensory processing difficulties on the family. Autism. (2011) 15:373–89. doi: 10.1177/1362361310386505
9. O'Nions E, Happé F, Evers K, Boonen H, Noens I. How do parents manage irritability, challenging behaviour, non-compliance and anxiety in children with autism spectrum disorders? A Meta-Synthesis J Autism Dev Disord. (2018) 48:1272–86. doi: 10.1007/s10803-017-3361-4
10. Larson E. Caregiving and autism: how does children's propensity for routinization influence participation in family activities? OTJR Occup Particip Health. (2006) 26:69–79. doi: 10.1177/153944920602600205
11. Pfeiffer B, Coster W, Snethen G, Derstine M, Piller A, Tucker C. Caregivers' perspectives on the sensory environment and participation in daily activities of children with autism spectrum disorder. Am J Occup Ther. (2017) 71:7104220020p1. doi: 10.5014/ajot.2017.021360
12. Pfeiffer B. Development and content validity of the participation and sensory environment questionnaire. Occup Ther Ment Health. (2018) 34:105–21. doi: 10.1080/0164212X.2017.1383221
14. Brannan A, Heflinger C, Bickman L. The caregiver strain questionnaire: measuring the impact on the family of living with a child with serious emotional disturbance. J Emot Behav Disord. (1997) 5:212–22. doi: 10.1177/106342669700500404
15. Khanna R, Madhavan SS, Smith MJ, Tworek C, Patrick JH, Becker-Cottrill B. Psychometric properties of the Caregiver Strain Questionnaire (CGSQ) among caregivers of children with autism. Autism. (2012) 16:179–99. doi: 10.1177/1362361311406143
16. Yves R. lavaan: An R package for structural equation modeling. J Stat Softw. (2012) 48:1–36. doi: 10.18637/jss.v048.i02
17. Hu L, Bentler PM. Cuttoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Model. (1999) 6:1–55. doi: 10.1080/10705519909540118
18. Carle AC, Weech-Maldonado R. Validly interpreting patients' reports: using bifactor and multidimensional models to determine whether surveys and scales measure one or more constructs. Med Care. (2012) 50(9 Suppl 2):S42–8. doi: 10.1097/MLR.0b013e318266519e
19. Reise SP, Morizot J, Hays RD. The role of the bifactor model in resolving dimensionality issues in health outcomes measures. Qual Life Res. (2007) 16:19–31. doi: 10.1007/s11136-007-9183-7
20. Samejima F. The general graded response model. In: Nering M, Ostini R, editors. Handbook of Polytomous Item Response Theory Models. New York, NY: Routledge (2010). p. 77–107.
21. Bock R, Aitkin M. Marginal maximum likelihood estimation of item parameters: application of an EM algorithm. Psychometrika. (1981) 46:443–59. doi: 10.1007/BF02293801
22. Koo TK Li MY. A Guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med. (2016) 15:155–63. doi: 10.1016/j.jcm.2016.02.012
23. World Health Organization. International Classification of Functioning, Disability and Health : ICF. Washington, DC: World Health Organization; 2001.
24. Eigsti I-M, de Marchena AB, Schuh JM, Kelley E. Language acquisition in autism spectrum disorders: a developmental review. Res Autism Spectr Disord. (2011) 5:681–91. doi: 10.1016/j.rasd.2010.09.001
25. Posar A, Visconti P. Sensory abnormalities in children with autism spectrum disorder. J Pediatr (Rio J). (2018) 94:342–50. doi: 10.1016/j.jped.2017.08.008
26. Devnani P, Hegde A. Autism and sleep disorders. J Pediatr Neurosci. (2015) 10:304. doi: 10.4103/1817-1745.174438
27. Hartley SL, Sikora DM, McCoy R. Prevalence and risk factors of maladaptive behaviour in young children with autistic disorder. J Intellect Disabil Res. (2008) 52:819–29. doi: 10.1111/j.1365-2788.2008.01065.x
28. Mak WWS, Kwok YTY. Internalization of stigma for parents of children with autism spectrum disorder in Hong Kong. Soc Sci Med. (2010) 70:2045–51. doi: 10.1016/j.socscimed.2010.02.023
29. Werner DeGrace B. The everyday occupation of families with children with autism. Am J Occup Ther. (2004) 58:543–50. doi: 10.5014/ajot.58.5.543
30. Keeton VF, Trask J, Whitney R, Bell JF. Overburdened and underprepared: medical/nursing task performance among informal caregivers in the United States. J Gerontol Nurs. (2020) 46:25–35. doi: 10.3928/00989134-20200811-05
Keywords: measurement, autism spectrum disorder (ASD), parent effort, pediatric, psychometric
Citation: Bevans KB, Hallock TM, Piller A and Pfeiffer B (2022) Psychometric Evaluation of the Parent Effort Scale. Front. Rehabilit. Sci. 3:780302. doi: 10.3389/fresc.2022.780302
Received: 20 September 2021; Accepted: 04 January 2022;
Published: 02 February 2022.
Edited by:
Carole A. Tucker, University of Texas Medical Branch at Galveston, United StatesReviewed by:
Carolina Saskia Fellinghauer, University of Zurich, SwitzerlandAlexander Shoshmin, Albrecht Federal Scientific Centre of Rehabilitation of the Disabled, Russia
Copyright © 2022 Bevans, Hallock, Piller and Pfeiffer. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Beth Pfeiffer, bpfeiffe@temple.edu