Psychometric Evaluation of the Parent Effort Scale

Objective: The Parent Effort Scale (PES) is a parent report questionnaire designed to quantify the level of effort required of caregivers to assist their children in developmentally appropriate home- and community-based activities. This manuscript describes the psychometric evaluation of the PES.

Method: Data collected from 304 parents of children ages 2–7 years (167 parents of a children with autism spectrum disorder and 137 parents of neurotypical children) were factor analyzed, calibrated using item response theory, and evaluated for construct validity.

Results: The final PES scales are reliable and valid measures of the level of parental effort required to assist children in dressing, personal hygiene, sleep, socialization at home, participation in community events, and access to healthcare. A total score reflects overall parental effort.

Conclusion: The PES can be used to plan and evaluate the effectiveness of interventions that aim to help parents enhance children's participation opportunities and thus, support their cognitive and social development.

Introduction

Participation in home and community contexts is essential for children's cognitive, social, and behavioral skill development, relationships, and health-related quality of life (1–4). Preschool-aged children with autism spectrum disorder (ASD) participate in fewer self-care, community mobility, leisure, social interaction, domestic chore, and education activities than their neurotypical peers (5). For these children, participation opportunities are restricted by sensory sensitivities and social, communication, and behavioral difficulties, which are often elicited or exacerbated by task demands and triggers due to a mismatch between the child and their environment (5–7).

Parents of young children, especially those with ASD and other neurodevelopmental conditions, use a variety of strategies to enable their child's successful participation in home- and community-based activities (8, 9). Strategies include strict adherence to set routines, constant supervision, use of repeated verbal and physical prompts to support task completion, reward or punishment systems, modifying the environment to limit exposure to aversive stimuli, removing children from challenging situations, and comforting children when they become distressed (9). These strategies are designed to improve child-environment fit. However, children's behavior challenges may continue despite parents' use of multiple caregiving strategies. Persistent behavioral challenges may erode parents' confidence and thwart future attempts to facilitate participation, further limiting children's opportunities for social learning as well as child and family quality of life (10).

Parents of children with ASD describe having to “pick their battles” when considering how to balance children's participation with their capacity to manage problem behaviors (10). They may abandon or delay task demands or participation attempts if problem behaviors outweigh their energy and ability to manage them (10). Pfeiffer et al. observed a decision-making process wherein parents of children with ASD considered the amount of effort needed to support their child's participation in relation to the meaningfulness of the activity (11). Parents restricted children's participation in non-essential or non-meaningful activities when they deemed it to be too “effortful” (11). Although parent effort significantly influences the quality of family life and opportunities for children to engage in activities that are essential for their development, the concept is under-represented in existing measurement tools. A measure of parent effort could be used to identify caregivers who are either expending very high (unsustainable) levels of effort or avoiding activities with excessive effort demands. Providers could use the measure to target skill-building interventions or environmental modifications that improve environmental fit and reduce parent effort demands, ultimately enhancing children's participation opportunities.

The Parent Effort Scale (PES) measures the amount of parent effort required to enable children's participation in common developmentally appropriate home- and community-based activities. Instrument content was informed by 34 parent/caregiver interviews regarding caregiver strategies for enhancing children's participation at home and in the community (11). All participants were full time caregivers of children ages 3–7 years with ASD. Initial versions of the PES items were generated based on interview results and thereafter, items were refined based on cognitive interviews conducted with parents of children with ASD (12). Here, we describe further refinement and validation of the PES through psychometric evaluation.

Methods

All study procedures were approved by the Institutional Review Board at Temple University.

Participants

Participants were 304 parents of children ages 2 years to 7 years (M = 4.2, SD = 0.9; Table 1). Participants were purposively sampled to represent parents of children with ASD (n = 167) and neuro-typical development (n = 137). Parents were recruited by collaborating schools and via ASD support groups and social media sites.

TABLE 1

Table 1. Child and family characteristics.

Measures

Child/Family Characteristics

Parents provided sociodemographic information including child age, gender, race/ethnicity, family income, residential community type (urban, suburban, small town, rural), and the primary language spoken at home.

ASD Symptoms

Parents completed the Gilliam Autism Rating Scale–3rd edition (GARS-3), a reliable and valid norm-referenced informant-report screener for ASD (13). A total Autism Index score characterizes children's restricted/repetitive behavior, social interaction, social community, and emotional response for all children, plus cognitive style and maladaptive speech for non-verbal children. The GARS-3 Autism Index score > 70 was used to classify children into ASD and neuro-typical subgroups.

Parent Effort Scale

Parents completed 34 PES items, each describing a home- or community-based activity (Table 2). For each, parents are asked “how much effort is it for YOU to help your child participate in this activity?” All items use a 5-point Likert scale with lower score indicating less parental effort: (1) none, (2) a little, (3) some, (4) a lot, (5) too much to participate.

TABLE 2

Table 2. Parent Effort Scale (PES) item response frequencies, n (%).

Fifteen PES items assessed the level of parental effort required to enable children to participate in four types of home-based activities: dressing (three items: e.g., putting on socks and shoes), hygiene/self-care (six items: e.g., bathing, toileting), sleeping (two items: falling and staying asleep), and social/play activities (4 items: mealtime with family, play with other children). The 19 remaining PES items assessed the level of effort required for parents to help their children participate in three types of community activities: social activities (five items, e.g., parties for another child, meals at home of family or friends), community outings (12 items, e.g., movies/theaters, library activity, using a public restroom), and receipt of healthcare services (two items: dentist and doctor appointments/procedures).

Caregiver Strain Questionnaire

A randomly selected subset of 135 parents (49 parents of children with ASD, 86 parents of neurotypical children) completed the Caregiver Strain Questionnaire (CGSQ). CGSQ responders and non-responders were alike on all demographic characteristics. The CGSQ overall strain score reflects objective (e.g., disruption of personal time, financial strain, effect on work), internalized (e.g., worry, guilt, and unhappiness), and externalized (e.g., negative feelings directed toward the child, relating poorly with child) experiences of caregiver burden (14). The CGSQ has strong reliability and validity evidence for families of children with ASD (15).

Procedures

All measures were administered to parents online using Qualtrics Survey Software. A randomly selected subset of participants (n = 126) completed the PES again approximately two weeks after initial administration to enable evaluation of the measure's test-retest reliability.

Data Analysis

Response category frequencies and test-retest reliability (intraclass correlation coefficient; ICC) were calculated for each item. Chi square tests of independence were conducted to compare item response category distributions between the ASD and neurotypical subgroups. We expected to observe shared variance within subsets of PES item that reflected similar or related activities (e.g., dressing). We also expected parental effort to be correlated across activity subtypes, reflecting a higher-order effort dimension. To test these assumptions, we fit a 2-level confirmatory factor analysis (CFA) model to the data using the lavaan package in R (16). The model specified that each PES item contributed to one of six factors (dressing, hygiene/self-care, sleeping, home-based social/play activities, community-based activities, or healthcare), which themselves contributed to a higher-order total effort factor. We evaluated model fit for the full sample using three indices: Comparative Fit Index (CFI ≥ 0.95), Tucker–Lewis fit index (TLI ≥ 0.95), and Root Mean Square Error of Approximation (RMSEA ≤ 0.08) (17). Assuming adequate model fit, items with loadings ≥ 0.70 were thought to contribute to the measurement of parent effort (18, 19).

Next, we fit separate unidimensional graded response models to PES item response data for each scale to estimate item response theory (IRT) discrimination and difficulty parameters (20). The discrimination statistic (a) measures the degree to which an item differentiates respondents by their level of the latent trait (e.g., parental effort for dressing). The IRT model also produces threshold parameters (b). For a measure with five response categories, the IRT model produces four item threshold statistics (b₁-b₄). Each b parameter indicates the level of effort required of parents who endorse a specific response to a particular item (e.g., “some effort is required to assist children in putting on socks and shoes”).

Items were selected for inclusion in the final PES scales based on factor analytic and IRT analyses. Internal consistency and test-retest reliability of the final PES scales were evaluated using Cronbach's α and ICCs, respectively. Once scale composition was finalized, we calculated PES scale scores for each respondent using Bayesian Expected A Posteriori (EAP) estimation (21), which uses an individual's pattern of responses and IRT parameters to estimate standardized theta scores. The theta scores for subscales were linearly transformed to standardized scores with a mean of 10 and a standard deviation of 3 as follows: (Θ*3)+10. We calculated a total PES scoring by summing scale scores and transforming the sum score to standardized T-scores (M = 50, SD = 10). We compared PES scores between the ASD and neurotypical subgroups using independent 2-group t-tests.

To evaluate construct validity of the PES, we tested for expected score differences between ASD and neuro-typical groups using generalized lineal models. For children with ASD, we used Pearson correlation to evaluate associations between PES scores and ASD symptoms as measured by the GARS Autism Index. We hypothesized that parents of children with ASD would report higher levels of effort and that ASD symptom severity would be positively associated with parental effort. Finally, we explored associations between parent effort and caregiver strain by estimating correlations between PES scores and the CSQ Global Strain score. We expected to observe moderate associations, because parent effort and caregiver strain are related, but distinct constructs.

Results

Overall, the sample included a higher proportion of wealthier families than exist in the general US population. The majority of families (73%) had annual incomes >$60,000 and 59% earned more than $80,000 annually. Families of children with ASD and those with neurotypical children did not differ on any socio-demographic characteristic, except child gender (ASD: 76% boys, NT: 53% boys). Additional sociodemographic information is shown in Table 1.

PES item descriptive statistics are shown in Table 2. Compared to parents of neurotypical children, parents of children with ASD reported significantly higher effort levels on all PES items. For the ASD subgroup, the most effortful activities included dental appointments, attending religious services, and going to the movies or theater. Parents of neurotypical children rated hair washing and bathing as the most effortful activities. For both subgroups, playing with toys or objects was the least effortful activity for parents.

In an initial version of the two-level CFA model, 4 of the 34 items had low factor loadings (<0.70). These items assessed the parental effort required for hair care, nail care, swimming, and going to an amusement park. These items were removed. We addressed local dependency for a single item pair (child party, parent party) by removing the later item. The two-level CFA model was an excellent fit for the remaining 29 items (CFI = 0.97, TLI = 0.96, RMSEA = 0.08). Items that contributed to each subscale were internally consistent. Cronbach's alpha statistics for the subscales ranged from 0.81 to 0.95 (Table 3). Test-retest reliability was moderate or good for all PES items (ICCs: 0.51–0.82) and scales (ICCs: 0.61–0.86) (22).

TABLE 3

Table 3. Parent Effort Scale (PES) scale statistics.

Discrimination and threshold parameters from the graded response models are shown in Table 4. In the current sample, each PES item discriminated among the varying levels of effort needed to support children's activity participation (M = 2.66, SD = 0.73, range = 1.50–3.96). For all items, category thresholds were ordered as expected. The probability of endorsing higher response categories increased as the level of parental effort increased. Item-person maps for each PES scale show that in the current sample, items assessed a wide range of parental effort levels, but the items provided more information about children for whom participation requires moderate to high levels of parental effort (Figures 1–6).

TABLE 4

Table 4. Parent Effort Scale (PES) item statistics.

FIGURE 1

Figure 1. Dressing item parameters and person parameters by subgroup.

FIGURE 2

Figure 2. Hygiene/self-care item parameters and person parameters by subgroup.

FIGURE 3

Figure 3. Home social item parameters and person parameters by subgroup.

FIGURE 4

Figure 4. Sleep item parameters and person parameters by subgroup.

FIGURE 5

Figure 5. Community participation item parameters and person parameters by subgroup.

FIGURE 6

Figure 6. Healthcare item parameters and person parameters by subgroup.

Parents of children with ASD had significantly higher PES subscale and total scale scores than parents of neurotypical children (Table 3; Figure 7). ASD symptom severity, as measured by the GARS Autism Index, was positively associated with all PES subscale scores (range: 0.27−0.49) and PES Total (0.60) (Table 5). PES subscale scores are moderately correlated with the CSQ Total: Global Strain score (range: 0.49–0.69; Table 5).

FIGURE 7

Figure 7. Parent effort scale distributions—total parent effort: children with ASD vs. NT children. F_{(1, 281)} = 268.8, p < 0.0001; Black bars = children with ASD, white bars = NT children.

TABLE 5

Table 5. PES concurrent validity.

Discussion

The purpose of this study was to refine and evaluate the psychometric properties of the PES, a caregiver-report measure of the parent effort needed to enable children's participation in home- and community-based activities. Similar to the International Classification of Functioning, Disability and Health (ICF) construct of participation, the PES defines participation as involvement in a life situation (23). At this level, the child is involved in the activity; however, the PES measures the environmental factor of parental effort required to support participation in that activity rather than the capacity of the child to complete the task (23). The PES measures parental effort in six domains: (1) dressing, (2) hygiene/self-care, (3) sleep, (4) social activities in the home, (5) social activities in the community, and (6) healthcare. Subscale scores are combined to generate a total score. CFA modeling provided support for the tool's structural validity after four items were removed. These included two personal hygiene items: “hair care” and “nail care.” Two additional items, “swimming” and “going to an amusement park” were removed because they failed to contribute to the measurement of social events in the community, perhaps because they are seasonal in nature. Another item, “family parties at another's home” was removed because it provided nearly identical information as “parties for another child in the community/at another's home” in this sample. The final PES scales were found to have adequate internal consistency and test-retest reliability. The final PES subscales and total score are reliable and precise.

As expected, parents of children with ASD reported expending significantly more effort than parents of neurotypical children and ASD symptom severity was positively associated with effort levels. For children with ASD, language delays (24), sensory sensitivities (25), sleep differences (26), and behavioral rigidity and maladaptive behaviors (27) may increase parent effort demands. When parents of children with ASD perceive that they are unable to manage these challenging behaviors, they may experience internalized stigma and further restrict their child's participation opportunities for fear of others' judgement (28). For parents of children with ASD, effort demands are greater for community-based activities than for activities done at home (e.g., dressing, hygiene, sleep, and home-based social activities). Due to this, parents must often weigh the cost-benefit of their child's participation in home and community activities (29). Family with children with ASD may participate in fewer activities outside the home because the gap in child-environment fit is greater. Community contexts are less predictable and environmental modifications to support children's participation are more challenging to implement (10).

Parent effort and caregiver strain were found to be moderately correlated, indicating that they are related, but distinct constructs. Whereas parental effort describes the amount of perceived work required to complete caregiving tasks, strain describes a perceived burden of more responsibilities and demands that result in negative psychological consequences (14). Parents who expend high levels of effort across multiple activity domains for prolonged periods of time will likely experience strain (30). In contrast, parents who expend moderate (and gradually less) effort to facilitate their child's participation in a one or a small number of domains may view the effort as rewarding and a source of pride (31).

By gauging the level of parent effort within and across activity domains and over time, the PES can be used to target and evaluate the effectiveness of interventions that aim to help parents enhance their child's participation and thereby, support children's cognitive and social development. PES subscales can be used to create a “parent effort profile” that distinguishes activity domains that require minimal parent effort (strengths) from those that demand significant effort (challenges). Providers could work with families to identify child or parent skills (or skill deficits) and environmental conditions that reduce parent effort requirements for some domains and increase effort demand for others. In this way, PES results can inform the design of interventions that ease parental effort and thus, increase opportunities for children to participate. When administered repeatedly, the PES could be used to gauge the effectiveness of these interventions.

Limitations of this study include participant homogeneity and sampling bias. The sample over represents wealthier, Caucasian, and suburban families whose participation opportunities and preferences may differ from those of less well-resourced, urban, or rural families and those with different cultural backgrounds. For example, families in this study may have more opportunities to participate in costly community leisure activities and less need for public transportation. The PES should be further validated in larger and more diverse samples, including parents of children with other neurodevelopmental and chronic health conditions that influence parenting effort and children's participation.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved by Temple University IRB. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

All authors contributed to the conception and design, acquisition of data, or analysis and interpretation of data. They all drafted and revised the manuscript's important intellectual content.

Conflict of Interest

KB is employed at Janssen Pharmaceuticals Inc., and AP owns and is employed by Piller Child Development, LLC.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. Swanson J, Raab M, Dunst CJ. Strengthening family capacity to provide young children everyday natural learning opportunities. J Early Child Res. (2011) 9:66–80. doi: 10.1177/1476718X10368588

CrossRef Full Text | Google Scholar

2. King G, Lawm M, King S, Rosenbaum P, Kertoy MK, Young NL, et al. Conceptual model of the factors affecting the recreation and leisure participation of children with disabilities. Phys Occup Ther Pediatr. (2003) 23:63–90. doi: 10.1080/J006v23n01_05

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Law M. Participation in the occupations of everyday life. Am J Occup Ther. (2002) 56:640–9. doi: 10.5014/ajot.56.6.640

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Lawlor MC. The significance of being occupied: the social construction of childhood occupations. Am J Occup Ther. (2003) 57:424–34. doi: 10.5014/ajot.57.4.424

PubMed Abstract | CrossRef Full Text | Google Scholar

5. LaVesser P, Berg C. Participation patterns in preschool children with an autism spectrum disorder. OTJR Occup Particip Health. (2011) 31:33–9. doi: 10.3928/15394492-20100823-01

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Kirby AV, Williams KL, Watson LR, Sideris J, Bulluck J, Baranek GT. Sensory features and family functioning in families of children with autism and developmental disabilities: longitudinal associations. Am J Occup Ther. (2019) 73:7302205040p1. doi: 10.5014/ajot.2018.027391

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Kirby AV, White TJ, Baranek GT. Caregiver strain and sensory features in children with autism spectrum disorder and other developmental disabilities. Am J Intellect Dev Disabil. (2015) 120:32–45. doi: 10.1352/1944-7558-120.1.32

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Schaaf RC, Toth-Cohen S, Johnson SL, Outten G, Benevides TW. The everyday routines of families of children with autism: examining the impact of sensory processing difficulties on the family. Autism. (2011) 15:373–89. doi: 10.1177/1362361310386505

PubMed Abstract | CrossRef Full Text | Google Scholar

9. O'Nions E, Happé F, Evers K, Boonen H, Noens I. How do parents manage irritability, challenging behaviour, non-compliance and anxiety in children with autism spectrum disorders? A Meta-Synthesis J Autism Dev Disord. (2018) 48:1272–86. doi: 10.1007/s10803-017-3361-4

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Larson E. Caregiving and autism: how does children's propensity for routinization influence participation in family activities? OTJR Occup Particip Health. (2006) 26:69–79. doi: 10.1177/153944920602600205

CrossRef Full Text | Google Scholar

11. Pfeiffer B, Coster W, Snethen G, Derstine M, Piller A, Tucker C. Caregivers' perspectives on the sensory environment and participation in daily activities of children with autism spectrum disorder. Am J Occup Ther. (2017) 71:7104220020p1. doi: 10.5014/ajot.2017.021360

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Pfeiffer B. Development and content validity of the participation and sensory environment questionnaire. Occup Ther Ment Health. (2018) 34:105–21. doi: 10.1080/0164212X.2017.1383221

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Gilliam J. Gilliam Autism Rating Scale, 3rd Edn. Austin, TX: Pro-Ed. (2014).

Google Scholar

14. Brannan A, Heflinger C, Bickman L. The caregiver strain questionnaire: measuring the impact on the family of living with a child with serious emotional disturbance. J Emot Behav Disord. (1997) 5:212–22. doi: 10.1177/106342669700500404

CrossRef Full Text | Google Scholar

15. Khanna R, Madhavan SS, Smith MJ, Tworek C, Patrick JH, Becker-Cottrill B. Psychometric properties of the Caregiver Strain Questionnaire (CGSQ) among caregivers of children with autism. Autism. (2012) 16:179–99. doi: 10.1177/1362361311406143

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Yves R. lavaan: An R package for structural equation modeling. J Stat Softw. (2012) 48:1–36. doi: 10.18637/jss.v048.i02

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Hu L, Bentler PM. Cuttoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Model. (1999) 6:1–55. doi: 10.1080/10705519909540118

CrossRef Full Text | Google Scholar

18. Carle AC, Weech-Maldonado R. Validly interpreting patients' reports: using bifactor and multidimensional models to determine whether surveys and scales measure one or more constructs. Med Care. (2012) 50(9 Suppl 2):S42–8. doi: 10.1097/MLR.0b013e318266519e

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Reise SP, Morizot J, Hays RD. The role of the bifactor model in resolving dimensionality issues in health outcomes measures. Qual Life Res. (2007) 16:19–31. doi: 10.1007/s11136-007-9183-7

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Samejima F. The general graded response model. In: Nering M, Ostini R, editors. Handbook of Polytomous Item Response Theory Models. New York, NY: Routledge (2010). p. 77–107.

Google Scholar

21. Bock R, Aitkin M. Marginal maximum likelihood estimation of item parameters: application of an EM algorithm. Psychometrika. (1981) 46:443–59. doi: 10.1007/BF02293801

CrossRef Full Text | Google Scholar

22. Koo TK Li MY. A Guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med. (2016) 15:155–63. doi: 10.1016/j.jcm.2016.02.012

PubMed Abstract | CrossRef Full Text | Google Scholar

23. World Health Organization. International Classification of Functioning, Disability and Health : ICF. Washington, DC: World Health Organization; 2001.

Google Scholar

24. Eigsti I-M, de Marchena AB, Schuh JM, Kelley E. Language acquisition in autism spectrum disorders: a developmental review. Res Autism Spectr Disord. (2011) 5:681–91. doi: 10.1016/j.rasd.2010.09.001

CrossRef Full Text | Google Scholar

25. Posar A, Visconti P. Sensory abnormalities in children with autism spectrum disorder. J Pediatr (Rio J). (2018) 94:342–50. doi: 10.1016/j.jped.2017.08.008

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Devnani P, Hegde A. Autism and sleep disorders. J Pediatr Neurosci. (2015) 10:304. doi: 10.4103/1817-1745.174438

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Hartley SL, Sikora DM, McCoy R. Prevalence and risk factors of maladaptive behaviour in young children with autistic disorder. J Intellect Disabil Res. (2008) 52:819–29. doi: 10.1111/j.1365-2788.2008.01065.x

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Mak WWS, Kwok YTY. Internalization of stigma for parents of children with autism spectrum disorder in Hong Kong. Soc Sci Med. (2010) 70:2045–51. doi: 10.1016/j.socscimed.2010.02.023

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Werner DeGrace B. The everyday occupation of families with children with autism. Am J Occup Ther. (2004) 58:543–50. doi: 10.5014/ajot.58.5.543

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Keeton VF, Trask J, Whitney R, Bell JF. Overburdened and underprepared: medical/nursing task performance among informal caregivers in the United States. J Gerontol Nurs. (2020) 46:25–35. doi: 10.3928/00989134-20200811-05

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Hastings RP, Taunt HM. Positive perceptions in families of children with developmental disabilities. Am J Ment Retard. (2002) 107:116–27. doi: 10.1352/0895-8017(2002)107<0116:PPIFOC>2.0.CO;2

PubMed Abstract | CrossRef Full Text | Google Scholar