First Simple makes last complex: construct irrelevant variance effects in the test of grammatical comprehension
-
1
Heriot-Watt University, Psychology, United Kingdom
Reliability of a test increases with test length, offering opportunities for insights on a phenomenon. However ‘artificial inflation’ may pose psychometric disadvantages to item score. An extraneous variable (a construct Irrelevant Variance) can negatively affect assessment outcome, with items towards the end of a test receive less attention.
The current study draws upon a widely used test for sentence comprehension; the Test for Reception of Grammar (TROG-2) (Bishop, 2003). Specifically, when looking at standardised scores, the normative data illustrated as developmental trend graphs depict age equivalent scores for each construct, and appear to follow an incremental format. However, in the final constructs, participants at the higher end of the age scale, 14 to 16y display a low performance. This study examines whether the order of the constructs or ‘blocks’, although representationally coherent in an incremental complexity scale, can affect TROG-2 score. TROG-2 is suitable for children from 4 years old to adults. It has been widely used across various clinical groups (hearing impairment, SLI, Neurological patients).
Methods
40 participants (18 male) were randomly allocated in two groups. All were of undergraduate education and English native speakers. Prior to being tested with TROG-2, each participant completed a Digit Span test to establish equal working memory in both group and a minimum inclusion score of 4 for each participant.
One half of the sample, Group A, were in the control condition whereby they were tested using TROG-2 in the correct order, whilst remaining half, Group B, were presented with the blocks in TROG-2 in the reverse order.
Results
Group A (N=20) was associated with a TROG-2 test score of M=16.35 (SD=2.35), Group B (N=20) who completed the study in the reverse order, was associated with a numerically overall higher test score of M=18.15 (SD=1.23). The difference in TROG-2 overall test scores between the two groups was significant, t(28.66)=-3.04, p=.005.
Overall TROG-2 scores were calculated on a pass/fail basis per block. In order to pass a block, all four items within a block must be successfully answered. Group B scored higher in all blocks individually excluding Block A, B, C, D, L, N (see Table 1 and Figure 1). Block T scores for Group A (mean rank=14.53) and Group B (mean rank=26.48) were significantly different (p=.001).
Discussion
The aim of this study was to determine whether the directionality of which the constructs or blocks are presented to participants would affect their block score and overall score on TROG-2, assessing whether the low standardised scores reported in the final blocks can be explained by task difficulty or whether it can be attributed to an order effect. The order manipulation was effective with participants in the reverse group making significant less error on the centre-embedded relative clauses. Data will be discussed in a more comprehensive model including the interaction with working memory reported in the literature for this specific linguistic construct.
Acknowledgements
References: Bishop, D. (2003). Test for receptive grammar, TROG-2. Pearson Assessment.
References
References: Bishop, D. (2003). Test for receptive grammar, TROG-2. Pearson Assessment.
Keywords:
TROG-2,
Language assessment,
Language comprehension,
construct irrelevant variance,
grammatical test
Conference:
Academy of Aphasia 56th Annual Meeting, Montreal, Canada, 21 Oct - 23 Oct, 2018.
Presentation Type:
poster presentation
Topic:
Eligible for a student award
Citation:
Esler
M and
Garraffa
M
(2019). First Simple makes last complex: construct irrelevant variance effects in the test of grammatical comprehension.
Conference Abstract:
Academy of Aphasia 56th Annual Meeting.
doi: 10.3389/conf.fnhum.2018.228.00033
Copyright:
The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers.
They are made available through the Frontiers publishing platform as a service to conference organizers and presenters.
The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated.
Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed.
For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions.
Received:
17 Apr 2018;
Published Online:
22 Jan 2019.
*
Correspondence:
Dr. Maria Garraffa, Heriot-Watt University, Psychology, Edinburgh, United Kingdom, m.garraffa@uea.ac.uk