Commentary: Relationships between mathematics performance and attitude to mathematics

A Commentary on
Relationships between mathematics performance and attitude to mathematics: Influences of gender, test anxiety, and working memory

by Dowker, A., and Sheridan, H. (2022). Front. Psychol 13:814992. doi: 10.3389/fpsyg.2022.814992

Introduction

Psychology is one of the scientific fields in which the dangers of publications based on a small number of subjects have been commented on very early and comprehensively: see for example Cohen's (1969) work on sample size calculation in the late 60s and Maxwell's (2004) report on the low power of many psychological studies due to insufficient sample size. In this context, I was surprised to read an article in Frontiers Educational Psychology published in March 2022 regarding the results of a 2 × 2 factorial design with 10 subjects per cell (Dowker and Sheridan, 2022).

The problem

The authors did not provide calculations of sample size. However, for a power of 0.80, a type I error of 0.05 and a medium size effect, 128 subjects are needed for a two-way ANOVA or for a multiple linear regression with three predictors (the other analysis the authors conducted) a total of 77 subjects. Since in the context of educational psychology, a large size effect is very unlikely, this study with 40 subjects is more than “somewhat underpowered” (p. 6). The authors resolve the problem by doing frequentist and Bayesian inference, which means that they double the number of statistical tests performed. This is not really the solution because the global type I error increases drastically.

Discussion

Please do not misunderstand the purpose of this commentary: it is not meant as a reproach to the authors but as a reflection on solutions.

It seems that the teaching of statistics and methodology in the cursus of psychology is insufficiently focused on sample size issues because this problem still arises in Frontiers and other psychological reviews. There should be a rethinking of how instructors teach this aspect in higher education, i.e. move more on conceptual issues and consequences than on technicalities. Focusing on real data, Spiegelhalter (2019) perfectly clarified the importance of sample size.

Then, it is the duty of a scientific journal's editors in chief to apply a minimum of statistical reviewing because they act as gatekeepers of scientific quality. First, they could edit guidelines for statistical reporting to help authors improve it. The American Psychological Association (2020) manual is very clear on the issue of sample size, but this matter is perhaps lost in the more than 700 pages of its seventh edition. Therefore, reviews need guidelines for authors – inspired by those of high-quality reviews – that involve the key statistical elements that are important for the study's validity (sample size, verification of the conditions for applying a statistical analysis, etc). These guidelines could also mention books or articles on this topic, that could help the authors on this issue. Second, a statistical review should be conducted by the editorial team or by qualified statisticians recruited as reviewers – which is done in some medical reviews – or by selecting at least one reviewer with strong statistical skills, who would receive the task of paying specific attention to statistical aspects. Finally, as Altman (2002) suggested, reviews could encourage letters adopting a critical approach to statistical reporting in published articles.

Author contributions

The author confirms being the sole contributor of this work and has approved it for publication.

Funding

Open access funding by the University of Lausanne.

Conflict of interest

The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Altman, D. (2002). Poor-quality medical research: what can journals do? J Am. Med. Assoc. 287, 2765–2767. doi: 10.1001/jama.287.21.2765

PubMed Abstract | CrossRef Full Text | Google Scholar

American Psychological Association (2020). Publication Manual of the American Psychological Association, 7th Edn. Washington, DC: American Psychological Association.

Google Scholar

Cohen, J. (1969). Statistical Power for the Behavioral Sciences. San Diego, CA, Academic Press.

Google Scholar

Dowker, A., and Sheridan, H. (2022). Relationships between mathematics performance and attitude to mathematics: Influences of gender, test anxiety, and working memory. Front. Psychol. 13, 814992. doi: 10.3389/fpsyg.2022.814992

PubMed Abstract | CrossRef Full Text | Google Scholar

Maxwell, S. E. (2004). The persistence of underpowered studies in psychology research: causes, consequences, and remedies. Psychol. Methods 9, 147–163. doi: 10.1037/1082-989X.9.2.147

PubMed Abstract | CrossRef Full Text | Google Scholar

Spiegelhalter, D. (2019). The Art of Statistics: Learning from Data. London: Penguin Random House.

Google Scholar