AUTHOR=Landrieu Yana , De Smedt Fien , Van Keer Hilde , De Wever Bram 

TITLE=Assessing the Quality of Argumentative Texts: Examining the General Agreement Between Different Rating Procedures and Exploring Inferences of (Dis)agreement Cases

JOURNAL=Frontiers in Education

VOLUME=Volume 7 - 2022

YEAR=2022

URL=https://www.frontiersin.org/journals/education/articles/10.3389/feduc.2022.784261

DOI=10.3389/feduc.2022.784261

ISSN=2504-284X

ABSTRACT=Assessing argumentative writing skills is not a straightforward task, as multiple elements need to be considered. In view of providing feedback to students and keeping track of their progress, evaluating argumentative texts in a reliable, valid and efficient way is important. In this state-of-the-art study, 130 argumentative texts written by eleventh graders were assessed by means of three different rating procedures (i.e., absolute holistic rating, comparative holistic rating, and absolute analytic rating). The main aim of this study is twofold (1) to examine the correlations between the three rating procedures and to study to what extent these procedures differ in assigning scores and (2) to identify key characteristics that determine the quality of an argumentative text, independently of the rating procedure used. Furthermore, indicators of midrange, weak and strong argumentative texts were studied in detail. The results reveal a certain level of agreement between the different procedures, indicating that all procedures are suitable to assess the quality of an argumentative text, however, each procedure has its own qualities and applicability.