Assessing complex skills in educational domains such as writing, designing, problem solving or art is a challenge. The method of comparative judgement is considered to be a valid and reliable alternative to the use of rubrics. In comparative judgement, students' products are evaluated by pairwise comparisons. Assessors only have to indicate which student work is the best in each pair, which is considered to be easier than assigning absolute scores to individual products. Each work is compared with several others and evaluated by several assessors. Based on these comparisons, the quality of each individual work can be estimated. This quality score reflects, so to speak, the shared consensus of the assessors.
The comparative assessment approach is based on Thurstone’s law of comparative judgement (1927), which states that it is possible to discriminate between objects on a single scale through a series of pairwise comparisons. Even though Thurstone already proposed the possibility of using comparative judgement for assessment in education, it was not until 2004 that Pollitt introduced the method in education by his paper ‘Let’s stop marking exams’. His work convincingly explained the merits of comparative assessment and provided the first evidence for a reliable summative assessment. Now, almost 2 decades later, various comparative judgement tools are available for education. Moreover, researchers around the world have investigated the reliability, validity and efficiency of the method. Each with their own approach, perspective and research focus.
In this Research Topic we aim to provide a state-of-the-art of comparative judgement in education for summative and formative purposes. We will introduce the history of comparative judgement and bring together current insights on validity, reliability and efficiency of the method. In their contributions, authors should present recent empirical research. In this way, this article collection offers the foundation for the future of educational assessment.
We welcome original empirical studies, original research and meta-analyses from around the world, as well as reviews and perspectives on the validity, reliability and/or efficiency of comparative judgement in educational settings to consolidate this topic and provide trends and challenges, as well as future research priorities.
Assessing complex skills in educational domains such as writing, designing, problem solving or art is a challenge. The method of comparative judgement is considered to be a valid and reliable alternative to the use of rubrics. In comparative judgement, students' products are evaluated by pairwise comparisons. Assessors only have to indicate which student work is the best in each pair, which is considered to be easier than assigning absolute scores to individual products. Each work is compared with several others and evaluated by several assessors. Based on these comparisons, the quality of each individual work can be estimated. This quality score reflects, so to speak, the shared consensus of the assessors.
The comparative assessment approach is based on Thurstone’s law of comparative judgement (1927), which states that it is possible to discriminate between objects on a single scale through a series of pairwise comparisons. Even though Thurstone already proposed the possibility of using comparative judgement for assessment in education, it was not until 2004 that Pollitt introduced the method in education by his paper ‘Let’s stop marking exams’. His work convincingly explained the merits of comparative assessment and provided the first evidence for a reliable summative assessment. Now, almost 2 decades later, various comparative judgement tools are available for education. Moreover, researchers around the world have investigated the reliability, validity and efficiency of the method. Each with their own approach, perspective and research focus.
In this Research Topic we aim to provide a state-of-the-art of comparative judgement in education for summative and formative purposes. We will introduce the history of comparative judgement and bring together current insights on validity, reliability and efficiency of the method. In their contributions, authors should present recent empirical research. In this way, this article collection offers the foundation for the future of educational assessment.
We welcome original empirical studies, original research and meta-analyses from around the world, as well as reviews and perspectives on the validity, reliability and/or efficiency of comparative judgement in educational settings to consolidate this topic and provide trends and challenges, as well as future research priorities.
