Editorial: Online assessment for humans: advancements, challenges and futures for digital assessment

Editorial on the Research Topic
Online assessment for humans: advancements, challenges and futures for digital assessment

The COVID-19 pandemic has forced educational institutions to adapt to a radically changed learning environment immediately, including adopting online assessment. However, the risks and long-term impact of such rapid evolution are unclear. This Research Topic created a space for rigorous academic debate concerning various online assessment uses in learning environments. The Research Topic, including seven papers, explores a wide range of topics related to online assessment, including power, privacy, and ethics in online invigilation (proctoring), algorithms and platforms, structural violence, gender, ethics, race, socio-economic challenges, the future of assessment, instrument design, issues of validity, reliability, and trustworthiness, the intersectionality of gender, race, socio-economic status, access, digital skills, social justice, agency, and the growing digital divide, as well as experience and learnings from online assessment. The Research Topic includes a wide range of paradigms and methodologies covering various topics. As the guest editors, we believe that the information and insights shared by the authors of this Research Topic will inform the ongoing discussions on the design, use, and interpretation of online assessments and bring new perspectives to the expanding landscape of online education.

Langenfeld et al., in Digital-first learning and assessment systems for the twenty-first century, discuss the emergence of digital-first learning and assessment systems (LAS) due to the COVID-19 pandemic. These LAS focus on the learner or test-taker experience while adhering to high-stakes learning and assessment standards. Digital-first LAS leverage human-in-the-loop artificial intelligence to enable personalized experience, feedback, adaptation, automated content generation, and automated text, speech, and video scoring. The paper proposes an ecosystem for designing digital-first LAS and highlights the need for theoretical and technological integration.

Fynn and Mashile's Continuous online assessment at a South African open distance and e-Learning institution delves into the perception of students on continuous assessment (CA) and the ethical challenges of its implementation in a South African Open, Distance and e-Learning institution. The authors suggest that open-ended submission dates, reflections, applications, and praxis assessments can address the ethical challenges, and institutions should provide devices and advocate for underprivileged students. They argue that CA provides the best opportunity for continuous and authentic assessment in higher education at scale.

Applying learning analytics in online environments: measuring learners' engagement unobtrusively, by Caspari-Sadeghi, explores the importance of data-driven decision-making (DDDM) in education, using learning analytics (LA) to collect and analyze digital traces of student engagement in online learning environments. The paper concludes that the LA research community needs to involve stakeholders to find solutions to challenges and encourage using LA to support teachers' interventions in enhancing student engagement and improving learning outcomes.

Kannan and Zapata-Rivera's article, Facilitating the use of data from multiple sources for formative learning in the context of digital assessments: informing the design and development of learning analytic dashboards examines Learning analytic dashboards (LADs) design and development. LADs are digital data visualization systems used in digital learning environments to provide students, teachers, and administrators with information about student engagement, experiences, and task performance. The goal is to develop actionable LAD systems that consolidate disparate sources of information and facilitate appropriate interpretation and use of data.

In Using content coding and automatic item generation to improve test security, Gierl et al. discuss the challenges educational testing organizations face in creating and managing large numbers of high-quality test items while ensuring security in the online testing process. They propose using automated item generation (AIG) to produce large numbers of test items and using content coding to manage the AIG model bank, which involves using a taxonomy to describe items and link information to create new types of meaningful knowledge structures about the testing organization.

Circi et al.'s article, Automatic item generation: foundations and machine learning-based approaches for assessments, highlights the need for AIG in educational assessments due to the increasing demand for items and the challenges in traditional item creation methods. The review emphasizes the potential benefits of AIG in reducing item generation time and cost and supporting customized measurement and learning needs, but also acknowledges the need for further research in this area.

Liao et al. explore the development of a real-time quality assurance system named “Analytics for Quality Assurance in Assessment” (AQuAA) in their work, Maintaining and monitoring quality of a continuously administered digital assessment. AQuAA is designed to maintain score comparability, crucial to high-stakes assessments, regardless of their delivery format. The article emphasizes the need for a combination of automatic processes and human expert review to ensure the quality and fairness of test scores. It highlights the need to customize these methods to the purposes and characteristics of different assessments.

In conclusion, this Research Topic has provided a comprehensive exploration of online assessment in the context of the COVID-19 pandemic and its implications for educational institutions. The Research Topic of papers covers a wide range of topics, including power dynamics, privacy, ethics, algorithmic systems, socio-economic challenges, and the intersectionality of various factors such as gender, race, and access. The insights shared by the authors contribute to ongoing discussions on the design, use, and interpretation of online assessments, offering new perspectives to the evolving landscape of online education. However, there are still areas that require further investigation. Future work should focus on addressing the long-term risks and impacts of rapid digital transformation in education, considering factors such as validity, reliability, trustworthiness, social justice, and the digital divide. Additionally, more research is needed to develop theoretical frameworks and technological integration for digital-first learning and assessment systems, continuous assessment implementation, learning analytics, item generation, and quality assurance. By continuing to explore these areas, researchers and practitioners can enhance the understanding and effectiveness of online assessment practices, ultimately improving educational outcomes for students in the digital age.

Author contributions

All authors listed have made a substantial, direct, and intellectual contribution to the work and approved it for publication.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.