Using Adaptive Comparative Judgement for assessing GCSE History NEA responses

Qualifications Wales has begun exploring stakeholders’ views on various aspects of assessing new qualifications.

Since Welsh Government announced the new Curriculum for Wales, Qualifications Wales has begun exploring stakeholders’ views on various aspects of assessing the new qualifications.

We initiated research, consultation and engagement activities with various stakeholder groups to help us reflect on traditional assessment approaches, and also to help us consider opportunities for technological and methodological innovation.

One of these innovative methods is Comparative Judgement (CJ) – an assessment approach that has been the subject of a number of recent educational studies. As a regulator, we felt we needed to review the existing research on CJ and inform ourselves more fully about this method of assessment.

We found that there was a paucity of research exploring the opinions of teachers who could potentially apply CJ to high-stakes assessments. Most of the existing research evaluates the merit of CJ assessment using the results of judging (e.g. correlation/reliability coefficients, average judging time), with only a few studies exploring judges’ experiences, methods or approaches used in judging. For CJ to be considered as a viable option in assessment, we believed that a more in-depth investigation of the opinions of assessors was needed to examine whether CJ could fit into a given assessment paradigm, and whether it could address the constraints that assessors currently face with more traditional assessment methods.

The in-house research study we publish today focuses on the considerations taken by teachers when comparatively judging NEA GCSE History essays, as well as their views on using CJ to assess certain tasks. At a more general level, this research investigates whether CJ could offer a feasible, more efficient alternative to the current marking and moderation process of extended essays.

This research, along with other publications on CJ, will help Qualifications Wales understand whether CJ could be realistically considered as an efficient, fair and robust method of assessing the work of students in Wales in the future. We consider that further research and development would be needed to implement CJ in a high-stakes qualification assessment context, and intend to work with awarding bodies and external stakeholders to explore this method within wider conversations about digital assessment.