Fairness in a year of transition

Earlier this year, I interrogated the question of fairness and looked at the rationale for a return to more normal assessment arrangements for 2022. At that time, there were still some questions over whether it would be possible for exams to go ahead – the Omicron variant was rampant and having a significant impact on education.

I am glad to say that summer exams have now happened and apart from a few bumps, which are unfortunately inevitable in a system as complex as public examinations, things have gone broadly well.

What does the picture look like now?

I have spoken to several school and college leaders over recent weeks and the feedback has been overwhelmingly positive. They have appreciated the return to more normal arrangements, particularly that a consistent and objective standard is once again being applied to all learners.  They’ve also welcomed the space to focus on their core purpose of teaching and learning rather than the distraction of deciding centre determined grades.

School and college leaders also confirm that learners have been grateful for a return to more normal arrangements. We know that anxieties are heightened this year, probably more than any other, for a variety of reasons:

• post-16 learners have not taken public exams before so will be more anxious than normal
• all learners (and indeed teachers) will be anxious about the impact of a disrupted education on their final grades.

As I said back in February, in situations like this you must consider the alternatives too. We know that learners are anxious, but a different set of anxieties arise with centre determined grades – the approach used last year. Learners worry that they may not be treated in the same way as their peers, introducing issues of fairness across learners and centres (relational fairness), others have reported the concern that they haven’t been assessed in a normal way: expressed to me by one headteacher as ‘imposter syndrome’.

I would like to take this opportunity to say well done to learners who have done their best to demonstrate what they know in difficult circumstances, to teachers and lecturers who have done their best to prepare learners for their assessments and to everyone in the wider education and qualifications system who have worked tirelessly and diligently to support them. The recovery of normal arrangements was never going to be easy, which is why this is very much a transition year.

A year of transition – adaptations and setting grade boundaries

As part of that move back to regular arrangements, adaptations were made to ease the assessment burden and we have announced a transitionary approach to standards.

First let’s look at the adaptations. In Wales, for most qualifications, changes were put in place in summer 2021 to reduce the assessment requirements this academic year. This allowed schools and colleges to focus their time on areas that were most important to the subject. This wasn’t intended to overly limit teaching, as much of the learning is needed for progression in the subject, but it did provide a means of reducing some of the pressure.

We will be undertaking work to look at how effective this approach was later this year. Overall, negative feedback has been limited to concerns about whether a small number of specific questions in some subjects should or should not have been asked given the stated adaptations. Learners, parents and educators have been using our regular exam series questionnaire, ‘Have Your Say’, to identify specific concerns and we have been and will continue to monitor exam boards closely to see their responses to any issues that arise.

The second main area of difference in this transition year is the approach to awarding – the setting of grade boundaries by exam boards to convert marks into grades.

We said earlier this year that we would require WJEC – the awarding body for Wales -to award grades so that they are broadly midway between 2019 and 2021; 2019 being the last year when all exams were marked by external examiners and 2021 the year when individual schools and colleges provided ‘centre determined grades’. This gives us a transition between the two approaches and a steppingstone to returning to the ‘established performance standard’.

In terms of fairness, some may say that this is not fair to learners past or future as there are no direct comparisons with other years. This is something that we considered when making this decision. However, we must recognise that learners have had the most extraordinary two years leading up to their exams.

The last thing we want to see is an overly harsh approach given the circumstances and context, and we do want to see a safety net for learners. This approach to grading is consistent with that taken in England, so learners in Wales will be treated in the same way as their peers across the border.

Are exams more difficult this year?

Those learners that have provided us with feedback, and that is only a small self-selecting sample, have been concerned that this year’s papers have been more difficult than usual. There may be many reasons why this is the case, and it is worth taking a moment to reflect on this.

First, we must remember that this is a return to unseen question papers taken under examination conditions – that will be new to many of this year’s learners. Also, AS and A level learners won’t have built up the resilience that learners acquire when they do GCSEs. Now resilience is not something that the qualifications system sets out to build, but it is a by-product and one headteacher described it to me as ‘exam stamina’ building resilience and character.

It may also be the case that learners are making comparisons with past papers from before the last round of GCSE, AS and A level reforms, which were assessed for the first time between 2017 and 2019. So, for some subjects there’s only one past paper as a valid reference point.

Another driver for this concern may stem from our pre-announced position of ‘generous grading’. This aim will be achieved through the adaptations and the approach to grading, but some may have interpreted this aim as meaning easier question papers. This was never the case, but if misunderstood then there could be a mismatch with expectations.

Whatever is driving this sense of difficulty, the system is used to identifying whether a question paper is systemically more difficult than a predecessor. Exam boards design question papers to have an equal level of demand so that they are comparable over time, but this is a difficult art and sometimes a question paper will be more demanding than intended. If this is the case, then grade boundaries are used to manage the impact on grades and provide fair outcomes for learners over time – a harder paper will lead to slightly lower grade boundaries. If the demand isn’t quite right in any subject this year, then this will be reflected in the overall level of difficulty as seen in learner performance and that same mechanism will kick in.

So, what’s next?

Now that exams themselves are over, the work to get to grades is very much ‘behind the scenes’ for most. Exam boards will be busy getting scripts marked by examiners, who are all practising, recently practising or retired teachers. There are processes in place to train markers in the use of mark schemes and to quality assure their work to make marking as consistent as possible.

Once the marking is complete the next step is to set the grade boundaries. This involves looking at a basket of evidence including completed exam papers and detailed statistics on how learners and individual question papers have performed. Importantly, the process also involves the judgement of senior examiners who know the expected performance standards and context for this year’s awards. As the regulator for Wales, it is our role to monitor this process throughout.

The next visible step for learners will be results days in August and I’m sure that people will then make their own minds up on whether the system is fair or not. We must remember that fairness does not tend to be a construct that can be measured absolutely – we often think of something being fairer than something else.

Considering the options, the context and the demands placed on the system for measuring attainment, I am confident that this year we have taken the best possible path – the alternatives would have led to too many inconsistencies, too much work for teachers and lecturers, and too many issues of relational fairness. Ultimately, the system that was put in place last year held public confidence because there was no viable alternative. That position has now changed.

I hope that everyone gets the grades that they need to progress as they intend, but do not despair if you don’t.  There are lots of options and choices that you can take, so be open-minded and be prepared – there will be lots more information about preparing for results days from ourselves and other across the education system in the coming weeks.

By Philip Blaker, Chief Executive of Qualifications Wales 

This article was first published in the Times Educational Supplement/TES magazine