Earlier today saw the release of the examination results that determine university entrance for students across the UK. These are A-levels in England, Wales and Northern Ireland, and Highers and Advanced Highers in Scotland. Those in Scotland and Wales were fortunate to have their results well in advance and were waiting for confirmation only. Unfortunately, the lead up to this annual event has been tainted by an unprecedented level of media attention. Sometimes this has been speculative, and in many cases has undermined the significance of the results for students themselves. The emphasis has been on the further advance of ‘grade inflation’. Yet students have worked through a difficult time with their teachers and should be treated with more respect. On the other hand, the governments in the four UK jurisdictions must accept a high level of valid criticism. This must especially fall on to the government and its agency, Ofqual, in England. The long-held belief across the world that A-levels represent the ‘gold standard’ in assessment has become tarnished in the process. The events of the last year have uncovered many uncomfortable properties of the ‘gold’ that lead many to now see it as pyrite or ‘fool’s gold’. One consequence is that universities and employers may well seek other ways to assess candidates using unregulated tests. There must be a radical reform, and fast, if anarchy is to be avoided.
UPDATE 11th August 2021. It didn’t take long for the knives to come out after only one day with the Times reporting ‘Call for urgent A-levels overhaul as grades soar’ and rumours that the position of Education Secretary, Gavin Williamson, was in doubt with a replacement hovering in the wings. With a similar theme reported on BBC News, it seems there is the smell of more radical reform in the air. “I love the smell of failed ministers in the morning………..”
Behind the headlines.
The main emphasis in media reports has been that ‘Top grades reach record high’ (BBC News). The Scottish Qualification Authority (SQA) reported a more stable outcome (note SQA will be disbanded later this year). Despite more top grades, the A-C attainment rate at Higher was 87.3%, down from 89.3% in 2020. Similarly, the A-C attainment rate at Advanced Higher was 90.2%, down from 93.1% in 2020.
Also, the results for BTECs top grades do not appear to have advanced as much as A-levels and some were even lower in many subjects. It is worth noting that a significant number of university entrants rely on BTEC qualifications to progress to Higher Education (see TEFS 5th April 2020 ‘To BTEC or not to BTEC, that is the question’).
In contrast, the proportion of A or A*s awarded at A-level rose to a staggering 44.8% overall after a record 38.5% in 2020. This was from 25.5% in 2019 that had remained a similar level for many years (source Joint Council for Qualifications covering the UK excluding Scotland – these numbers differ slightly from the Ofqual figures released today that cover England). It was Northern Ireland that breached the 50% mark with 50.8%. Wales was close behind at 48.3%.
However, this does not mean that a high number and proportion of students have achieved A or A* in all their A-levels. Mining into the Ofqual reports and data indicates that the overall percentage of students at this level across three A-level subjects was 6.9% (4.3% in 2020 and 1.6% in 2021). This comes to 12,945 achieving 3 or more A* grades, with some getting that in 4 or 5 subjects. The proportion who achieved at least one A* with other A-levels was 16.1% (13.9% in 2020 and 9.5% in 2019). However, these also represent a significant expansion in the ‘high achievers’ who expect to progress to degrees in the higher tariff, elite universities.
Disappointment looms.
It seems that teachers and schools had translated their earlier predicted grades into actual final grades. This is perhaps not surprising. But expansion of the number of top grades from a record high last year has left some disappointment in its wake. Most universities realised (some too late) that making too many offers based on predicted grades could force them into accepting high numbers this autumn. Many simply reacted by restricting more offers to students since January. But students who were rejected still went on to get equally high grades this week and must compete for places in clearing instead. This has now become a harsh reality with medical schools in particular backed by the government into offering cash incentives for students to go elsewhere or defer for a year. Add to this the numbers who deferred from last year and it gets even more frantic. The clearing process started today with UCAS trying to reassure students. But with over 150,000 students chasing declining places on offer across many subjects, this will lead to more disappointment in the coming days.
How fair was it this year?
It seems Ofqual is satisfied with an outcome that shows the system has maintained its old unfairness and gaps. In its equality analysis, Ofqual concluded in relation to equality gaps “We found that, of the many between-group comparisons examined in our modelling, the majority showed no notable change from 2020 to 2021 or from 2019 to 2021”. However, their analysis is complex with results presented in a way that will appear opaque to most observers, even those used to dealing with lots of data. It seems, despite reassurances, there has been a widening of regional and disadvantage vs advantaged gaps. There will have to be more forensic analysis of the data alongside the 2020 data finally being released.
The most striking gap appears in the differences between types of school. Datalab provide a more accessible overview of the 2021 data and clearly show how independent schools have advanced faster this summer. The gap between them and other schools is now eyewatering.
The Financial Times, along with Schools Week, were amongst the first to spot that “the jump in grades also saw fee-paying schools awarding more top marks, while adding to regional disparities that the opposition Labour party said deepened existing social divides”. It explains why so many families seek to pay for independent schools. This is one to the standout features of the data released by Ofqual. There was a staggering 9.3% rise in the number of A and A* grades since 2020 with 70.1% achieving A or over. Other types of school failed to approach this level by a big margin. The reasons behind this are unclear, but there should be a urgent investigation of the procedures in some schools.
The Sutton Trust provided their view earlier today with ‘The Sutton Trust comment on Results Day 2021’ and its chair, Peter Lampl, saying “We’re seeing growing gaps between independent and state schools at the top grades”. Their more caustic view out today was in ‘Another omnishambles? Inequality and a-level results in 2021’. In terms of access and gaps, they rightly note that “the entry gap has grown, from about 24 percentage points in 2020, to almost 28 percentage points today”. This can only get worse if changes are not made to the assessment process.
Universities and employers are voting with their feet.
The ‘omnishambles’ described by the Sutton Trust has already been noted across universities and employers alike for some time. Students in the UK must already be aware that many university courses set additional tests on a similar basis. UCAS lists a plethora of these its www site at ‘Admissions Tests’. The idea that A-Level exams alone are a good or reliable measure is frankly losing favour with the further expansion of such tests across the sector. Most of the tests include an element of ability assessment in the form of verbal and numerical reasoning questions. A good example is the BioMedical Admissions Test (BMAT) used in many countries including the UK (see specification here). It is highly likely that such tests will now spread fast across many subjects and universities. No doubt the independent schools will prepare their students for the experience with enthusiasm, if they are not doing so already.
To add to this, there has been an inexorable rise in employers administering their own tests in an effort to select the most able candidates. All include assessing elements of verbal and numerical reasoning in the process. I have some experience of how this was done years ago.
In the 1980s, I attended two days of interviews at a major multinational industrial employer. One afternoon was devoted to ‘psychometric testing’ that we were assured was ‘not an intelligence test’. To my surprise, this notion could be dismissed as early as 10 questions into a 300-question multiple choice paper to be completed in one hour. Yes, there it was, a verbal reasoning question. It turned out that the series of 300 questions was spattered with 30 to 40 verbal and numerical reasoning challenges, some very hard. I divided my time accordingly to compensate for the degree of difficulty and finished on time. Others were caught out and upset by the experience. I made my feelings on such blatant dishonesty very clear to the management the next day and withdrew my application.
Yet this approach has been rising across the employment sector with some students preparing in earnest. But at least now it is openly explained to them. The result is a horde of suppliers offering a service for employers and free or ‘for sale’ tests for students to try out. See Psychometric tests | Prospects.ac.uk for a good overview. It is becoming a major industry that can only expand as a lack of confidence in school examinations and degree grades spreads further. It surely must be obvious to everyone that universities and employers are primarily interested in the ability and potential of candidates. Attainment and evidence of studying a subject are also very important and A-levels can confirm this. But they cannot alone gauge potential or ensure success is possible.
Moving forward.
‘Moving forward’ is an annoying term used by people ‘in charge’ who have managed to create an ‘omnishambles’ and are trying to wriggle out of their responsibility. It usually heralds a determination to return to making the same old mistakes. The government and Ofqual are no different, and they are determined to go back to examinations alone to assess students. Already demonstrated as highly unreliable by trenchant critics, notably Dennis Sherwood, it is astounding that further reforms are not in the pipeline. See Sherwood’s article ‘The broken school exam system needs an upgrade’ from last year and more recently ‘Higher education to the rescue: only universities can put sufficient pressure on Ofqual to deliver reliable grades’ amongst other critical offerings on the Higher Education Policy Institute site.
Even the Ofqual interim chief regulator, Simon Lebus, is reported as saying that teacher-assessed grades (TAGs) are likely to give a “much more accurate” reflection of what students are capable of this year than exams. But instead of concentrating on making examinations more reliable and better at assessing the ability and potential of students, the same problems are promised when ‘moving forward’.
A consultation by Ofqual on ‘Proposed changes to the assessment of GCSEs, AS and A levels in 2022’ has emerged and we are awaiting a response. However, it seems there is a determination to continue to set the same exams but make them easier with “the provision of advance information on the focus of the content of exams, in the majority of subjects at GCSE, AS and A level”. The idea circulating that the grades could be recalibrated to a 1-9 scale as a way to solve the problem is laughable. This does not change the essence of the assessment, it merely ‘rearranges the deckchairs’ on the sinking ship.
The ‘Gove effect’.
The current problems stretch back many years, and the A-level system suffers from the ‘Gove effect’. This began under Michael Gove as Education Secretary from 2010 to 2014. It led to Ofqual beginning to reform the main qualifications from 2012 (GCSE, AS and A level reforms. 9th September 2012). The outcome of their consultation was released in 2013. By January 2016, a new approach was in place (House of Commons Briefing Paper ‘GCSE, AS and A level reform (England)’).
The result was a system that put undue emphasis on a final examination and led to schools working almost exclusively to the examination. This was exacerbated by examinations based mostly upon knowledge and attainment. Schools and parents with tutors, armed with more resources and better preparation were at a tremendous advantage over others with fewer resources available. The examinations have become more a test of the ability of teachers and the schools than that of the student. The success of independent schools this year is surely ample evidence of this. But the government approach has been to adopt a deliberate strategy bound to widen the gap between the advantaged and disadvantaged. It also led to the inevitable chaos when examinations had to be cancelled in 2020. ‘A radical overhaul of examinations is needed as soon as possible’ (TEFS 15th January 2021).
Scholars and scientists instinctively look at all the evidence when making an informed judgement. A scientist would never accept the results of a single experiment at one point in time as the final evidence. Replicates and different independent confirmatory experiments are devised to test the conclusions. Similarly, with ample evidence in schools available about the ability and attainment of students, it makes sense to use it all to reach a more reliable judgement. Indeed, recent events will have persuaded many that a hybrid assessment of students over time by teachers alongside reformed examinations would provide a better picture of candidates. More a 3D view than 2D sketch.
The problem has been around for many years.
The issue of how to decide who goes to university has been around for a very long time. Back in 2004, under a very different government, a review of admissions by the Higher Education Steering Group (HESG) considered various options (see ‘Fair admissions to higher education: recommendations for good practice.’). They recognised that “equal examination grades do not necessarily represent equal potential” and that “it is fair and appropriate to consider contextual factors as well as formal educational achievement, given the variation in learners’ opportunities and circumstances”. Given this observation, it is astounding that so little has been done since and the problems are being swept aside again this year.
In February 2019, the Sutton Trust reaffirmed its commitment to a contextualised admissions process in ‘Moving the Dial on Contextual Admissions’. There it stressed that there is little incentive for universities to change and that “Use of A-Level results of students before they have even entered university is a deeply flawed measure of quality.” These are strong words and you can read more of these arguments in their detailed report ‘Admissions in Context: The use of contextual information by leading universities’ that shows the extent to which thirty leading universities already use contextualise admissions data.
It is fair to assume that the current examination situation will not be tolerated by universities for much longer as they seek out better assessment methods. A-levels will become of lesser importance in time.
Alternatives.
In 2004, the HESG also looked at alternatives. This included consideration of additional tests such as the USA based SAT (originating in 1926 and first called the ‘Scholastic Aptitude Test’) that “may help to uncover hidden talent”. The system in the USA of school ‘Grade Point Average’ (GPA) attainment and results of ‘Scholastic Aptitude Tests’ or SATs meets many requirements and is likely to be fairer in the form used back then.
Indeed the Sutton Trust carried out a rigorous comparison between the SAT and A-Levels in 2001 in ‘A Pilot of Aptitude Testing for University Entrance’ with tantalising conclusions. Although there was a general correlation evident in the comparison of A-Level grades and SAT score, the extent of ‘scatter’ across those from low-achieving, high-achieving and independent schools was an alarming feature of the data. A significant number of students in low-achieving schools had high SAT scores and yet achieved very low A-Level scores. That some of these might then thrive at university, when given some fair resources and a chance, should surely come as no surprise.
But the SAT itself has been changed many times over the years and was reformed again in 2016. Currently, the final score is determined by a combination of reading, writing and a maths test with an optional essay. It was originally designed to eliminate rote learning as much as possible. However, the emphasis has changed. A very sound use of English is necessary and the new format from 2016 is less about ability and more aligned with the school curriculum than the former test. This means that it becomes inherently socially biased towards those with a sophisticated and nuanced understanding of English that they could acquire through tutoring or a better learning environment at home.
The reforms have inevitably led to USA universities abandoning the SAT and looking for alternatives better suited to their desire to measure ability and potential. Last year the university of California decided to drop SAT (and its close cousin, ACT or American College Testing) from their entrance criteria (see ‘University of California to drop SAT and ACT as admission requirements’ The Guardian 22nd May 2020). The conclusion was that they were seeking to avoid discriminating against disadvantaged students. Those in the UK would have to consider this more carefully when devising reformed examinations.
Tarnished gold.
The result of this year’s examinations is bound to have an impact on the so called ‘gold standard’ of A-levels. Going back to the old ways in 2022 is not a solution. The system has been exposed as unreliable and inherently biased. The results reflect wide gaps in provision across different schools and not the potential of students caught in the middle. Failure to accept the need for more radical reform will amount to a lost opportunity. Without change, the ultimate outcome will be a rise in unregulated alternative assessments across a range of private enterprise test providers. That is the road to anarchy where the well-resourced and more advantaged will continue to prevail over the rest.