Why marketisation invalidates Progress 8
My book, ‘Learning Matters’, was published before ‘Attainment 8’ and ‘Progress 8’ were finalised and published. This article is therefore something of a postscript.
I have downloaded and studied the DfE publication, Progress 8 and Attainment 8 measure in 2016, 2017, and 2018, Guide for maintained secondary schools, academies and free schools, October 2016
You can find it here
It takes some digesting, but its purpose at least is clear. All quotations are in italics.
Progress 8 aims to capture the progress a pupil makes from the end of primary school to the end of secondary school. It is a type of value added measure, which means that pupils’ results are compared to the actual achievements of other pupils with the same prior attainment.
The new performance measures are designed to encourage schools to offer a broad and balanced curriculum with a focus on an academic core at key stage 4, and reward schools for the teaching of all their pupils, measuring performance across 8 qualifications. Every increase in every grade a pupil achieves will attract additional points in the performance tables.
Progress 8 will be calculated for individual pupils solely in order to calculate a school’s Progress 8 score, and there will be no need for schools to share individual Progress 8 scores with their pupils.
It sets out to use the market mechanism to compel all secondary schools to provide a broad and balanced academic education for all students, defined mainly by the EBacc subjects. Unlike many on the liberal left, I have no problem with this direction of reform of the curriculum, but my argument in support of it is completely different to that of the government.
What can be wrong with having well educated plumbers, actors, motor mechanics, shop assistants, footballers, tennis players, care workers etc. as well as more broadly educated teachers, doctors, lawyers and engineers?
Of even more importance is the richness of such a broad and balanced curriculum as a vehicle for engaging, inspiring and developing the intellect of all students at all stages of cognitive development through well established developmental teaching and learning methods that are proven to be effective. This principle is described and explained in this article.
So what is wrong with the Progress 8 approach? Any teacher that tries to read the DfE paper will start to worry. The sheer detail and complexity of it is boggling. Does teaching school students things they need to know, in a way that they can understand, and which develops their general intelligence in the process, really have to be this instrumental and suffocatingly dull, depressing and off-putting to teachers?
My answer is, maybe if it works.
In order to fully understand why it will not work, it is necessary to study some research recently carried out by the BBC ‘Newsnight’ Team. To understand my argument it is necessary to carefully read the whole of this article, from which the following is extracted.
Four researchers had access to 160 secondary school academies in England, building a dataset that covered the tenure of 411 head teachers. The research, by Alex Hill, Liz Mellon, Ben Laker and Jules Goddard, is being published in the Harvard Business Review. Their work found that heads tended to fall into one of five “types”
“Surgeons” are head teachers who act decisively to try to turn around schools. On arriving in a school, they exclude an average of around a quarter of the final-year students and drive resources into final-year students. They fire around a tenth of staff. They have a dramatic immediate impact.
The clear finding here is that surgeons appear to make the most dramatic improvement in the short term, averaging around a 10% improvement per annum in exam results with only a slight loss of financial strength.
That remarkable performance in improving GCSE results explains why ‘surgeons’ are the most sought-after heads. The paper reveals that they are paid, on average, £154,000 a year. ‘Philosophers’ get £103,000 and ‘architects’ just £86,000. ‘Soldiers’ and ‘accountants’ both get about £100,000 a year. ‘Surgeons’ are also most likely to get awards. Almost two-thirds have a national honour, like a knighthood.
There are, however, profound problems for the system with the strategy pursued by the ‘surgeons’. Their strategy is about rapid turnaround, so they invest aggressively in children about to take exams, and exclude tough children up-front. ‘Surgeons’ dropped into a school would be expected to get rid of an average of 28 per cent of final year students to dramatically boost that year’s exam results. That generates rapid improvement in results. But it is not sustainable. Many of these heads leave within two years – and the ‘surgeons’ schools’ results decline rapidly in the year after the head moves on.
Their strategy is time-limited – you can only improve results modestly through removing pupils. And if a school has moved resources from younger children to focus on those taking exams imminently, eventually it will have to deal with children whose education was damaged by the strategy at the start of secondary school, when resources were diverted from them into older children.
The four other ‘categories’ of head teacher produce less spectacular GCSE results, but better long term performance for their schools and by implication, for the quality of the education provided to their students.
So what has this to do with ‘Progress 8’?
Progress 8, like everything else about the English marketised education system, is focused onto producing a concise school performance indicator that drives school parental choice in a system of competitive school league tables and the OfSTED school inspection and reporting system, which bases its school gradings on the Progress 8 outcomes rather than on qualitative judgements of teaching and learning observed by skilled, experienced and independent inspectors. OfSTED has evolved into a system of short inspections by a small number of inspectors armed with performance data that has already been processed to determine the outcome of the inspection for the school. The classroom visits are largely to provide context and examples for the final report.
The Progress 8 system is just the same in this respect except that the freedom of judgement of the inspectors is even more constrained as demonstrated by the threatening sections.
The floor standard for a school is the minimum standard for pupil achievement and/or progress that the Government expects schools to meet. Floor standards do not apply to special schools, independent schools, pupil referral units, alternative provision or hospital schools.
In 2016 (or 2015 for those schools that chose to opt in a year early), a school will be below the floor standard if its Progress 8 score is below -0.5, and the upper band of the 95% confidence interval is below zero. If a school’s performance falls below this floor standard, then the school may come under scrutiny through inspection. Confidence intervals are explained in more detail on page 21-22 and in Annex D.
Schools in which pupils make on average one grade more progress than the national average (a Progress 8 score of +1.0 or above) will be exempt from routine inspections by Ofsted in the calendar year following the publication of the final performance tables.
Coasting schools definition
The Education & Adoption Act 2016 introduced new provisions to define schools that are ‘coasting’. In March 2016 the government published its response to the consultation on coasting schools. This confirmed that a ‘coasting’ school was one where data showed that over a three-year period, the school had failed to ensure that pupils reached their potential.
The document also set out the Department’s proposed definition of a coasting school. This is based on the same performance measures that underpin the floor standards. In 2016, a secondary school will be coasting if:
- In 2014 fewer than 60% of pupils achieved 5 A*-C at GCSE (including English and maths) and less than the national median achieved expected progress in English andin maths and;
- In 2015, fewer than 60% of pupils achieved 5 A*-C at GCSE (including English and maths) and less than the national median achieved expected progress in English andin maths; and
- In 2016, the school’s Progress 8 score is below -0.251
A school will have to be below the coasting definition in three consecutive years to be defined as coasting.
327 schools opted in to Progress 8 in 2015. For these schools, if they meet the definition above, but in 2015 have a Progress 8 score of -0.25 or above they will not coasting.
Schools will be excluded from the coasting measure in 2016 if:
- they have fewer than 6 pupils at the end of key stage 4; or
- less than 50% of pupils have key stage 2 assessments that can be used as prior attainment in the calculations of Progress 8; or
- the school closes within the academic year (except if they reopen as a converter academy).
Schools will be excluded from the coasting measure in 2014 and 2015 if:
- they have fewer than 11 pupils at the end of key stage 4; or
- less than 50% of pupils have key stage 2 assessments that can be used as prior attainment in the calculations of expected progress; or
- the school closes within the academic year (except if they reopen as a converter academy).
Any school that is excluded from the coasting measure in a particular year cannot be defined as coasting until it has three consecutive years of data that meets the coasting definition. No school will be identified as coasting until after the revised 2016 secondary performance tables are published in January.
Subject to Parliament agreeing to the Regulations, the coasting definition will apply to all mainstream maintained schools and academies with the relevant key stage 4 data. It will not apply to PRUs, special schools, alternative provision academies or maintained nursery schools.
I had to pinch myself to realise the extent to which these threats will now dominate the school experiences of heads, teachers and their students.
It does not seem to have occurred to the DfE statisticians that this monumental package has at its heart a fatal flaw – a classic case of not seeing the wood for the trees.
The judgements are based on the results of high pressure SATs exams taken over a short period in Y6 to establish the base line ‘prior attainment’, followed by the results of high pressure GCSE exams taken over a short period in Y11.
The assumption is that all the thousands of hours of teaching and learning that took place in between can be validly and reliably reduced to a single Progress 8 measure for every student (kept secret from them), which are then aggregated to form a judgement on whether the school should fail and close or succeed in the market: on whether the head should get the sack or a huge performance related pay rise and a knighthood.
So what about the BBC Newsnight research?
If all schools are incentivised to appoint ‘surgeon’ type heads (in primary as well as secondary schools) that rapidly move onwards and upwards after producing dramatic exam results for their schools, what if Secondary school A is fed in Y7 by a Primary School who took their SATS under a ‘non-surgeon’ head, but these same students five years later take their GCSEs in Secondary school B ruled by a ‘surgeon’ head in the full flush of his/her epic GCSE results generating reforms – or vice-versa?
At the beginning of this article I implied that my book, ‘Learning Matters‘ has nothing to say about ‘Progress 8’, but it has. Part 3 is entitled, ‘Spectacular School Improvement’. This is mainly about the disastrous consequences of Blair’s spectacular school improvement that was driven by the ‘Vocational Equivalent Scam’, designed to justify the education policies of the time. But Progress 8 will have perverse consequences, as market based incentives always do, because although he ended the curriculum and exams scam of the Blair era, Michael Gove failed to recognise that the marketisation paradigm is still corrupting, stultifying and degrading the school experiences of our students and their teachers. Progress 8 will fail to capture the essential qualities at the core of high quality, developmental education.