No evidence for academy success – Mumsnet-inspired check reveals empty rhetoric
When Education Secretary Nicky Morgan wrote for Mumsnet justifying the mass academization of England’s schools, she probably didn’t think it would trigger a Mumsnet firestorm.
One thousand, mostly-negative comments were posted. The first was a pithy question:
‘Would you ask Radio4 More or Less to check all your statistics?’
More or Less took up the challenge.
It’s important to know the difference between sponsored academies, mostly under-performing schools, and converter academies, mostly high-performing schools judged good or better before conversion, Stephen Machin, UCL economics professor, told the programme. This made it difficult to compare the performance of academies as a group with non-academies.
Professor Machin had surveyed early sponsored academies set up under Labour which found there had been improvement. At the same time intake quality had risen after the schools became academies.
More or Less checked Morgan’s claims:
‘The most recent results show that the percentage of pupils achieving the expected level in reading, writing and maths at the end of Key Stage 2 in primary academies has risen by 4%, from 67% in 2014 to 71% in 2015.’
DEBUNKED: Morgan was only referring to sponsored academies not all primary academies. Sponsored academies comprise just a third of primary academies. Primary converter academies and non-academies had improved at the same rate. (It should be remembered that the improvement rate of sponsored academies is calculated from a lower base because they were underperforming schools – this would make the improvement rate larger than rates calculated from a higher base.)
‘Additionally, when it comes to secondary, it's a similar story with converter academies which are performing 7.2% above the national average, with 64.3% of pupils achieving five good GCSEs.’
DEBUNKED: Morgan switched from sponsored academies to converter academies. She also switched from rate of improvement to overall achievement. Morgan’s claim was true but ‘was not very informative’. Converter academies were high achievers before they converted.
The programme’s conclusion based on these two claims: ‘We have learnt nothing about whether the switch to academies made them better or worse.’
Using statistics in this way was deceptive, More or Less said. Stephen Machin explained: as noted above the intake of Labour’s secondary academies improved after they were sponsored. It was misleading, therefore, to compare the improvement of academies over time without taking this rise in intake quality into account. Such a comparison was not comparing like with like.
What evidence was there, then, to support the claim that schools improved after becoming academies? None, replied Professor Machin, especially at primary level. There had been his report into Labour’s sponsored secondary academies but it was not possible to apply the findings to mass academization. The type of school becoming academies post May 2010 had altered significantly.
No evidence, then, for the superiority of academies. But we’ve been fed ‘evidence’ about academy success since they first began. This misinformation exploded when the Academies Act was passed in summer 2010. Education secretaries, ministers, over-praised ‘superheads’, much of the media and various Think Tanks have spun academy success using the kind of empty statistics in Morgan's Mumsnet article.
This empty rhetoric, regularly debunked on this site, has now been widely exposed.
You can listen to the More or Less feature on academies here at 09.52.
Want to find out if your grasp of statistics is as tenuous as minister’s? Try the LSN statistics quiz here.