‘Academy chains are disproportionately found amongst the top 20 of secondary school groups’, says the Education Policy Institute (EPI) in its report comparing the performance of local authorities (LAs) and multi academy trusts (MATs).
But this is potentially misleading.
EPI researchers used a ‘contextual measure of performance’ adjusted for ‘different progress and outcomes of pupils with different characteristics’. There’s nothing wrong with that. The OECD qualify PISA results by adjusting for socio-economic factors in their commentaries (although it’s still the raw results making headlines and not the finding that when the socio-economic background of students and schools is accounted for, state schools outperform private ones).
What makes the EPI conclusions potentially misleading is that the majority of the top 20 secondary performers, whether LAs or MATs, have only a tiny number of schools. Fourteen of the ‘top 20’ had five or fewer. And just three: Outwood Grange, Harris and ULT had fifteen or more.
It’s unreliable to judge performance on a small number of schools. With just four secondary academies, Rodillian Academy Trust, a MAT slapped with a Financial Notice to Improve in February, is named top performer. Brent LA is ranked third. It has two maintained secondary schools. Gorse Academies Trust, ranked fourth, has just three.
With such small numbers, any fluctuation in results has a disproportionate effect on the performance of the whole.
This same applies at the bottom end where low rankings can do great harm to reputations. Yet the average number of schools in the twenty ‘lowest performing’ MATs and LAs at secondary level was 3.75. Two local authorities had just one secondary school each: Southend-on-Sea and Nottingham.
These two were damned because of the performance of just one school.
The situation is reversed at Key Stage 2 (KS2). LAs are ‘disproportionately represented’ among the top twenty. This is to be expected because the majority of primary schools are not academies but still under LA stewardship. The number of schools is also larger – fifteen of the top twenty had 25 schools or more. Tower Hamlets, ranked 17, and Lewisham, ranked 20, had the highest number of schools: 61.
But three of the top 20 primary performers, The Spencer Academies Trust, Diocese of Westminster Academy Trust and Transform Trust had seven, five and six academies respectively.
Ten of the twenty lowest performing MATs and LAs at primary level had fewer than nine primary schools. The Education Fellowship Trust (now defunct) had eight. Poole LA had seven. Three MATs and one LA (Rutland) had just six schools. And one of the LA schools in Rutland, Exton and Greetham CofE Primary School, was so small that its KS2 results weren’t published. Four MATs had just five schools.
Conclusions about group performance based on results in a tiny number of schools are misleading. They may give a boost to MATs and LAs at the top but are potentially damaging to those at the bottom.