Free schools had ‘particular success’ in key stage 1 Assessments, says schools minister Nick Gibb in the Department for Education media blog. They outperformed ‘other type of schools for the fifth year in a row’.
He doesn’t give actual figures but the statistics appear to back him up. 87% of free school pupils reached the expected standard in phonics in year one compared with 83% in converter academies and 82% in local authority maintained schools. At the end of key stage one, 79% reached the expected standard in reading, writing and maths compared with 77% in converter academies and 76% in LA maintained schools.
Although the difference is small, it’s still an achievement, you might say. That’s until we consider sample size. There were just 188 free schools where pupils took the phonics test and 180 which assessed pupils at the end of KS1.
A sample of 180+ schools is too small to come to any reliable conclusion about their performance as a group. This is a tiny number is compared with 10,800+ LA schools. Such comparisons, then, should be used with caution.
Worse than ignoring the small sample size, Gibb appears to take no notice of advice on comparability over time for key stage 1 data on the statistical release. It makes it clear that this year’s results are ‘not directly comparable’ to those of previous years. This is because changes had been made both to frameworks and expected standards in the last three years.
‘It would therefore be incorrect and misleading to make direct comparisons showing changes over time,’ the DfE statisticians warn.
Yet Gibb’s boast implies free schools outperformed other schools in both phonics and the end of KS1 assessments for the last five years. This implication is wrong. This year’s KS1 assessment results can’t be compared with earlier years.
What, then, is the data showing free schools outperformed other schools for five years consecutively? It can’t be KS1 figures for reasons given above. It must year 1 phonics. But five years ago, the number of free schools entering pupils for this test would have been even tinier than today.
Gibb has, of course, given the more favourable figures: phonics in year 1 not year 2. By the end of year 2*, the proportion reaching the expected standard in free schools is only slightly higher (94%) than in converter academies (93%) and LA schools (92%). Hardly any difference at all.
The schools minister has implied a comparison which is ‘incorrect and misleading’. He has ignored the importance of sample size. He has trumpeted free school phonics scores when the difference between them and other schools is negligible one year later.
Spinning data in this way makes ministers look either disingenuous or incompetent.
*Phonics table downloadable here
AMENDMENT: 28 September, 08.34: The words 'for the last five years' have been added to the sentence which previously said, 'Gibb’s boast implies free schools outperformed other schools in both phonics and the end of KS1 assessments' to make it clear I was not referring to Gibb's comparison of results for 2019 only but to his implication that free schools had outperformed other schools over five years.