20% of free schools which entered pupils for GCSE in 2017 were below the Department for Education (DfE) floor standard of -0.5 for Progress 8 (P8) scores according to data published in Schools Week*.
The proportion failing to reach the floor standard rises to 48% if University Technical Colleges (UTCs) and Studio Schools (both technically free schools) are included.
It appears flagship schools, free schools, UTCs/Studio Schools, are more likely to fall below the P8 floor standard than any other types of school.
Is this a fair analysis? The answer is No. DfE number-crunchers make it quite clear:
‘The numbers of free schools, UTCs and studio schools with year 11 pupils are too small to allow robust conclusions to be drawn about their performance at the end of key stage 4, or compare between years.’
Why, then, have I used the data to show poor performance at P8 in free schools, UTCs and Studio Schools?
The answer is to draw attention to how stats can be used to mislead readers by ignoring both context and warnings.
But that’s what the New Schools Network (NSN), the taxpayer-funded charity which promotes free schools, has done. It’s issued a press release headed ‘Free Schools Top League Table at Key Stage 4’ which says free schools had an average P8 score of 0.10. This puts them at top position with converter academies.
The figure is correct but the DfE warning has been ignored. And NSN hasn’t included UTCs/Studio Schools despite both being free school types. Including UTCs/Studio Schools in data used by NSN would have reduced the average P8 score substantially.
It’s not materially misleading for NSN to say free schools top the P8 league tables based on the average score. Equally, it’s not materially misleading to say free schools are more likely to have a P8 score which doesn’t meet the P8 floor standard.
But it’s still misleading because the number of free schools is too small to reach a definitive conclusion about the performance of free schools compared with other types of schools.
UTCs and Studio Schools face a double whammy when P8 is calculated. First, pupils don’t enter the schools until the start of Key Stage 4 but Progress 8 measures a pupil’s academic progress during Key Stage 3 as well. UTCs and Studio Schools have no control over progress, or the lack of it, made by their pupils in Key Stage 3. At the same time, UTCs and Studio Schools have a curriculum which doesn’t easily match the subjects used to assess P8.
The headline above, then, is misleading. Whatever your opinion of free schools (mine - heavily hyped but little different to other schools), it is unreliable to use data to trumpet their alleged superiority. The NSN and the DfE should take note.
*October 13 2017, not yet available on line.