Our challenge: Find any evidence in the data for sponsored academy performance
The Education Bill is due to have its final vote in the Lords this week. Once this goes through, hundreds of "underperforming" primary and secondary schools will be forced to become sponsored academies. Yet at no point in the debate over the last few months has the government produced any evidence that sponsored academies improve faster than similar maintained schools.
This is our challenge. Below are links to spreadsheets based on DfE data on school performance. We challenge anybody who believes that sponsored academies are the solution for struggling schools, to find any evidence for this in recent schools data.
When the latest school-by-school GCSE results were released this month, the DfE press release claimed that sponsored academies "are transforming the fortunes of pupils". However the only evidence they could find to back this up was that "results in sponsored academies open for 2 academic years have improved by 2.3 percentage points since 2014".
The government claim was that in one subset of the group of sponsored academies (the 77 out of the 503 secondaries open exactly two years), a very modest increase was achieved. They couldn't claim that sponsored academies did better than schools overall (even though they start from a lower base) because they didn't. They carefully avoided any reference to sponsored academies open 5 years or more (over half of sponsored secondaries): although their results are on average still 10% below the overall schools GCSE benchmark, their results fell last year. (Data on length as sponsored academy v results in crease is on p12 here.)
Do you believe a school's results will improve more if it becomes a sponsored academy than if it stays in the LA maintained sector? Please find any evidence for this in the data & feel free to post your results on LSN.
Primary Schools: Maintained schools improve their results at a faster rate
Downlaod the data here: KS2 Regression 2015
The key question is whether a school will improve faster if it stays as a maintained school or if it becomes a sponsored academy. So the question is not whether sponsored academies do better than schools overall (the normal government comparison until this year) but whether they improve faster than schools starting from a similar point. So if a schools has, say, 38% of pupils achieving the GCSE benchmark would it do better as a sponsored academy? The whole basis of the Education Bill is that becoming a sponsored academy would improve its performance much faster, but the data (below) indicate the opposite.
The basic methodology, used for primaries and secondaries, is to compare schools with similar prior year results. Schools are grouped into five groups, or "quintiles", to achieve roughly the same number of sponsored academies in each group.
To carry out analysis, you need to be able to use Excel Pivot Tables, the most basic tool of the data analyst. As well as adding columns to calculate the change in results between 2013 & 2015 and between 2014 & 2015, I have added columns (FSM range and 14acquin4b, for instance) to enable grouping by level of disadvantage and by those prior year results. (To change the grouping, simply go to the Lookup tab and alter the figures.)
This shows that, when compared to similar sponsored academies, results of maintained schools improve at a faster rate:
This spreadsheet also includes regression analysis, which requires a greater degree of statistical knowledge. Indeed I asked a colleague, who is more of a specialist in this area, to help me here. The conclusion was that "taking 2013 score into account, and ignoring other factors, then Sponsored Academy status is associated with lower improvement (and this is significant at 99% level)".
See this post for analysis showing maintained schools improved more whether you look at 2013-15, 2014-15 or at Level 4, Level 4b or Level 5 results.
Secondary Schools: Maintained schools improve their results at a faster rate
Download the data here: 2015 GCSE results
Again, to enable comparison of similar schools, they have been grouped into quintiles by prior year GCSE results. Again, feel free to experiment as all that is needed is basic knowledge of Excel Pivot Tables.
As for primary schools. when similar schools are compared the improvement in non academies is consistently greater than in sponsored academies
Can you meet our challenge?
The data is here. The preliminary analysis and grouping has been done. There are, of course, individual sponsored academies that perform very well. But the issue is not the exceptional few, but the performance of sponsored academies overall.
Please let us know (in comments below or in new posts) if you find any errors in the analysis I have done, or if you can find any evidence that even suggests that - where similar schools are compared - results in sponsored academies improve at a faster rate.
Or if you find anything we haven't spotted (there is a lot of data), please do post it to LSN.
The coverage here is on sponsored academies, because that is the focus of the Education and Adoption BIll. The same problems do not seem to exist for Converter Academies, which are generally Good or Outstanding schools that have chosen to convert. These normally do become autonomous and, crucially, do not have to join an academy chain. When adjusting pivot tables,t ake care with the filters: these ensure that only mainstream schools are included and only those with results in both years.