Our challenge: Find any evidence in the data for sponsored academy performance

Henry Stewart's picture
 2

The Education Bill is due to have its final vote in the Lords this week. Once this goes through, hundreds of "underperforming" primary and secondary schools will be forced to become sponsored academies. Yet at no point in the debate over the last few months has the government produced any evidence that sponsored academies improve faster than similar maintained schools.

This is our challenge. Below are links to spreadsheets based on DfE data on school performance. We challenge anybody who believes that sponsored academies are the solution for struggling schools, to find any evidence for this in recent schools data.

When the latest school-by-school GCSE results were released this month, the DfE press release claimed that sponsored academies "are transforming the fortunes of pupils". However the only evidence they could find to back this up was that "results in sponsored academies open for 2 academic years have improved by 2.3 percentage points since 2014".

The government claim was that in one subset of the group of sponsored academies (the 77 out of the 503 secondaries open exactly two years), a very modest increase was achieved. They couldn't claim that sponsored academies did better than schools overall (even though they start from a lower base) because they didn't. They carefully avoided any reference to sponsored academies open 5 years or more (over half of sponsored secondaries): although their results are on average still 10% below the overall schools GCSE benchmark, their results fell last year. (Data on length as sponsored academy v results in crease is on p12 here.)

Do you believe a school's results will improve more if it becomes a sponsored academy than if it stays in the LA maintained sector? Please find any evidence for this in the data & feel free to post your results on LSN.  

And also do check the results on which my recent posts (KS2 and GCSEs) have been based. All the analysis for those posts is in these spreadsheets.

Primary Schools: Maintained schools improve their results at a faster rate

Downlaod the data here: KS2 Regression 2015

The key question is whether a school will improve faster if it stays as a maintained school or if it becomes a sponsored academy. So the question is not whether sponsored academies do better than schools overall (the normal government comparison until this year) but whether they improve faster than schools starting from a similar point. So if a schools has, say, 38% of pupils achieving the GCSE benchmark would it do better as a sponsored academy? The whole basis of the Education Bill is that becoming a sponsored academy would improve its performance much faster, but the data (below) indicate the opposite.

Methodology

The basic methodology, used for primaries and secondaries, is to compare schools with similar prior year results. Schools are grouped into five groups, or "quintiles", to achieve roughly the same number of sponsored academies in each group.

To carry out analysis, you need to be able to use Excel Pivot Tables, the most basic tool of the data analyst. As well as adding columns to calculate the change in results between 2013 & 2015 and between 2014 & 2015, I have added columns (FSM range and 14acquin4b, for instance) to enable grouping by level of disadvantage and by those prior year results. (To change the grouping, simply go to the Lookup tab and alter the figures.)

This shows that, when compared to similar sponsored academies, results of maintained schools improve at a faster rate:  

This spreadsheet also includes regression analysis, which requires a greater degree of statistical knowledge. Indeed I asked a colleague, who is more of a specialist in this area, to help me here. The conclusion was that "taking 2013 score into account, and ignoring other factors, then Sponsored Academy status is associated with lower improvement (and this is significant at 99% level)".

See this post for analysis showing maintained schools improved more whether you look at 2013-15, 2014-15 or at Level 4, Level 4b or Level 5 results.

Secondary Schools: Maintained schools improve their results at a faster rate

 Download the data here: 2015 GCSE results

Again, to enable comparison of similar schools, they have been grouped into quintiles by prior year GCSE results. Again, feel free to experiment as all that is needed is basic knowledge of Excel Pivot Tables.

As for primary schools. when similar schools are compared the improvement in non academies is consistently greater than in sponsored academies

 

Can you meet our challenge?

The data is here. The preliminary analysis and grouping has been done. There are, of course, individual sponsored academies that perform very well. But the issue is not the exceptional few, but the performance of sponsored academies overall.

Please let us know (in comments below or in new posts) if you find any errors in the analysis I have done, or if you can find any evidence that even suggests that - where similar schools are compared - results in sponsored academies improve at a faster rate.

Or if you find anything we haven't spotted (there is a lot of data), please do post it to LSN.

 

Note

The coverage here is on sponsored academies, because that is the focus of the Education and Adoption BIll. The same problems do not seem to exist for Converter Academies, which are generally Good or Outstanding schools that have chosen to convert. These normally do become autonomous and, crucially, do not have to join an academy chain. When adjusting pivot tables,t ake care with the filters: these ensure that only mainstream schools are included and only those with results in both years.

 

 

Share on Twitter

Comments

Janet Downs's picture
Tue, 02/02/2016 - 08:37

Have you sent this challenge directly to the DfE, Henry?  If so, I wonder what the DfE robot (sorry, spokesperson) will churn out.  Will it be the one about 'strong' sponsors turning round 'failing' schools?  If so, it ignores the number of 'weak' sponsors - those where focussed inspections have resulted in Ofsted criticisms.  It also ignores that fact that 'strong' sponsors such as Harris took over schools such as Downhills which were already improving.  

Or will it be the one about how sponsored academies improve at a faster rate than all other schools?  This, of course, ignores the fact that improvement rates in these sponsored academies are calculated from a lower base (as we've pointed out ad nauseam).  

Or perhaps the robot will remain silent.


Shaun Whitfield's picture
Wed, 03/02/2016 - 14:18

These facts should also be sent to Lucy Powell. I despair of the number of education open goals Labour misses.


Add new comment

Already a member? Click here to log in before you comment. Or register with us.