DfE Confirms LSN Analysis: Academies Perform No Better Than Similar Schools

Henry Stewart's picture
 14
Earlier this year we published a series of posts, summarised here, analysing the DfE data release and showing that academies performed no better than similar secondary schools that had not become academies. (And, on some measures, performed worse.) While the DfE claimed faster growth rates for academy GCSE results we found this was due to starting from a lower base. If you compared academies to similar state schools we found the difference disappeared.

For instance while academies starting with less than 35% achieving 5 GCSEs (including English and Maths) grew their results by 8% in 2011, so did non-academies starting below 35%. When the DfE argued that most academies had not existed long enough for a fair comparison, we focused on long-established academies. Comparing with similar schools, by levels of deprivation, we again found no clear advantage to academies.

Today the DfE published its own analysis. It followed a similar methodology and established a group of 'similar schools' (based on levels of prior attainment, deprivation and previous outcomes). It came to very similar conclusions. To quote the key headings from the comparison (p 9-10):


  • "Having started from a slightly lower base, results for pupils in Sponsored Academies were broadly the same as in a group of similar schools in 2011."

  • "If equivalent qualifications are excluded, results in Sponsored Academies were lower than in a group of similar schools."

  • "Results for FSM and non-FSM pupils were broadly the same in Sponsored Academies and the group of similar schools."

  • "Results for pupils with and without SEN were broadly the same in Sponsored Academies and the group of similar schools."

  • "In both Sponsored Academies and the group of similar schools, White pupils were the lowest performing ethnic group."

  • "Results for pupils whose first language was English or other than English were broadly the same in Sponsored Academies and the group of similar schools."

  • "In both Sponsored Academies and the group of similar schools, pupils with first language other than English outperformed pupils whose first language was English."




This graph, taken form p11, shows the comparison. Local authorities in recent years have, faced with underperforming schools, had a choice. They could convert them to academies or seek improvements as state schools. We found that, despite the extra funding available for those choosing the academy route, they did no better than similar state schools.

The DfE appears to have now confirmed that academies do no better than non-academies. And note the point that academies did slightly worse when GCSE equivalents were removed. As the Daily Telegraph pointed out some months ago, if you remove equivalents, overall % GCSEs drop by 6% in all schools but those for academies drop by 12%. The DfE report confirms this fact, with a slightly larger gap: 5.8% in all state-funded schools (including academies) against 12.5% in academies alone (fig 3.2 in the report).

This difference is significant when looking at Section 2, which seeks to argue that academy results are growing faster. Michael Gove quoted from this section in his Spectator speech today, claiming far higher growth in results for academies. This speech was curious. Gove berates those schools that use Btecs, diplomas and other GCSE equivalents: "The students were told these qualifications would equal up to 4 GCSEs - but employers regarded them as worth much less than a single GCSE." He then goes on to quote results of academies that are heavily based on using the very qualifications he so disdains. (eg, He claims that in Harris South Norwood Academy , 100% of students achieved 5 GCSEs. However, once equivalents are removed, just 46% of students achieved 5 GCSEs including English and Maths.)

Indeed, while Section 1 did quote the figures without GCSE equivalents (and showed academies then performing less well than similar schools), Section 2 of the report at no point gives any figures without equivalents. Given that Gove sees these equivalents as of no value and most will from 2015 no longer count as GCSE equivalents, it is curious that the DfE bases its claim for greater growth purely on figures that include those equivalents. Given their far greater use by academies, it is likely that most or all of any growth gap would disappear once equivalents are removed.

And Section 2 is the one place where the DfE disagrees with our analysis. They claim academies GCSE results grew by 5.7%, while similar schools grew by only 3.4%. Again this difference could be explained by use of GCSE equivalents. However I would also question the figures. We found that taking the group of non-academies with results below 35% in 2010 showed an increase of no less than 8% in 2011. I would ask the DfE to release the data it is using, so the detail can be checked.

So the DfE acknowledges that academies are generally doing no better than similar non-academies, but claims their results are growing faster. However, being heavily based on use of GCSE equivalents, these claims are highly suspect. The data continues to fail to show any ringing endorsement for academies, and leaves the question of why they haven't performed better given the vast amounts of funding poured into those schools.
Share on Twitter

Comments

Janet Downs's picture
Wed, 27/06/2012 - 06:32

Thanks, Henry, for bringing this to our attention. It will be interesting to see how many media commentators highlight the DfE's own findings or quietly ignore them. It hasn't even made it to the DfE website's News section although the Spectator speech with its punchy soundbites, overblown rhetoric and misleading data is featured.

And talking of misleading data, Gove said in his Spectator's speech that WLFS was one of the "most-oversubscribed in the country". Not quite, as the LSN analysis below shows:

http://www.localschoolsnetwork.org.uk/2012/04/how-reliable-are-the-schoo...

The above thread ended with a warning that when anyone boasts about over-subscription numbers it's as well to treat them with healthy scepticism.

Allan Beavis's picture
Wed, 27/06/2012 - 10:29

Yes Janet.

And oversubscription of a particular school - whether maintained or academy - is no proof that the system overall has improved.

Harris Schools are frequently pulled out of thin air by supporters in an attempt to spin the untruth about academy success in much the same way as Gove and Sam Freedman point to New Orleans as an education "miracle." They ignore the fact that most older schools were swept away by Katrina, that philanthropists poured money into new charters to flex their corporate social responsibility and that the rest of Louisiana's schools still do badly. The Academy/Charter effect is an illusion and delusion

Janet Downs's picture
Wed, 27/06/2012 - 13:19

Allan - I started to post a reply about New Orleans but it became too long so I've posted a new thread.

Allan Beavis's picture
Wed, 27/06/2012 - 09:03

Thanks for highlighting this, Henry.

Since the DfE admit that Acadmies perform no better than maintained schools, why the forced conversions?? And the financial "incentives" to become Academies, cash diverted away from other schools in desperate need for resources and renovations?? Michael Gove hasn't delivered on his mission and it is now high time that full details of Academy funding are published.

Seeing as the data doesn't show a ringing endorsement for Academies, then it is surely impossible to accord Gove himself with much credit for improving schools? He can take credit for his ability to grab headlines on the back of speeches and announcements but beneath the orations is the sense that a lot of hot air has been blown to over-promote a project that has been vastly expensive and which might have been steamrollered forwards at breakneck speed not to improve children's education and life chances but Gove's own political ambitions.

Janet Downs's picture
Wed, 27/06/2012 - 10:41

It's also important to remember, Allan, the point Henry clearly makes. That is, when equivalents are removed the results are not as good as may appear at first glance. Take Harris Academy, South Norwood, as an example. The average entry per pupil in 2011 was 13.9 qualifications, but 7.9 were non-GCSEs. And although I think it's unfair to judge schools on EBacc because the 2011 cohort would already have started their courses before Gove announced its retrospective introduction, only 2% of pupils at South Norwood achieved EBacc. Some people (but not I, because I think it's unfair) would regard that as an alarming state of affairs.

http://www.education.gov.uk/cgi-bin/schools/performance/school.pl?urn=13...

Ricky-Tarr's picture
Wed, 27/06/2012 - 10:57

It is alarming Janet.

As a great man said recently:

We suffer from assumptions formed in the past that only a minority were ever able to attain academic excellence.

We used to have a system which educated the top 25%, the A stream, those favoured by wealth or exceptional talent, exceptionally well. But which failed to stretch, develop and challenge the majority.

I don't believe an educational system which fails to give every child the chance to excel is morally defensible...... it cannot be right to deny any child access to excellence, to the best that has been thought and written.
That is every child's inheritance, which none should be denied.


Any part of that you disagree with, by the way?

Janet Downs's picture
Wed, 27/06/2012 - 11:09

Allan - another important point to make is that some sponsored academies, despite supposing to be all-ability, have an intake heavily skewed towards the top end. This is particularly the case with academies that were once City Technology Colleges (CTCs). The 2011 cohort at the Brooke Weston College, Northampton, for example, had 58% high attainers and only 2% low. Haberdashers' Aske's, Hatcham, has 46% high attainers and only 6% low. Harris City Academy Crystal Palace, one of the much-lauded Harris chain, had 56% high attainers and only 4% low*.

The predecessor school of Harris City Academy Crystal Palace was Harris CTC which was formed in 1990/91 from the undersubscribed Sylvan High School. As it's over 20 years since Sylvan High School closed, any comparison between the performance of Harris City Academy Crystal Palace in 2011 and its LA maintained predecessor in 1990 is meaningless. I doubt, very much, for example, that the last intake at Sylvan High School matched the intake in terms of ability at Harris City Academy Crystal Palace in 2011.

It's laughable that Gove should have compared this academy with its LA predecessor. Did he realise that the latter was closed ten years before the end of the last century? Or did he hope that no-one would notice?

I think this is another example of Mr Gove's misleading statistics to put in the ever-growing file, that's if I can stop laughing.

*data from School Performance Tables on DfE website

Fiona Millar's picture
Wed, 27/06/2012 - 15:39

Any academies that were previously CTCs, like the Harris school, will have benefitted from the very lax admissions arrangements permitted for this schools at the time. Changing the intake is the easiest way to bring about results.
Harris Schools, which may well be good in other ways, have quite a clear track record of using GCSE equivalents to boost their legal table positions.

Ricky-Tarr's picture
Wed, 27/06/2012 - 09:36

Setting aside all the statistical jiggery-pokery, there are some facts that hit home with all the force of a freight train.

I pricked up my ears at this part of Gove's speech because it relates to schools fairly near my home:


Consider the case of Crystal Palace City Academy, part of the Harris Federation. In its final year as a LEA school, only 9% of pupils achieved 5 GCSEs at A*-C. Last year, 100% did (95% with English and Maths). Or another Harris school – South Norwood – where 29% of pupils reached that measure in its last year as an LEA school; 100% last year.

Janet Downs's picture
Wed, 27/06/2012 - 10:24

Henry, the DfE report said that results in sponsored academies improved at a faster rate than in a group of similar, non-academy schools although results in both types of school were broadly the same. Do you know if the rate of improvement was calculated before or after the equivalent qualifications were removed given that sponsored academies did worse than similar non-academies after equivalents were stripped out? If the calculation were made before the equivalents were removed then this would surely make the improvement rate for sponsored academies appear greater than it was.

In any case, both types of schools (sponsored academies and similar non-academies) improved. If they hadn't then sponsored academies would have out-performed similar non-academies by a wide margin. This would suggest that improving academies and similar non-academies are using similar methods - PriceWaterhouseCooper found this to be the case in 2008 (see FAQs above) and Ofsted's Annual Report (2010/11) found that good or outstanding academies shared certain characteristics with similar LA maintained schools. Those characteristics, of course, did not include academy status.

Claims that academy conversion is essential to increase standards are looking increasingly hollow.

http://www.localschoolsnetwork.org.uk/2012/05/good-or-outstanding-school...

Ricky-Tarr's picture
Wed, 27/06/2012 - 10:39

Claims that academy conversion is essential to increase standards are looking increasingly hollow.

Do you seriously believe that LA schools would have bothered to make efforts to improve had there not been academies around to shame them into it?

If you do believe LA schools would have improved anyway, could you let me know why they didn't get round to it during 13 years of rising revenue?

I repeat that bit about Harris CP:

In its final year as a LEA school, only 9% of pupils achieved 5 GCSEs at A*-C.


9% - and that's where it would have stayed if not for academies.

Allan Beavis's picture
Wed, 27/06/2012 - 13:09

The argument put forward by the school reform brigade in America - that the existence of charters drives up standards also in the surrounding public schools - has been rehashed in Britain to justify the Academies programme.

There is no evidence that this has happened in America. If it had, the US's rankings in the PISA tables may well have shown significant improvement over the 20+ years that charter schools have sprung up all around the US.

The New York Times reported not so long ago that the New York Schools Department had made tests easier, apparently in an exercise to demonstrate that the upsurge of Charters in the City had contributed to a rise in standards. Rachel Wolf of the New Schools Network went through a period when her every media appearance included talk of New York significantly closing the achievement gap. Not only was this claim based on dodgy methodology (by Caroline Hoxby) but it seems that after Joel Klein left office and the New York City Education Department re-instated the level of tests they had had before, results plummeted.

The idea that charters or academies can "shame" other schools into improving is therefore without foundation. As Henry here has pointed out, Academies do no better than maintained schools. There have been numerous schools that improved, long before Academies came on the scene and I wonder how many failing Academies are being shamed into improving to compete wih their neighbouring and excellent maintained schools?

I remember when Liz Sidwell, the Schools Commisioner, joined Twitter she explained that the reason she was visiting so many academies was to ensure that they did not fail! I never saw her mention maintained schools once and when I asked her why she did not, she sidestepped the question. I wondered whether the government actually wanted them to fail.

Harris Federation schools are trotted out as proof of excellence in much the same way as the American pro-charter supporters fling out HCZ, KIPP and New Orleans. Move away from this small group of priveledged charters and the failure of the system is laid bare when the most authoritative research to date shows that under 20% of charters actually outperform public schools. Harris Federation schools, whatever their disputed achievements, are not representative of academies as a whole. If they were, I daresay the DfE figures brought to our attention on his post would be much better.

Interestingly, Downhlls was improving but Gove had to burn the goalposts in order to claim eventual success for the school on the back of his Academies policy. But what are the goalposts for existing failing Academies? Perhaps they should be forced to become excellent maintained schools instead? No - this is all about Gove advancing his personal political ambitions at the huge cost of young people's lives.

Janet Downs's picture
Wed, 27/06/2012 - 13:35

Allan - one of the rhetorical tricks used by Gove and his supporters is to make a statement with which everyone agrees, something about every child having the chance to succeed, for example, and then use agreement with this statement to imply agreement with Gove's policies. Or, more likely, to say that opposing Gove's policies means disagreeing with children's entitlement to a good education. Gove and his supporters then label the opposition with such descriptions as "enemies of promise", "happy with failure" and so on. This earlier thread tackles Gove's "enemies of promise" head on.

http://www.localschoolsnetwork.org.uk/2012/01/i-am-not-%e2%80%9chappy-wi...

Janet Downs's picture
Wed, 27/06/2012 - 14:14

It's irreverent, I know, but when Mr Gove says, "How are the children?" it reminds me of the child catcher in Chitty Chitty Bang Bang.

http://www.youtube.com/watch?v=zUnhfvGdmmw

Add new comment

Already a member? Click here to log in before you comment. Or register with us.