Stories + Views

Posted on

29/01/12

go to 23 comments

Academies: The Evidence of Underperformance

The massive release of data by the Department for Education (over 200 pieces of data on each of over 5,000 secondary schools) makes possible a thorough analysis of how well different types of school have performed. The evidence is clear and overwhelming: Academies have not been the success story that their supporters have claimed. Instead there is a clear record of under-performance.

The overall figures have long been clear, for the key measure of % achieving 5 A-Cs at GCSE including English and Maths:

Academies: 47%
Non-academies: 60%

The data now includes a figure for the % achieving 5 A-Cs including English and Maths but without counting non-GCSE qualifications like Btecs. Here the difference is even more stark:

Academies: 34%
Non-academies: 54%

The gap is huge but this is an unfair comparison. We know that the raw % pass rate (though currently Ofsted’s  favoured figure) is closely related to the ability of the students at entry. We know that schools in disadvantaged areas tend to achieve lower % for 5 A-Cs and we know that the early academies were more likely to be in disadvantaged areas. So does this explain the discrepancy?

The answer is a resounding no. To analyse this, I split the data into five comparison groups according to the % of students on free school meals. The first group, the most advantaged, is of schools where less than 10% are on FSM and so on up to the most disadvantaged where more than 40% are on FSM. Academies still perform worse than comparable non-academies.

The figures with GCSEs only show a bigger difference:

 

The message is clear. When academies are compared to comprehensives with the same level of disadvantage, their results are worse.

Another example: the % making expected progress in English and Maths:
Chart of % making expected progress in English.

 

The same is true for % of FSM students acheiving % A-Cs (with English and Maths), with equivalents and without. The same is true for % FSM students making expected progress in English, and also this % in Maths. Indeed Academies seem to underperform virtually however you analyse it. The one exception I’ve found is Best 8 Value Added (the value added in a student’s top 8 GCSEs) where Academies rating is slightly better, though within the margin of error. However even here the difference is explained by the use of Btec and other equivalents. On the rating for value added in English and value added in Maths, Academies are slightly below average.

This government claims that Academies have such a strong proven track record that every school could convert to them. They quote schools like Mossbourne and Burlington Danes in support. However this is policy-making by anecdote not by evidence. Both those schools are outstanding but they are clearly, from the data the DfE release this week, not the norm for academies. It is likely to be other factors than their academy status that is the cause of their success. If government education policy was genuinely evidence based, then they would now be looking at converting academies to LA-supported non-academies, in the hope that this would raise their results.

Data Notes: The Academies  figure, here and throughout this post, refers to the category of sponsor-led academies, of which there are 249. It does not include the ‘converter academies’, of which there were just 25 at this point. Non-academies include those classified as community, foundation, CTCs or voluntary aided schools, 2,681 in total. Special schools are not included.)

Share this page:

Comments, replies and queries

  1. Tim Bidie says:

    ‘Polish Officer: You’re supposed to keep a look out like soldiers. Not talk like old women. What are your names?

    Jones: Jones, sir.

    Pike: Pike, sir.

    Walker: Smith.

    Jones: Walker.

    Walker: Oh thanks very much.

    Polish Officer: It’s no good you try and give me falsies.

    Top skills!

    But we can’t see, from your piece, how long each of them have been academies for?

    • Tim, the DfE data doesn’t say when they became academies. However, as I note at the bottom, this data is based only on the sponsor-led academies and not on the more recent ‘converter academies’. Also the majority, 174 of them, had no 2008 results and so presumably were new build academies and so would have been open as academies for between 5 and 7 years. (The academies which haven’t yet reached Year 11, and so have no GCSE results, are of course not included.

      PS: Do you really know that Dads Army dialogue by heart or did you make it up?

      • Julie Davies says:

        I’ve got a few questions:

        How many successful Labour academies were new-build freshly-opened schools with just year sevens in the first year, like Mossbourne?

        How many successful Labour academies were struggling schools that improved on becoming academies?

        How many Labour academies have closed or failed OFSTED inspections?

        How many primary schools became academies under Labour?

        How many Tory converter academies have been high performing schools and selective schools?

        What percentage of Tory converter academies have been struggling schools?

        What percentage of primary academies have been good and outstanding prior to conversion?

        Is there somewhere I can find any of this out? Anybody know?

        • Julie, I can answer some of these questions but not all. The data gives results for 2008, 2009, 2010 and 2011. So we can find out how many did not have results in 2008 and were therefore presumably new builds. (Mossbourne’s first GCSE results were in 2009 and it was among the first 20 academies.) As I’ve said above there were 174 of these.

          One interesting fact from the data is that new build schools do less well than existing schools of similar advantage or deprivation. So, for schools with over 40% FSM, new build academies achieved 44% s A-C EM while existing schools achieved 50% (existing academies being equal with existing non-academies). The conventional wisdom is that a new school, with new buildings, new head, fresh intake will do better. The data indicates the opposite, that they do worse.

          The converted struggling schools did generally do better on becoming academies. Academies in the 30-40% FSM band, for instance, went from 32% 5 A-C EM in 2008 to 46% in 2011 But so did struggling schools that did not become academies. Those in that band went from 33% in 2008 to 47% in 2011, exactly the same 14% increase.

          Tory converter academies: The first 25 of these are included in the data (presumably the number that had converted by last summer). I don’t think any were struggling. Gove made a condition that the first ones were rated Outstanding. The data tells us that on average these 25 converter academies achieved 76% 5 A-Cs including English and Maths in 2010 and that none of these converters had more than 20% of their students on free school meals.

          But if you are looking for a conclusion that Labour academies were a success, Tory ones a failure, the data does not support it. It is the Labour sponsor-led academies that are indicted here as under-performing, especially the new builds.

          I don’t think any primary schools became academies under Labour. I don’t think that was an option then. I think you will find all converting primaries were Good or Outstanding as I understand that was a condition of conversion.

          You could ask a Freedom of information question of the Department for Education to find out how many labour academies closed (did any?) or failed Ofsted. Also the data doesn’t identify which schools are selective so you’d need an FOI request for that too.

          • Julie Davies says:

            Thanks for your work on this. They were genuine questions and I wasn’t looking for a party political answer. I have a Labour academy in my area. Nice idea, and nice people, but it’s not a high performer these days. Schools that were way below it have lapped it in ten years and stayed as community schools.

            The new schools issue is an interesting one. We have a converter academy which was newly set up at the same time as the first academies. Its first results weren’t great either though it had all the demographics in its favour.

            Mossbourne’s reputation is that it was Hackney Downs rebranded, whereas it opened nine years after Hackney Downs closed. Did their results take the ‘new school hit’?

            Incidentally, there is a big issue about free school meals to dig into. The eligibility criteria rule out a lot of hungry children and the pupil premium isn’t reaching them or their schools.

      • Tim Bidie says:

        Only 203 Academies in 2010, mainly failing schools but 1114 further schools applied for Academy status in that year.

        These figures suggest that a more comprehensive picture of Academy performance will emerge in the years ahead.

        We must all hope for the best.

        Intuitively, I must say that the Academy program looks like a much better way forward, given that retaining the status quo was clearly not an option.

        ‘Ian Foster, Chair of the Board – Academies Enterprise Trust:

        ‘It seemed to me so often in recent years that, with a few notable exceptions, the Academies programme provided failing schools with reward and opportunity whilst excluding the best schools from the freedoms and opportunities that becoming an academy offers. I am very pleased that the new policy addresses that whilst, at the same time, continuing to allow more challenged schools to become academies.’

        P.S. ‘Don’t tell him your name, Pike!’

        • Tim – hoping for the best is NOT an option. You quote Ian Foster, Chair of the Academies Enterprise Trust, who is hardly free from bias. He says there have only been a few “notable exceptions” but on the whole academy conversion has been a huge success. However, academy conversion has not delivered in a large number of cases. There might actually be genuine reasons for this – an intake heavily skewed to the bottom end such as St Adhelm’s in Poole. But conversion is NOT a magic bullet.

          Academy conversion is a chimera. The much touted autonomy over curriculum, resources and so on was already enjoyed by English schools – that has been the case since local management of school was introduced (see link below for more information).

          Where academies have been successful, it is for the same reason that other schools have been successful which could include a combination of good leadership, high expectations, enthusiastic staff and so on. What is needed is thorough and unbiased research into what makes a successful school. We also need to be clear about what we mean by a “successful school”. Is it one that is merely an exam factory, one that serves the needs of industry, or one that provides a fully rounded education?

          This is where the debate is. Instead, we get damaging policies backed up by dubious data.

          http://www.localschoolsnetwork.org.uk/2011/06/the-myths-of-academy-conversion/

          • Tim Bidie says:

            Janet-Thanks, as ever. I very much agree with your definition of the debate. Naive, I know, but wouldn’t it be great if that debate could be conducted outside politics?

            Not many here may agree but I very much see the Academy system, supported by both Labour and Conservatives, as a chance to move education, once and for all, out of the political arena and into the control of hands on educationalists.

            Given the ‘fait accompli’ of so many schools now becoming academies, I, for one, certainly hope for all the success possible for their youngsters.

  2. I have another.

    How many converter academies are high performing secondary modern schools working alongside selective schools and how do post-16 opportunities compare for ebac?

    • Alan, I can’t answer that. The DfE data doesn’t detail which are selective and which aren’t. You’d have to put in a Freedom of Information request to get that answer.

      • Thanks, Henry. Another glaring omission that could affect the data.

        • Jane Eades says:

          Thanks very much, Henry, you must have spent the weekend working! On the table I have downloaded but haven’t had time to look at it does identify which schools are comprehensive, selective or ‘modern’. Of course, that refers to the school’s intake policy. A comprehensive school next to a selective school would hardly be what we would call truly comprehensive.

          Do we also know how many of the New Labour academies were actually “failing” schools? One aspect I noticed was that many of the schools which became sponsored academies already had improving exam results before conversion and some had excellent results. (Let us also not forget the 6 private schools and the extra cash).

  3. Is there any data available for how academies and non-academies have improved this year over previous years’ results?

    Cheers,
    Colin

    • Colin – the info is available on the DfE schools performance tables site. You would need to find each school individually and then scroll down to find the figures for GCSE results from 2008 – 2011.

      It would be a mammoth task to look at the 3,000+ secondary schools in England to find out which ones had improved their results or not. However, on another thread I’ve asked Ben Taylor if he would do it for grammar schools, schools classified as “modern” and those schools classified “comprehensive” which do not have comprehensive intake (eg Marlowe Academy in Kent, a selective area). He said the analysis could be done “fairly easily” so perhaps we can expect the answers soon. Or perhaps not.

  4. Julie – thanks for the information about Hackney Downs. Until I read your post I had been under the impression from media reports that Mossbourne had grown out of the “infamous” Hackney Downs. But, as you say, Mossbourne was established in 2004 and Hackney Downs had closed in 1995 (during the last Conservative administration). Its closure was acrimonious according to two TES articles I’ve found (linked below).

    This demonstrates how pervasive propaganda can be. I don’t live in London and consider myself fairly knowledgeable about education matters. However, I’d believed as a “fact” that Mossbourne was, as you say, “Hackney Downs rebranded”.

    Thanks for putting me straight.

    http://www.tes.co.uk/article.aspx?storycode=14996

    http://www.tes.co.uk/article.aspx?storycode=89265

  5. Tim – the academy conversion will not move education “out of the political arena”. The Secretary of State is intimately involved with academies through their funding agreements. And the present Secretary of State does not trust the “educationalists” although he keeps saying they should make decisions. But these decisions must adhere to what the government wants. Schools which resist academy conversion are called enemies of progress; local authorities where schools don’t purchase the government-approved phonics teaching materials are named and shamed; and secondary schools which don’t “persuade” their pupils to take EBac subjects find themselves pilloried in league tables (see typical article below).

    Anthony Crosland, Secretary of State for Education 1965/7, said in an intervew: ‘The nearer one comes to the professional content of education, the more indirect the minister’s influence is. And I’m sure this is right …generally I didn’t regard
    either myself or my officials as in the slightest degree competent to interfere with the curriculum. We are educational politicians and administrators, not professional educators.’

    Yet all governments during the last 25 years have sought to interfere in education. Dismantling local authority support; controlling schools from the centre; showing favouritism to one type of school; demoralising teachers by constantly harping on how bad English state education is; hiring a chief inspector of Ofsted who publicly boasts about how low teacher morale is a good thing; allowinging free schools to employ unqualified teachers; encouraging excessive emphasis on raw results – none of these things is taking the politics out of education.

    As I’ve said before: “They create a prison and call it freedom.”

    http://www.telegraph.co.uk/education/leaguetables/9041349/GCSE-league-tables-pupils-shunning-tough-subjects.html

    http://www.localschoolsnetwork.org.uk/2011/12/%e2%80%9cthey-create-a-prison-and-call-it-freedom-%e2%80%9d-schools-education-providers-and-autonomy/

  6. Tim Bidie says:

    Indeed Governments have interfered in education forever and a day!

    Crosland had mixed reviews himself regarding balance, particularly from his wife (and why not!):

    ‘On the one hand, taking a lead from Boyle, some Tories had expressed their dissatisfaction with selection at eleven. Many of them had ‘experienced at first hand the frustration and anger of parents whose children had failed the 11+’. On the other, there were those on Crosland’s own side who remained deeply committed to the grammar schools that had, in many instances, facilitated their own social mobility.

    Crosland himself was a product of the public school system and did not feel the personal
    gratitude and allegiance to grammar schools felt by some of his colleagues. His wife
    famously quoted that he had pledged ‘to close every f*****g grammar school in England’

    http://www.reflectingeducation.net/index.php?journal=reflecting&page=article&op=viewFile&path%5B%5D=95&path%5B%5D=100

    I personally believe that localism is better served at the lowest, most local level administratively possible.

    That means that I believe in schools set free to run themselves, with a remote and benign regulatory regime.

    Communications today are so instantaneous that the regulator’s job is a great deal more straightforward than hitherto.

    Clearly you regard the current educational regime as despotic, yet benign despotism has many qualities, though I concede that democracy, the least worst form of government, is not one of them.

    The Ofsted Chief Inspector also said:

    “The essential truths are that a poor leader runs a poor school; a good leader runs a good school. A good teacher can make a difference in a classroom; a poor teacher makes little or no difference. I think we know what makes a good school. We just need to make sure it happens on the ground now.”

    That will resonate with many.

    Great quote of yours, should have been an Eagles lyric.

    This one won’t help:

    ‘We cannot change anything until we accept it. Condemnation does not liberate, it oppresses.’

    • Tim – brilliant quote at the end (Who said it, by the way?). I think it needs to be sent to Michael Gove next time he criticises English teachers (unless they’re in the favoured academies, of course):

      “Condemnation does not liberate, it oppresses.”

  7. Leonard James says:

    ‘The massive release of data by the Department for Education (over 200 pieces of data on each of over 5,000 secondary schools) makes possible a thorough analysis of how well different types of school have performed.’

    I’m having trouble finding this information. Is there a link to it?

  8. Link to 2012 school performance tables is below. Link to DfE Statistics is also below – click on the data you want to have a look at. You’ll find details of performance of different types of schools in DfE: GCSE and Equivalent Results in England, 2010/11 (Revised). Details re performance of ethnic minority children, FSM and so on is DfE: GCSE and Equivalent Attainment by Pupil Characteristics in England, 2010/11.

    http://www.education.gov.uk/schools/performance/

    http://www.education.gov.uk/rsgateway/DB/SFR/

  9. [...] of Local Authority control and run in part by private companies or voluntary groups have performed consistently worse than other comparable schools.    In healthcare, although the evidence is somewhat complicated, [...]

Want to follow comments on this post? Use the RSS feed or subscribe below

Reply

Academies: The Evidence of Underperformance

1 Trackback