NFER Research on Academy performance seems to back up LSN analysis ...

Peter Martin's picture
 3
See here.

Key Findings from NFER published July 2014 (added by Janet Downs directly from the link provided by Peter Martin, her emphasis):

1 'Attainment progress between Key Stage 2 and Key Stage 4 outcomes, such as capped point score and percentage achieving 5+ A*-C grades including English and maths, is higher after two years in sponsored academies compared to similar non-academy schools.
2 There was no significant difference in attainment progress after two years between converter academies and similar non-academy schools, suggesting the school performance benefits are limited, at least in the short term.
3 Attainment progress in sponsored academies compared to similar non-academies is not significantly different over time when the outcome is measured as GCSE points, excluding equivalent qualifications such as BTECs. This suggests that sponsored academies either use more equivalent qualifications, or that their pupils do better in them. The same was found for converter academies, though to a lesser extent than in sponsored academies.'

And from the full report (added by Janet Downs, her emphasis):

'Analysis of 2013 exam results appears to show more progress amongst converter academies than all non-academy schools, especially among the very first converters, that became academies in 2009/10. These schools were all rated ‘outstanding’ by Ofsted at the time, so greater progress made in 2013 might be better explained by pre-existing differences rather than the impact of academy status.'

'A more robust longitudinal analysis shows no significant difference in attainment progress after two years between converter academies and similar non-academy schools, suggesting the school performance benefits are limited, at least in the short term. This could be interpreted as mean reversion counteracting a positive academy impact, though mean reversion has been partially addressed by excluding non-academy schools from the analysis that are not a good comparison with academy schools. A longer time frame may be needed to fully assess the relative performance of converter academies, but the data so far suggests academy status has made no difference to the progress made in converter academies, compared to similar non-academy schools over the same time period.'
Share on Twitter

Comments

rogertitcombe's picture
Wed, 05/11/2014 - 18:25

Peter and Janet - I did a lot work on sponsored academies and how the vocational curriculum route was used to inflate C grade-based results. The resulting curriculum degradation and restriction of opportunities for a large proportion of pupils to access a broad and balanced education, are outcomes that do not register on KS2 to C grades progress calculations.

I managed to obtain the 2006 results for a small number of Academies from Local Authorities (at that time Academies were not bound by FOI) and these revealed an alarming pattern of curriculum degradation along the same lines as the 2004 ‘most improved’ schools (my 2005 TES study) but with even more draconian outcomes in terms of restricting access to mainstream curriculum, not just to less able pupils, but in some cases to all pupils.

In one Academy, there was no GCSE science at all on the KS4 curriculum: just 1% of pupils gained an A*-C in history, and 6% in geography. This school achieved an impressive 61% 5+A*-C but only 15% when English and maths were included. Despite this in its 2004 OfSTED inspection the Lead HMI Inspector wrote, “Standards in science lessons are rising…”. She went on to note that the curriculum was “sound” for most pupils but unsatisfactory for those with Special Educational Needs (SEN), and that it had been “broadened” at KS4 and “unusually” at KS3, “where pupils in Y9 were taking up to two vocational GCSE courses”. The overall conclusion was that the academy was, “improving rapidly”, the quality of leadership was “sound” and “the new principal is providing good leadership”.

In another Academy, despite achieving 34% 5+A*-Cs, just one pupil achieved an A*-C pass in double award science (the science course recommended for all pupils at the time by DCSF), one in Spanish, and none in history or geography.

In a third Academy just 9% of pupils gained an A*-C in double award science, 5% in history, 2% in geography, 1% in French and 3% in German, yet 48% gained 5+A*-Cs. The only comment in the 2006 OfSTED inspection report related to these results was, “The secondary phase curriculum is satisfactory”. The judgements on the sixth form were however damning. The curriculum provision was graded as “inadequate, lacking breadth and balance, and offering only a limited range of courses”. The Lead Inspector did not made the obvious link between the poverty of provision for mainstream academic subjects at KS4 and the ability of the school to provide a full range of opportunities in the sixth form.

This was before the corrupting and distorting effects of 4 x C vocational equivalents was recognised leading to the requirement to include English and maths in the 5A-C figure. The result of this has been to spread the curriculum degradation of the rest of the curriculum to these vital core subjects.

On 29 January 2014 I posted a thread on LSN with the provocative title, ‘Is school improvement a good thing?’. In my post I mentioned my research described above.

This is one of the anonymous responses:

“Roger – many thanks for your post – it contains so much of importance. I completely agree with you about the false concept of ‘school improvement’. I can give an example from my own experience. To get the % of maths C+’s up the school employed a range of strategies including the following:
- Pupils began studying the GCSE curriculum in Y7 and as soon as they were able to get a C they sat the exam (many of them in Y8). There were many, many resits until the magic C was achieved.
- From Y9 the C/D borderline pupils were taught in small groups with multiple teachers – all other groups were larger with just one teacher (and the groups got bigger through the year after each round of exams).
- Maths was given more time on the timetable at the expense of everything else. Maths teachers were ‘encouraged’ to provide daily ‘maths intervention’ classes in the morning before school and at the end of the day.
- Pupils were rewarded for attendance with free take-away food. C/D borderline pupils were ‘paid’ with shopping centre vouchers if they got their C in Y10 instead of Y11.
- Pupils were withdrawn from other lessons to do extra maths in the fortnight leading up to the exam.
- Pupils were entered for multiple exam boards.
- Pupils were entered for multiple routes (linear and modular) at the same time.
- Private tutors were bought in by the school to work one-to-one with individual C/D borderline pupils.

The overall effect is to increase the % getting C in maths but at the expense of higher and lower achievers in maths. It also impacts on the results in all other subjects because of loss of timetable allocation and withdrawal of students from classes on an ad hoc basis. The pressure on pupils to achieve the pass was immense and destructive and led to lower levels of commitment and motivation in other subjects. Regarding the relationship between use of equivalents and lower attainment in GCSE’s – your point about less skilled teaching staff being employed is correct, but a more important point is that once pupils get used to a much lower level of demand in the ‘equivalents’ lessons they often find it very difficult to raise their game to the level needed to perform in a more demanding subject. ‘Cut and paste’ assignments and poorly structured, low-level brush-stroke analysis is often sufficient in BTECs but is no good in academic subjects like history, English literature or physics.”

I am reminded of the oft-quoted aphorism from the consumer finance sections of the broadsheet press: ‘If it seems too good to be true then it usually is’.

Gove was right to end these abuses. However not only did they wreck the educational opportunities of hundreds of thousands of pupils, they also rendered the sorts of comparisons attempted by NfER in this thread as difficult to reconcile with the collateral damage not collected by the 'improvement' data.

And that's just the sponsored Academies. When it comes to converter academies this thread just adds to the growing mountain of evidence that they don't even produce the illusion of raising educational standards.

Janet Downs's picture
Thu, 06/11/2014 - 11:52

Despite the NFER research dated July 2014 finding academy conversion and sponsorship didn't make much difference, schools minister Nick Gibb was still telling the Commons on 27 October:

'Results over a number of years show that established sponsors are delivering sustained improvements, helping to transform the life chances of thousands of pupils.'

Andy V's picture
Thu, 06/11/2014 - 17:21

From my perspective the initial wave of 'converter' academies can and should be automatically discounted because the criteria was that it was only available to existing grade 1 (outstanding maintained schools). From this it follows that they are only of interest if following conversion they were inspected and graded 2 or lower. But even this has wrinkles in that between Sep 10 and Sep 13 the grade criteria changed; not the least impact being the removal of Satisfactory and operation of the brand new grade Requires Improvement, which makes it even more difficult to accord any downward trend to academisation rather than revised inspection criteria (i.e. raising the bar).

It may perhaps be less controversial to try and assess the impact of chain/sponsored academisation only. I suggest that because it avoid the mulitple academy trusts setup without a sponsor (e.g. a single converter allied to a local primary) and means almost inevitably evaluating the impact of a school graded 3 (R/I) that was unable to attain grade 2 within 2 years and then the schools graded 4 that went directly to sponsored route.

Add new comment

Already a member? Click here to log in before you comment. Or register with us.