The way the DfE measures school performance is ‘limited’ says Audit Office, and asks if DfE expenditure on oversight and intervention was money wasted.

Janet Downs's picture
 4
The way the Department for Education (DfE) measures school performance is ‘limited’, says National Audit Office (NAO), because it only focuses on education performance as measured by test results and Ofsted inspections. These are important factors, NAO said, but don’t reflect ‘the full range of children’s outcomes’. (The NAO didn’t define the ‘full range’ except to say some schools can have ‘safeguarding, leadership and financial issues’. Immediately we’re up against the difficulty of deciding whether schools are ‘underperforming’. The NAO appears to be concentrating on Ofsted judgements despite saying focussing on these was ‘limited’.)

The NAO mentioned floor targets in passing and cited the number of schools which fell below the benchmark. (But it didn't mention some before-floor schools are actually doing a good job, or that some above-floor secondary schools had inflated their results by a disproportionate use of ‘equivalent’ exams.)

So, did the NAO think the estimated £382m pa hurled at oversight and intervention was money well spent? Its findings included:

1The DfE relies on ‘limited measures’ – schools’ own financial reporting and ‘ad hoc intelligence’ ie whistleblowers – to flag up problems.

2The DfE hasn’t stated clearly defined the roles and responsibilities of external oversight bodies. The pace of change and mixed messages from the DfE and Ofsted has resulted in confusion about oversight of safeguarding, the role of local authorities in relation to academies and academy sponsors’ responsibilities (I would add the responsibilities of academy trustees).

3Decisions about interventions don’t appear to be made consistently. The NAO noted the Government’s preferred option was matching ‘underperforming’ schools with a sponsor but the DfE had been slow in intervening when some maintained schools were judged ‘inadequate. (The NAO didn’t seem to consider delaying intervention might actually be a good thing – there’s less chance of forced academy conversion or accusations of bullying if non-academies are allowed to work with LAs to improve. Avoiding conversion would save the taxpayer the cost of pre-opening, assorted grants and ‘rebranding’.)

448% of ‘underperforming’ schools had improved after intervention while 52% remained the same or deteriorated. But 59% of schools which received no formal intervention improved. This doesn’t mean, however, that no intervention is better than intervention, the NAO stressed (but it does show that formal intervention doesn’t automatically result in improvement).

5The DfE doesn’t know the costs of different interventions. The NAO found the grant given to sponsors to take over ‘underperforming’ secondary schools had fallen in real-terms between 2010/11 and 2013/14. (But the DfE still offers £150,000 for sponsors for ‘pre-opening’ secondary schools and start-up grants seem to be offered long after an academy has started up. This raises the question of how much the Government was previously paying sponsors if the amount of grant is supposed to have fallen.)

6The DfE doesn’t know enough about the effectiveness of LA oversight of schools. The NAO found the DfE had ‘serious concerns’ about 16% of LAs and ‘some concerns’ about 68% in 2011. (It’s unclear how the DfE came to its conclusion that 84% of LAs presented at least ‘some concerns'. But if the situation was as bad as the DfE claimed then it’s surprising school improvement services in only 11 of the 152 LAs have been inspected - 9 were judged Ineffective, but one, Isle of Wight, has been re-inspected and found to be Effective).

7The DfE doesn’t know why some academy sponsors are more successful than others. Not all sponsorships have resulted in improvement (that could be because school improvement is based on factors not linked to sponsorship). Ofsted can’t inspect academy sponsors or chains so there’s no independent evaluation of their work – this is a flaw.

8Oversight of an increasingly diverse school system is still developing – this has resulted in ‘inconsistent action’ from the DfE and others.

The DfE agrees with the facts presented by the NAO but not its analysis, conclusions and recommendations. (That’s probably because the NAO has highlighted weaknesses in the Government’s approach and implied that academy sponsorship as a way to improve schools is not a magic bullet.)

NOTE Words in brackets are the author’s comments.

ADDENDUM Read 'The 13 Most Important Points in the NAO Academies and Maintained Schools Oversight report' by Laura McInerney of Academies Week here.
Share on Twitter

Comments

Andy V's picture
Fri, 31/10/2014 - 12:24

All of which begs the interminable question, what outcomes should be reviewed and how should they be measured/gauged/assessed?

On the one hand the NAO are stating the profoundly obvious (e.g. education is about more than examinations and qualifications) but on the other hand they make no tangible indication as to what should be covered. So all in all it is something of a waste of time, effort and paper. It certainly take the topic no further forward. Indeed, the hollowness of the report leads me to raises questions about the value for money the NAO represents.

Janet Downs's picture
Fri, 31/10/2014 - 17:10

Andy - I can only assume the NAO was hidebound by its remit ie to decide whether formal interventions of the kind pushed by the Gov't (ie academy conversion with a sponsor) are effective and value for money.

To do this it had to measure 'outcomes' and whether these had improved or not. The NAO said the Gov't measure of outcomes was 'limited' but had, nevertheless, to measure these outcomes. Their research found formal intervention didn't raise outcomes in the way the Gov't envisaged in over 50% of cases.

But then, as you say, we have the question about how schools should be assessed. The NAO appeared to concentrate on Ofsted judgements (although it made a passing reference to floor standards). But some high-performing schools are not doing a good job according to Ofsted eg Chatham Grammar School for Boys, Park View School in B'Ham.

At the same time some below-floor schools are nevertheless doing a good job with their pupils according to Ofsted.

For example, Cuckoo Hall Academy was judged outstanding a few years ago - it will be one of those schools which hasn't been inspected for more than four years - but last year its KS2 results fell from 86% to 54%. Is it 'outstanding' or is it 'failing'? And is it a reliable judge of a school to assess it on one year's results alone (or, indeed, only on results)? If Cuckoo Hall wasn't already an academy it would no doubt have had brokers beating down the door. Normally Ofsted would inspect a school where results had suddenly declined but that hasn't happened in this case. And Ofsted reports for Cuckoo Hall Academy's predecessor school have disappeared from Ofsted's website. The only Ofsted record for Cuckoo Hall is its academy conversion letter. Some coincidence?

Andy V's picture
Fri, 31/10/2014 - 17:26

Janet, We may have to do the polite agree to disagree on this topic. This is issue is too important to be addressed through assumptions. That is to say, the NAO should either have audited the effectiveness of the measures currently in place or done that and then commented on the perceived narrowness of the prevailing measures but, and it is a big BUT, it was necessary to suggest or recommend changes (e.g. stop doing this and start doing something else).

It was not my understanding of the NAO of the report that Ofsted had been singled out. Rather political interference and DFE see-sawing bore the brunt and Ofsted inspection findings were cited as being too heavily relied upon by politicians and the DFE.

I was appalled to note that the report explicitly implied that any school graded 3 or 4 was failing, which is simply untrue and woefully misleading. To require improvement is not the equivalent of failing. Indeed, it could be argued that anything other than special measures is not failing (i.e. serious weaknesses although a different grade than RI has a commonality in that the school leadership is judged as having the ability to bring about the necessary improvement whereas school leaders in schools requiring special measures are judged not to have that ability). Thus for the NAO to label all RI schools, their staff and pupils as failing is preposterous.

Janet Downs's picture
Fri, 31/10/2014 - 17:38

Andy - I was explaining how the NAO's report was hidebound by its remit (to judge schools according to the Government's own flawed measure). That's not to excuse it. Explaining doesn't mean I agree with their narrow definition of improvement. Neither do I agree with their assumption that Requires Improvement schools are failing - that's another thing that is pushed by the Gov't - in my opinion the decision to downgrade Satisfactory (as in satisfying the laid-down criteria) to Requires Improvement was a deliberate ploy to redefine Satisfactory schools as 'failing' and therefore make them ripe for academy conversion.

That said, according to the Government's own definition of 'improvement' the Government's preferred method of improvement has been found wanting. At the same time, the NAO criticised the way the Gov't had been inconsistent and that Ofsted didn't inspect academy chains in the same way as LAs.

The NAO also implied that the Gov't had been too keen on describing LAs as failing - the high proportion the Gov't said had 'some concerns' or worse, for example. Yet Ofsted has only inspected 11 of this high proportion. This suggests the high proportion wasn't really that high.

Add new comment

Already a member? Click here to log in before you comment. Or register with us.