What are these weaknesses in measuring school attainment the Auditor wrote about? Is the Gov't aware of them or are these 'weaknesses' too useful for propaganda?

Janet Downs's picture
 19
'The current measures have some weaknesses' said the National Audit Office in its recent report*. But what were these ‘weaknesses’ in measuring school attainment?

The Department for Education and academics told the NAO they were concerned these measures were unfair to schools in challenging circumstances. This seems to clash with propaganda pushed by the Government that there are ‘no excuses’ for failing to reach mandatory and ever-rising benchmarks. It’s encouraging someone at the DfE recognises judging schools merely on results can be misleading and there might be situations, especially when combined, which handicap schools. It’s a pity this insight doesn’t reach ministers.

The NAO noted the DfE was introducing a new progress-based measure from 2016. This was designed to be fairer than judging schools on test results. But the NAO warned these new measures would in the short term make it difficult to measure performance year-by-year. At the same time, Ofqual warned that changes in exams may 'affect national and school-level trends'.

That’s unlikely to stop politicians making such comparisons and trying to identify trends. For example, DfE number crunchers said it wasn’t possible to compare Key Stage 2 writing tests with earlier years because the test fundamentally changed in 2012. But that didn’t stop the DfE press office from comparing the 2014 SAT results with 2009 in one of its press releases. Neither did it stop the Tories from issuing party literature making the same erroneous comparison.

In other words, the many changes, all running concurrently, will make it impossible to measure whether performance has risen or not. And it doesn't address the issue of whether a rise in ‘performance’ is actually a rise in the quality of education offered in schools. As we’ve seen in the past, schools can adopt strategies which raise headline results but don’t have a positive effect on the education pupils receive. These include teaching to the test, overuse of equivalent exams (this scam’s been reduced, fortunately), ignoring certain subjects or other important skills and using methods such as tweaking oversubscription criteria to put off applications from pupils who are likely to reduce a school’s league table position.

Neither does it address an issue which will no doubt become prominent when the performance measures kick in: whether it’s possible to measure ‘progress’ fairly. Children aren’t identical – they don’t develop or ‘progress’ uniformly. They have growth spurts; they enter puberty at different ages; their learning can shoot forwards, stand still and even drop back - all at different times. The way children develop is affected by many circumstances combining nature and nurture. Expecting children to progress at the same rate could be as unfair on schools as the present flawed measure focusing merely on exam results.

*National Audit Office: Academies and Maintained Schools: Oversight and Intervention, October 2014 (page 21)
Share on Twitter Share on Facebook

Be notified by email of each new post.





Comments

Roger Titcombe's picture
Sat, 01/11/2014 - 12:42

Where do we start? What can the NOA be expected to know about education? Not much more than PWC I would guess. The following is from my book - out soon!

Individual Personal Development, not a tyranny of Testing

Enhancing the general cognitive development of individual pupils should become the key objective of the primary and secondary school curriculum and there should be no place in our schools for cognitively depressing and intellectually suffocating practices like teaching to the test and press ganging pupils onto courses with teaching methods designed for the primary benefit of the school rather than the learner.

Teaching for cognitive development will and should result in pupil attainments covering the full range of grades from A*-G appropriately reflecting the bell curve distribution of cognitive ability in the general pupil population. Attaching school performance indicators to exam results that are not soundly related to the cognitive ability of individual pupils is counter productive in educational terms and ultimately for all concerned in the education system.

School league tables based on crude performance indicators are an invitation to ‘gaming’ and a disincentive for schools to adopt the developmental approaches to learning that lead to cognitive growth. They are also false indicators of school quality because their very nature precludes taking due account of the fact of continuously variable pupil cognitive ability.

The 1988 Education Act will eventually have to be repealed or drastically reformed. This current period of what I call ‘Educational Lysenkoism’ (after the ideological Soviet theory of agriculture that became the compulsory orthodoxy under Stalin) will eventually be consigned to history as an essential lesson in how not to run a national education system.

This is not going to happen easily or quickly but a start can be made. The first essential step is to challenge the fallacy of the denial of the role of cognitive ability in predicting and evaluating educational outcomes. The second is to promote the design of the curriculum for the cognitive growth of individual pupils, not the accumulation of high stakes, target-related qualifications for the school.

The centrally controlled initiative roll-outs of New Labour rarely achieved the desired outcome and the free market based Academy and ‘Free School’ project promoted by the present Conservative led government are unlikely to be any more successful. So long as there are competitive school league tables driven by high stakes testing the education system will always be blighted and corrupted by perverse unintended outcomes arising from behaviourist incentives.

Effective learning and national cognitive growth (a positive Flynn Effect) requires the liberation and encouragement of teachers and schools to investigate, discuss, devise and apply approaches designed to secure cognitive development at classroom and individual pupil level.

Local Education Authorities should be re-established, liberated, educated, empowered and encouraged to promote cognitively stimulating learning in all their schools. London Mayor Boris Johnson has stated he wants control of the London education system. He is right to recognise the need for locally administered schools. The Learning Trust in Hackney provides an example from which a start could be made. Other good LA practice is emerging.

The few remaining ex-LEA Education Department staff with knowledge and experience that have survived the era of ‘Children’s Services’ reorganisation and the current cull of public sector employees should be attached to local school consortia to facilitate such shared work with more such experts trained and recruited. The disbanding of the Academies and Free School Division of the DfE would result in a multi £billion saving to the taxpayer that could be redirected for support of locally managed school improvement within a reformed culture.

University Schools of Education should again take a leading role informed by truly independent research. With competition between schools replaced by co-operation across schools (as has begun in Hackney and elsewhere) a start could also at last be made on restoring the sadly lost professionalism of teachers, which must be rooted in an appropriate degree of peer moderated classroom autonomy with regard to teaching methods, rather than ‘operative’ type ‘delivery’ of externally imposed initiatives. Only this, not performance related pay or ‘fast tracking’ will attract top graduates from the best universities back into a teaching profession with the necessary restored status.

OK it isn't going to happen any time soon, but that doesn't mean it's not right.

Janet Downs's picture
Sat, 01/11/2014 - 13:02

Roger - while it's true the NAO doesn't educate children it is required to express an opinion on whether the Government's preferred intervention (sponsored academies) is effective in raising a school's Ofsted judgement. The NAO showed the Government's preferred intervention isn't as effective in raising a school's Ofsted judgement as other methods in achieving its goal of improved Ofsted results.

Andy pointed out on the other thread about the NAO report that the NAO had assumed that 'Requires Improvement' schools were 'failing'. This is, of course, a direct consequence of downgrading 'Satisfactory'(satisfying the criteria) to 'Requires Improvement' (needs to 'improve' in order to satisfy the criteria). If only one example were needed of the politicisation of Ofsted, that's it. Downgrading Satisfactory shoved more schools into an unsatisfactory category making them ripe for academy conversion.

Ofsted aside, the NAO admitted there were flaws in how school attainment is measured. Its low-key statement about 'some weaknesses' is actually significant. It doesn't, of course, scratch the surface of what is wrong with the way schools are judged. But it did at least recognise there were 'some weaknesses'. And the NAO admitted the DfE had admitted the present way of judging schools could be unfair on schools coping with challenging circumstances. I wonder if school ministers endorse this.

Andy V's picture
Sun, 02/11/2014 - 09:06

"This is, of course, a direct consequence of downgrading ‘Satisfactory'(satisfying the criteria) to ‘Requires Improvement’ (needs to ‘improve’ in order to satisfy the criteria)."

For me this is an erroneous statement, an understandable and easily made one, but nevertheless erroneous. The grade Satisfactory was not downgraded it was withdrawn, scrapped, removed entirely (including the criteria it was based on). It was replaced with Requires Improvement, the criteria for which were different from the latter. This then was not a simple exercise of renaming/relabelling a grade category that continued to be judged using the same criteria. No, the grade and underpinning criteria are very different between to former and the new grades.

Neither can I agree that this action was, "[the] one example were needed of the politicisation of Ofsted, that’s it. Downgrading Satisfactory shoved more schools into an unsatisfactory category making them ripe for academy conversion." It is my recollection that SMW's position was that the minimum level of education should be Good not Satisfactory. The issues as to why the SoS Educ decided that the only DFE intervention strategy was academisation is a separate issue.

With regard to the NAO conclusions, I was perplexed and concerned that when they cited that, "59% of schools which received no formal intervention improved" there was no attempt made to clarify how improvement had come about and which category of schools were involved. I say this because it is highly likely that these school had insightful leadership teams that were able to use the section 5 report and data (e.g. raiseonline, internal tracking, internal lesson obs) to bring about the necessary improvements). These schools could easily straddle maintained and non-sponsored converter academies. This is another reason why I found the NAO report had a hollow / superficial ring to it.

Janet Downs's picture
Mon, 03/11/2014 - 14:10

Andy - the NAO wrote:

'2,181 schools (of 3,696) that received no formal intervention improved (this does not mean that it is better not to intervene formally, and may be because schools received informal support, or due to other factors that our analysis could not control for).'

I understand from this statement that the 'no formal intervention' schools could have received some kind of support which doesn't merit the description 'formal intervention'.

The NAO defined 'formal intervention' as:

1 'warning notices (a formal letter raising concerns about a school’s performance);
2 changing a school’s governing body; and
3 appointing a sponsor, which, for a maintained school, has the effect of turning
it into an academy.'

The NAO described 'informal intervention' as:

1 'drawing on external expertise from former headteachers or Ofsted inspectors, facilitating school-to-school support or
2 partnerships with external organisations, such as universities.'

Not only does informal intervention appear to be more effective than formal warning notices, replacing governing bodies and turning a school into a sponsored academy, it's likely it would cost taxpayers a lot less.

Andy V's picture
Mon, 03/11/2014 - 14:26

Janet, Thank you for the full response.

One of the main points I was making was that there have always been schools fortunate enough to have senior leaders with the nous to address and remedy issues in their schools, which necessarily reflects well on the colleagues who also step up the plate and play their part in turning things around. There is a great deal that could and indeed should be learned from this 59%, and then shared across all schools in the form of best practice.

In turn this begs the question as to why the NAO did not make this a recommendation in the report.

Janet Downs's picture
Mon, 03/11/2014 - 15:10

Andy - the recommendations (p11) included:

'The Department should undertake more work to understand the relative costs and
effectiveness of different oversight and intervention activities.'

which is I suppose the nearest we're going to get to telling the DfE to look at the 59% of schools which improved despite not being subject to formal intervention and stop wasting taxpayers' money on expensive and less-effective academy sponsorship which the DfE has for the last four years stubbornly promoted as the only solution to 'failing' schools.

Roger Titcombe's picture
Sun, 02/11/2014 - 09:53

Andy - I think we are agreed that Requires Improvement is not a 'fail' grade - in the sense that such a school can be described as 'failing OfSTED' if it RI. The logic of the notion that all schools should be at least 'Good' means no schools should 'Require Improvement'. This is not logical as is it implies that 'Good' Schools do not 'Require Improvement'. There are no schools that are so 'outstanding' that they cannot be improved.

I live opposite a rural C of E primary school that has been graded 'Outstanding'. However in April a 'Section 8 Monitoring Report' was issued. This is just a letter to the head entitled 'No Formal Designation Monitoring Report'. It is very critical of a single aspect of the school's safeguarding procedures. This is in relation to the congestion in the rural lanes surrounding the school and regular flouting of the zigzags in front of the school by parents. The congestion and traffic chaos every morning and afternoon really do put children at risk as the parked cars back up considerable distances along three narrow lanes that have no footways. Us locals have been complaining to the school for years to no avail, but now OfSTED has spoken. The root of the problem is that it was once a very small rural school serving a catchment of largely professional people. Unsurprisingly its SATs results have been excellent and so it has been regularly expanding, resulting in the next nearest school on a nearby council estate struggling.

I don't think the OfSTED grading system is really fit for any purpose other than driving competition. Inspection should be much more nuanced. I don't think there should be any 'grades' at all except 'Special Measures'. The reports should draw attention to the strengths and weaknesses of every school and parents should be left to make their own minds up about they think is most important to them. I would expect all inspection reports to recommend improvements in some form. The inspections should be carried out by reformed LEAs that are themselves subject to HMI monitoring. The LEAs should provide the support needed for schools to respond to their advice.

Isn't this what David Cameron and Ed Miliband should be addressing in relation to 'English devolution' - more power to locally elected bodies?

So I think Janet is right to draw attention to the confusion.

Janet Downs's picture
Mon, 03/11/2014 - 15:01

Roger - perhaps Ofsted should grade schools in the same way as it judges LA school improvement services: 'effective' and 'ineffective'. This would avoid the confusion over 'Requires Improvement' which has been used retrospectively to claim schools which were previously 'satisfactory' (as in satisfying the criteria) all now required improvement. It would also do away with the anomaly that schools judged Good also require improvement if they are to reach the hallowed heights of Outstanding.

The NAO pointed out that some schools hadn't been inspected for over four years. These are more likely to be those judged Outstanding who are exempt from further inspections. But Ofsted inspections refer to one point in time - it's misleading to assume an Outstanding judgement awarded in 2008 still stands.

Andy V's picture
Sun, 02/11/2014 - 10:16

1. I did not say or imply that it was wrong to draw attention to anything

2. My point regarding Janet's comments - I and all regular contributors to LSN know she is supremely competent to address .-, was that Satisfactory was not "downgraded" to Requires Improvement rather it was replaced by the latter (including a completely different set of criteria)

3. At no stage did I comment on, let alone indicate support for or opposition to, the linguistics of the grade titles

I have no idea as to the relevance of the CoE Primary section 8 inspection and the topic in hand?

Roger Titcombe's picture
Sun, 02/11/2014 - 12:56

Andy - I am not disagreeing or seeking an argument with you about anything. The point about my local primary school is that it has been judged to be Outstanding and yet it still apparently 'Requires Improvement' in relation to safeguarding. I don't understand what a Section 8 Monitoring Report with 'No Formal Designation' is. Is the school still 'outstanding' despite have been served with a Section 8 Monitoring Report in respect of safeguarding?

I was rather hoping you might shed some light on what I (and I suspect many others) find confusing.

Andy V's picture
Sun, 02/11/2014 - 13:26

"I am not ... seeking an argument with you ..."

Nor I with you. Indeed, all I did was respond in a direct and succinct manner, so quite where the idea of an "argument" came from I've no idea.

Andy V's picture
Sun, 02/11/2014 - 13:06

Try:

"Monitoring inspections of schools not in a category of concern

Other schools not in a category of concern

We will inspect a school not in a category of concern, if we have serious concerns as a result of:

a qualifying complaint
a request by the Secretary of State

other information which is brought to Ofsted’s attention."


http://www.ofsted.gov.uk/schools/for-schools/inspecting-schools/inspecti...

It strikes me that you answered your own question before you posed it. From your comment at 9.53 am it seems probably that Ofsted received a complaint/request/information regarding safeguarding arising from the concerns you related about traffic.

Thank goodness for search engines: I used, Ofsted section 8 inspection.

Roger Titcombe's picture
Sun, 02/11/2014 - 13:59

Thanks Andy. The report requires the school and the LA to take certain actions.
Certainly 7 months on there is no obvious change in the traffic chaos and parents still sometimes stop on the zigzags. I am still unsure whether the school can still claim to be 'Outstanding'. Or is it now a school that Requires Improvement? I find this a bit tough on the school as the remedies are largely the responsibility of the LA and the police.

OK, this is a bit of a distraction from Janet's thread, but I think it draws attention to what I see as general confusion in relation to Outstanding schools that nevertheless Require Improvement.

Andy V's picture
Sun, 02/11/2014 - 14:15

I've no idea why you think I might know and equally have no idea what the definitive answer is. Perhaps you should try posing the question to Ofsted directly.

Speaking as a professional - as senior school leader - I would think it reasonable to say that if the school can demonstrate that it has taken all reasonable steps to address the issue with the LA, local highways authority, parents and also taken steps to place adequate numbers of staff outside the school frontage to help supervise/chaperon (safeguard the children) then the governing body and head have done everything they can. This after all is nothing more or less that you or I would have done had it been our school.

Janet Downs's picture
Mon, 03/11/2014 - 14:19

Following my comment above about how informal intervention would likely cost a lot less than formal intervention, the NAO had this to say about the expense of academy sponsorship:

'Turning an underperforming school into a sponsored academy involves a substantial
initial investment. Sponsors receive grants to cover the costs of activities such as staff
recruitment, project management and leadership development. Although...the cost of all the main formal interventions is not known, the cost of this intervention is likely to be greater than that of other approaches such as changing governors or issuing a warning or pre-warning notice.' (p35 full report).

So, the most expensive option - converting a school to a sponsored academy - is less effective than other intervention methods both formal and informal. And the other interventions are likely to be less disruptive.

Janet Downs's picture
Mon, 03/11/2014 - 14:49

Interesting comment in the full NAO report re academy chains 'paused' from taking on more academies. The DfE told the NAO in August there were 18 paused chains (p39). But the announcement in March said there were 14. It appears, then, a further four chains have been stopped from taking on more academies. Why no publicity?

Just over a year ago, the DfE issued a press release with the heading 'New generation of academy sponsors driving school-to-school improvement'.

But now we discover that school improvement (as measured by Ofsted judgements) is more likely to come about if informal intervention (support from other heads) or formal intervention which is less costly than academy sponsorship takes place.

Yet another DfE press release shown to be hollow rhetoric.

Rosie Fergusson's picture
Thu, 06/11/2014 - 20:11

Certainly as far as primary schools are concerned OFSTED will ignore aggregate attainment figures below floor standard if progress and value added figures are good. However the low attainment may have triggered an inspection but a challenging school where progress is good can expect to have a fair hearing e.g a primary school in inner leeds got 35% Key stage 2 overall figure but some months later got a good rating from Ofsted. The proposed testing of pupils in reception is solely to set a baseline so such schools can finally get the accolades they deserve for their hard work thorughout Key stage 1


Rosie Fergusson's picture
Thu, 06/11/2014 - 20:14

safeguarding is a limiting judgement i.e if the school gets safeguarding judged as requiring improvement then they cannot be outstanding


Andy V's picture
Fri, 07/11/2014 - 09:58

It is my understanding that Ofsted don't ignore aggregate attainment but rather it is taken in the context of the whole inspection. Thus if the evidence is such that pupils are making solid progress and achieving well against their starting points - and there are no issues with safeguarding, L&M and BSP - then a school is unlikely to be graded below 3.


Add new comment

Already a member? Click here to log in before you comment. Or register with us.