DfE “Coasting” measures: the worst it could possibly have found?
When introducing the concept of “coasting” schools, it was saidthat the aim was “to identify schools that have previously fallen ‘beneath the radar’ in leafy areas such as Oxfordshire and Surrey; results seem good, as they have high-attaining intakes, but the department says children are not being pushed to reach their full potential.” It is hard to think of any set of measures that the DfE could have used that would have been less suited to this purpose than those that were chosen. It is not clear if it is deceit or incompetence, but all three measures being used in secondary schools are directly related to the intake. Schools with “high attaining intakes” are very unlikely to be defined as coasting.
GCSE 60% benchmark: Closely related to intake
The first measure of “coasting” is a school that falls below 60% of pupils achieving 5 GCSEs including English and Maths. There is a close correlation between how likely a school is to pass this benchmark and the average KS2 point score: Of schools whose children have an average KS2 point score of 30 or more (where a 4a is equal to 29 pts and a 5c to 31 points), precisely none would be caught by this measure. However of schools with an average point score below 26 (where 4b is equal to 27 pts), 97% fail to reach the benchmark. The government claimed that the “coasting” measure will affect schools whose “results seem good” but children are not being pushed. However schools whose results seem good are specifically excluded by this measure from being classed as coaching. Either the policy makers at the DfE have very limited numerical understanding or, for the first two years of this new measure, they never intended to include these schools.
Expected Progress: Also directly related to level of intake
The other two measures of coasting are % of pupils making “expected progress” in English and in Maths. This chart shows how the likelihood of not meeting this criteria relates to the school’s intake in terms of KS2 points: Of secondary schools with an average pupil KS2 point score of less than 26 (where a 4c is 25 and a 4b is 27), 70% in English and 82% in Maths fall below the threshold. In contrast for schools with an average point score of 30 or more (where a 4a is 29 and a 5c is 31), only 2% in English and 1% in Maths fall below the threshold. The 163 English grammar schools are all in that top category (along with 35 comprehensives) and therefore certain to avoid being classed as “coasting”.
“3 levels of progress” does not stretch the “more able”
The reason is simple. The expected 3 levels of progress means a stretching grade C for a pupil entering with a 4c grade at age 11, but only a grade B for a pupil entering with a 5a. As Michael Wilshaw has argued, grade 5 pupils (especially those on 5b or 5a) should be achieving As. Take an example: If a grammar school had 100% of its intake entering with level 5s (as some grammars do), then it should be expected to get them to As or A*. However “3 levels of progress” only requires them to achieve a B and being above the median would in 2014 have meant getting 74% to a B in English or 67% in Maths. If this grammar got no As or A*s and got only 70% of its students to a B in Maths, then it would still not be deemed “coasting”.
Expected progress: “the very worst indicator”
The independent Education Datalab has described the expected progress measure as “the very worst indicator routinely published about schools“. I would agree entirely and have previously written about how it is a “deeply flawed measure“, a “very silly measure” and an example of “how to use data badly“. This might look like a bit of an obsession but the fact that all three posts are in our top 20 viewed, in 2015, suggests there is wide interest in the subject. Education Datalab has also shown how the likelihood to be coasting, on the current definition, is directly related to the school’s intake. On the current definition, 30% of schools with the weakest intake are set to be rated as “coasting” but virtually none of those with the strongest intake.
Schools with disadvantaged intakes far more likely to be termed “coasting”
Nicky Morgan specifically stated that “For too long a group of coasting schools, many in leafy areas with more advantages than schools in disadvantaged communities, have fallen beneath the radar.” However under the current “coasting” definition, those schools are set to continue to fall below the radar. Taking again the “expected progress” element, those schools with low levels of disadvantaged children will be far less likely to fall below the threshold: Secondary schools with over one in five students categorised as disadvantaged are far more likely than average to fall below the threshold for “expected progress”, with over 55% falling below one of the thresholds. Of the least disadvantaged schools, only 21% fall below the threshold in English and 11% ijn Maths. The reason is that level of disadvantage in a school correlates with lower average KS2 point score. And, as we saw above, lower KS2 point score makes it far harder to hit the threshold.
Will “coasting” be based on school intake or on school results?
There is some sense in finding schools where students are not achieving their potential, if they then receive the support they need to improve. However the current definitions mean that whether a school is defined as “coasting” will depend more on its intake than its results. As the Education Bill continues through the Lords, Lord Lucas wrote to Lord Nash expressing concern that a grammar school that was “achieving respectable results but failing to stretch its pupils would never be caught by the proposed coasting definition”. In response, Lord Nash acknowledged that this was true and would continue to be true until 2018. “I do accept, however, that the attainment element of our proposed interim definition may prevent a school like this being defined as coasting in 2016 and 2017.” From 2016 the GCSE benchmark will be replaced, in the definition of coasting, by the new Progress8 measure. As the coasting definition is based on three years of figures it will take until 2018 for the measure of coasting to no longer include the GCSE benchmark.
Can the DfE produce a genuine value added measure?
The DfE is currently consulting on its definition of “coasting” and it may be that it will change it. (And I will submit this post to the consultation.) However, since the abolition of Contextual Value Added, the DfE has found it very difficult to produce a measure that genuinely measures value added, without being biased to schools with stronger intakes. The standard current measure (before Progress8 comes in) is Best8 Value Added. This is meant to show the genuine value a school adds, regardless of student starting points. It is one of the main measures used by Ofsted to judge the performance of a school. However if we plot the value added figure against average KS2 pts for school, we find a clear correlation: Under Best8 Value Added, 1000 is the national average. Each GCSE grade represents 6 pts. The 51 pt difference between the top category and bottom category therefore represents a huge 8.5 grades. This indicates a strong bias towards those whose students arrive with high grades.
Why has the DfE produced such an inappropriate measure of coasting?
Did the government never intend to tackle "underperforming" schools in leafy areas? Or does it really not understand the measures it has chosen to use? This may change in 2018 with Progress8. However there are already major doubts being expressed about this measure: for instance, Schools Week found that it "undervalued low ability pupils" and Tom Sherrington (headguruteacher) has described it as "data garbage".