Why 3 Levels of Progress is a Very Silly Measure

Henry Stewart's picture
 1
Note: See How to use data badly: levels of progress for an updated version of this, including 2014 GCSE figures

Increasingly schools are being encouraged, by the Department for Education and by Ofsted, to focus on achieving three levels of progress for every student. (Three levels represents going from Level 3 at age 11 to a D at GCSE, level 4 to a C and level 5 to an A.) The flaw in this approach became clear this week when Ofsted criticised schools where students who achieved a level 5 in SATs only reached B at GCSE, even though this represents the desired 3 levels of progress.

3 levels of progress (also termed "expected levels of progress") sound fair and equitable, being based on individual value added and expecting the same of all students. But a quick analysis of the statistics shows that the level of progress to be expected depends on the starting point of the child. For those achieving a 5a in Maths, fully 99% make 3 levels of progress. But for those starting with a 3c, only 16% made 3 levels of progress in 2012.

Note: This post has been edited & updated: The transition matrices include two tables for transition in each of Maths and English, one based on SAT levels for the student in that subject and one based on SAT levels across English, Maths and Science. This post originally included only the latter, but now includes both. Thanks to @giftedphoenix for this work on this, including showing the full transition tables.

Across England, % of students making 3 levels of progress - by SAT level (in that subject)

     English    Maths
3c40%25%
3b56%42%
3a70%60%
4c51%56%
4b71%76%
4a85%90%
5c70%67%
5b92%86%
5a99%96%


 

Across England, % of students making 3 levels of progress - by SAT level (average of English, Maths, Science)

SAT     English      Maths
3c33%16%
3b49%26%
3a66%45%
4c45%44%
4b64%68%
4a82%87%
5c66%68%
5b88%92%
5a98%99%


 

Sheet3 Chart 2A Perverse Measure

Imagine two teachers in a school which sets for Maths:

In Teacher A's top set, whose students all achieved 5a and 5b, 80% are on target for 3 levels of progress and to get a B at GCSE. Only a handful are on target for more than 3 levels of progress.

In Teacher B's lower set, whose students all achieved 3b or below, only 50% are on target for 3 levels of progress.

Imagine a performance review based on the 3 levels of progress. Teacher A would receive lots of praise for the 80% achieving the target and, under performance related pay, expect a rise. And it is clear that Ofsted would regard it as very wrong for Teacher B, with only half the class achieving the 3 levels, to get any rise at all.

The 3 levels of progress measure suggests Teacher A's class is performing well above national average and Teacher B's class are performing well below it. The reality is the opposite. Teacher A's class are progressing below the national average for students of their ability and are an example of this week's Ofsted criticism of too low expectations for high attainers. On the other hand Teacher B's class are performing well above the average for students of similar ability across the country.

Let's Abandon the 3 Levels Measure now.



Sir Michael Wilshaw is right to say that most students who achieve level 5 at age 11, and certainly those starting on 5b or 5a, should be expected to achieve a grade A or better at GCSE. So let's abandon the measure of 3 levels of progress, used by the DfE and in the Ofsted Data Dashboard, that sets an expectation of level 5 students achieving only a B. 3 levels of progress is too low a target for high attainers, while being a very tough target for less able students.

There have been many dubious measures that have come out of the DfE over the years but this is possibly the silliest.

 

Date Source



Key Stage 4 results, and transition tables: Taken from Raise Online library – with thanks to Heather Leatt

 

 

 

 
Share on Twitter

Comments


Add new comment

Already a member? Click here to log in before you comment. Or register with us.