"Making Expected progress": A Deeply Flawed Measure

Henry Stewart's picture
 0
Note: See How to use data badly: levels of progress for an updated version of this, including 2014 GCSE figures

The DfE judges a secondary school to be "under-performing", and liable for direct intervention, if less than 40% of students achieve 5 GCSEs including English and Maths and if children are failing to make the expected level of progress in English and Maths.

The use expected levels of progress sounds reasonable and I have never seen it questioned. Then idea is to balance the absolute 40% rule and ensure that, if schools are making good progress from a low start, they are not deemed under-performing. The problem is that it is a deeply flawed measure and does not perform that purpose. There are two problems with it:

1) How Expected Progress Depends on Prior Attainment



The DfE has decreed that all students are expected to make three levels of progress from KS2 (age 11) to KS4 (age 16). However it is much harder to get a student from a 3B to a GCSE D grade (3 levels) than it is to get a 5B student to a B grade (also 3 levels). Overall 69% of students in English and 70% of students in Maths made the expected 3 level so progress in 2012. But the success differs hugely according to the starting point:



Across England 81% of "high prior attainment" students (those on level 5 at age 11) make 3 levels of progress in Maths, but only 33% of "low prior attainment" students (those on level 3 or below at age 11) do so.

High expectations are a good thing. But this measure gives a low expectation for the level 5s while providing a very tough one for level 3s. The effect is that whether a school  is deemed to be making sufficient progress with its students is much more likely to be defined by the intake than by the value it adds. Rather than balancing the absolute 40% measure, it reinforces any bias due to the low level of achievement at age 11.

2) The Random Effect



Students graded 4C, 4B and 4A are all expected to get a C grade. However this represents a full 3 levels of progress for those on 4C but only 2.33 for those on 4A.


  • As a result, in English, 84% of those on 4A at age 11 make the expected progress but only 49% of those on 4C.

  • 68% of students on 3A make the expected progress in English, but only  37% of those on 3C



This means there is a somewhat random element in the measure, depending on how many of a school's students are at the top of their level and how many near the bottom.

The Alternative



Schools are encouraged to use the "expected levels of progress" measure to set targets for their students but in fact it makes little sense to do so. In particular it does not provide sufficient challenge to those students arriving with a 5A or 5B. When it was briefly used in our school, parents rightly asked why their child's target was only a B when clearly they were capable of more. The alternative is to take the national tables on student progression and ensure each student's target is at least what is being achieved by 50% of students on that grade, or - for a target of outstanding progress - in the top 15% or 20%.

That means a student with a 5A in English should have a target of an A* (as 52% of 5A students achieve that nationally) and a 5B student should have an A target (as 60% of 5B students achieve that). A target of B for a 5C student may be reasonable as only 29% of 5C students achieve an A.

The three levels of progress target is simple and easy to understand. However it is deeply flawed and its simplicity means it is no use as a measure across all student abilities. It also does mean that the DfE's new proposals to judge schools on value added, not expected levels of progress, is a step forward.

Advice for Schools Below the Floor



If you are told that you are below the floor and your students are not making sufficient progress, be prepared to challenge the analysis. Work out how your 'low', 'medium' and 'high' prior attainment students do compared to other schools. The full table of school-by-school data for this can be downloaded from here.

 

Data Note:



Figures on progression by 'low', 'medium' and 'high' students are taken from the detailed school-by-school 2012 GCSE results.

 

Share on Twitter

Add new comment

Already a member? Click here to log in before you comment. Or register with us.