The EEF casts new light on ‘The Attainment Gap’

Roger Titcombe's picture
 6

The Education Endowment Foundation (EEF) has published what is probably the most comprehensive study yet on ‘The Attainment Gap’, which has been the principal concern of ‘The Social Mobility Foundation‘, ‘The Sutton Trust’ and successive incarnations of the Department for Education and its Opposition shadows over the last three decades.

The EEF summarises ‘The Attainment Gap’ as follows.

“The gap in outcomes between those students from the least well-off backgrounds and their classmates is already evident by the time they begin school, aged 5. Over the next 11 years of full-time education, it worsens [to] grow from age 5 to 16:

  • there is a 4.3 month gap at the start of school between disadvantaged children and their classmates;
  • this more than doubles to 9.5 months by the end of primary school; and
  • then more than doubles again, to 19.3 months, by the end of secondary school.”

‘The Attainment Gap’, is based on DfE school performance data at KS1, KS2, and KS4. These data are primarily intended to drive the marketised education system in terms of OfSTED judgements and league tables. This makes them ‘high stakes’ measures for schools, which has implications for their validity.

There is a moral dimension of ‘The Attainment Gap’ arising from those students from the least well-off backgrounds performing worse than their more affluent classmates. This has led to the political right using the issue to condemn comprehensive schools in favour of academic selection and/or extreme regimes of instruction-based teaching backed up by coercive sanction-driven discipline systems.

The fundamental reason for the apparent ‘attainment gap’ is that pupils from poorer backgrounds tend to have lower cognitive abilities. That this fact is unrecognised, ignored or rejected for ideological reasons is why the ‘attainment gap’ has persisted despite all the Labour and Conservative government initiatives, ‘zero tolerance of failure’, head sackings, school closures, Academisations and Free School promotions of recent decades.

None have made any impression because they are remedies for something that does not exist, based on an incorrect diagnosis of the problem.

The key EEF findings are as follows. 

  1. The widest attainment gaps relate to FSM and SEN pupils, but not those that have English as a second language.

While it is often assumed that not having English as the first language must be a significant disadvantage for primary pupils, the facts do not support this. The EEF study is just the latest to come to this conclusion.

Cognitive Ability Test (CATs) data, however, do show a clear link with Socio-Economic Status (SES), for which eligibility for Free School Meals (FSM)  is a reliable proxy. This relationship between poverty and cognitive ability has been known to heads and governors of secondary schools for decades, which is why after the 1988 Education Reform Act compelled all schools to compete on the basis of the raw aggregated attainment of their pupils, those schools that had powers over their admissions policies (Academies and Religious Schools) were increasingly inclined to find ways of cutting down the numbers of FSM, and therefore less cognitively able, children they had to admit.

For many years the Cumbria LEA produced CATs/School Attainment data that showed this relationship, so enabling judgements to be made about the relative effectiveness of its schools that took account of the cognitive ability profiles of admission cohorts. When Labour took control of the county council it banned the production of these performance data. For the whole of its period of office, successive Labour governments regarded any mention of schools’ cognitive ability admissions profiles as being, ‘an unacceptable excuse for failure’. This has remained the view of subsequent Conservative-led governments leading directly to the continuation of ‘The Attainment Gap’ fallacy.

Academisation has taken away the ability of Local Authorities to impose economically and therefore cognitively balanced catchment areas and has brought about an admissions free for all. This has resulted in those schools located in affluent post codes (therefore admitting fewer less bright children on the basis of proximity) getting better aggregated exam results, climbing the local league tables, having more Y7 applications and therefore being able to deny entry to ever greater numbers of poorer pupils living further from the school, so increasing the mean intake cognitive ability, therefore getting better exam results etc. etc.

The converse has happened to schools located in the heart of poor communities. This admission inequality issue has been successfully addressed by the introduction of CATs driven fair banding, which while being widely adopted by Academies, has not been available to LA schools, except in the London Borough of Hackney. I investigated this in detail using real CATs data in my study of Mossbourne Academy and the Hackney Fair Banding admissions system. This appears as Part 4 of my book, ‘Learning Matters’.

That school students from less affluent postcodes achieve less well than their more affluent classmates is therefore explained by the associated differences in mean cognitive ability, which also accounts for the generally lower attainment of students with various categories of SEN.

  1. The gap persists in all types of secondary schools

Attainment 8 scores for all pupils is higher in ‘Outstanding’- or ’Good’-rated schools, than (on average) in schools rated as either ‘Requires improvement’ or ‘Inadequate’. However, the size of the Attainment 8 gap between disadvantaged and non-disadvantaged pupils is all but identical across all four Ofsted-rated categories of school. It is not, as might be expected, a problem that predominates in schools classified as under-performing: it is found to a similar degree in all types of schools. An EEF chart shows the GCSE outcomes (Attainment 8 scores) for disadvantaged pupils and non-disadvantaged pupils grouped according to their school’s overall effectiveness, as assessed by Ofsted.

‘Outstanding’ schools are no more effective than ‘Special Measures’ schools in closing ‘The Attainment Gap’.

The entire focus of government education policy has been directed onto increasing the numbers of pupils in ‘good’ or ‘outstanding’ schools. At successive Prime Ministers Questions, regardless of what Jeremy Corbyn actually asks, Theresa May never fails to mention the success of post 2010 Conservative governments in increasing these numbers.

Yet the EEF finds that this has had no effect at all on reducing ‘The Attainment Gap’. This is a devastating fact that calls into question the entire rationale of both government education policy and the relevance of OfSTED in improving the attainment of lower attaining students.

Schools have been recently receiving their Inspection Data Summary Reports (IDSAs) from OfSTED. This is the latest national framework for judging school performance and effectiveness. It has replaced ‘RAISEonline’ and before that the PANDAs that I was familiar with as a headteacher in the 1990s. The methodology is to attempt to judge the attainment and progress of pupils in KS1, KS2 and KS4 in each of four groups. The first three of these relate to Low, Middle and High attainers in the previous Key Stage (or in the Reception Year for KS1). But the fourth category is ‘Disadvantaged’ pupils, which is those eligible for FSM.

The OfSTED judgements for all schools will therefore depend on the progress made by those pupils in the school judged to be ‘disadvantaged’ by their eligibility for FSM. The IDSA goes further by identifying such pupils in ‘scatterplots’, the implication being that their individual progress or lack of progress will count for or against the school’s overall OfSTED judgement. However such pupils may well be performing in accordance with, or better than their CATs scores and still be judged to be ‘underperforming’ on the false assumption that they have average cognitive abilities when these are in fact lower. Without CATs data is impossible to determine any independent effect of eligibility for FSM on attainment.

However, if there is indeed no ‘attainment gap’, and all school students perform, on average, in accordance with their cognitive ability as determined by CATs, then a lower performance by the ‘disadvantaged’ group is exactly what should be expected. This will result in schools being penalised by OfSTED just for having greater proportions of FSM pupils in the school. Despite constant denials, this has always been the general pattern of OfSTED school judgements.

This is exactly what has happened. The ‘left establishment’ blogsite ‘Reclaiming Schools’ published this article on 30 January 2018.

“Schools in North East England are under attack again. According to Progress 8 scores, its schools are the least effective in the country, with the highest percentage coming ‘below the floor’. But Progress 8 is a flawed and misleading measure. It assumes that social factors make no difference. Once again, the Government are in denial about poverty and the economy. It’s so much easier to attack teachers again.”

This reaction is entirely predicable. The political left prefers to interpret ‘The Attainment Gap’ as a form of prejudicial class-based discrimination further confirming the malign outcomes of growing social and economic inequality. The ‘Reclaiming Schools’ schools article goes on as follows.

“Poverty has a big impact on pupils’ progress: on average, students on free school meals score -0.5 on Progress 8. (-0.5 is also the threshold for ‘below the floor’.) Schools with large numbers of FSM students are far more likely to score below. In the North East, 17% of students are FSM (13% nationally). In some places it’s worse:

  • 24% Middlesbrough
  • 23% Newcastle
  • 20% Sunderland
  • 19% South Tyneside
  • 19% Hartlepool.

Not surprisingly, all these areas have large numbers of schools ‘below the floor’.”

It is not ‘poverty’ that is the cause of the lower attainment, but the lower mean cognitive ability that prevails in these impoverished communities. I can see why this truth is difficult to accept for Labour politicians and the trade unions, but the truth it is. We know this from ‘GL Assessment’, the commercial marketer of the national Cognitive Ability Tests (CATs) formerly provided by NfER. P10 of this 2009/10 report provides the irrefutable data.

The 146,075 students without FSM obtained a mean combined score of 102 (55th percentile)

The 27,536 FSM students obtained obtained a mean combined score of 92 (30th percentile)

The purpose of the Report is given in its introduction.

GL Assessment provides updated tables and progress charts to enable schools to estimate
pupils’ GCSE subject grades based on their CAT scores. This report explains how to use these tables and charts. It also gives guidance on setting targets and discussing them with individual pupils. Information on how the estimates (or indicators) were developed, and on how to calculate estimates for groups, is included in the appendix. The results of a recent large scale study looking at the relationship between CAT scores and pupil/school factors is included at the end of the appendix [p10] for your information.

It could not be clearer. The attainment of FSM students would be predicted to be far below their more affluent classmates. See the charts for maths in Figure 1 of p2. These predicted that while 75% of non-FSM students were predicted to obtain a C+ grade, only 28% FSM students would be so predicted. Similar extreme differences can be found in other predictions of attainment.

Where the ‘Reclaiming Schools’ article is correct, is that the schools in the North East are indeed being unfairly targeted. On the basis of the GL Assessment predictions, the schools would appear to have obtained far better GCSE results for their FSM students than their CATs scores would have predicted.

I am the former head of an inner-urban school in Barrow-in-Furness, where the mean intake CATs score was 85, but the astonishing progress of our students was never recognised as unsurprisingly the aggregated GCSE results never met the ‘floor targets’ of the time. No head or Labour politician could ever argue locally that the reason lay with the low mean Cognitive Ability of the admission cohorts. The eventual ‘solution’ was the closure and demolition of all three mainland non-religious schools in the town and their replacement by an £multi-million Academy scheme that proved to be a massive failure from its opening, leading to hundreds of Barrow pupils now travelling by bus and train every day to attend the LA schools in the nearby towns of Dalton and Ulverston.

All of this is confirmed by a ‘Schools Week’ report that concluded that schools with lower FSM numbers were more likely to be judged ‘outstanding’ and/or to improve their OfSTED grades.

As the concerns  from both ends of the political spectrum play into their own deeply rooted belief systems, there is little political appetite in any of the main political Parties or the unions for questioning whether the  ‘Attainment Gap’ actually exists.

  1. Regional variations in ‘disadvantaged’ pupil attainment

The EEF reports as follows.

Looking at the attainment gap on a regional basis, it is the performance of pupils in London which stands out. [The EEF tables] show that a majority (51%) of London’s FSM-eligible pupils achieved A*-C grades in GCSE English and maths in 2016. In the neighbouring South East barely one-third of FSM-eligible pupils did so. London’s attainment gap was 19 percentage points; the South East’s was 34 percentage points. FSM-eligible pupils in London were 52% more likely to get 5 or more good GCSEs in 2015 than FSM-eligible pupils in other parts of the country.

The reasons for the transformational improvements in pupils’ outcomes in London in the past 15 years have been much debated. It is not possible to identify for sure why it happened – the causal mechanisms – as the reforms introduced were not robustly evaluated. Researchers have proposed a number of plausible explanations for what has been termed ‘the London effect’, notably: improvements at primary schools from the late 1990s; the London Challenge and other initiatives within secondary schools; and a significant influx of pupils from high-attaining immigrant families.

The last line is significant. While it is likely that ‘high attaining immigrant families’ are likely to be a factor, such high attainment matches the higher than national average mean cognitive ability scores of some ethnic groups that are strongly represented in London schools.

This is especially relevant to London, not just because of the high proportion of children from immigrant parents, but also because so many schools take the CATs tests provided by GL Assessment, enabling cognitive ability data to be taken into account by the schools.

The London data are therefore critical to my argument, because it is established that although many immigrant communities remain over-represented in all manner of negative social data, the children nevertheless perform at school according to their (sometimes higher) cognitive ability, not their (often lower) parental affluence.

This effect helps account for the success of so many of the Hackney secondary schools, including Mossbourne Academy, as described and explained in my book, ‘Learning Matters’. My research on matching CATs scores to the GCSE performance of individual students in my own Cumbria headship school showed that there was no significant link between eligibility for FSM and attainment for students with similar CATs scores.

Mossbourne Academy and other Hackney schools have in the past been praised by OfSTED for the high attainment of ‘disadvantaged’ pupils. However the ‘fair banding’ admissions system in effect ‘reserves’ spaces in the high CATs score admission bands for FSM students with high CATs scores. Such students perform well (in accordance with the CATs score) despite their FSM ‘disadvantage’. High FSM postcodes, while being characterised by lower mean CATs scores, still contain some pupils with high CATs scores and ‘fair banding’ admissions policies can find and accept them.

Regrettably the EEF has not included CATs data in its analyses, so it is unable to test my assertions. If it revisits its work, I am confident that it too will conclude that ‘The Attainment Gap’ is an illusion arising from the failure to recognise that FSM eligibility is, on average, not just a proxy for Social Economic Status (SES), but also cognitive ability.

The data needed should be readily available from the Hackney LA, other Multi Academy Trusts (MATs) that use ‘Fair Banding’ admission policies, as well as from GL Assessment.

  1. How can the ‘Attainment Gap’ be successfully addressed?

Clearly, it does not need to be reduced if it does not actually exist. However, the ‘EEF Toolkit’ does contain important evidence for improving the levels of understanding of all school students through effective teaching and learning approaches. The problem with this is that the DfE and many Academy and Free School MATs do not seem to take any notice of the EEF toolkit conclusions, preferring their own evidence-light marketisation ideology-based approaches.

The true pattern is that cognitively developmental approaches are the most effective. The educational developmental trajectory of school students is not fixed by their CATs scores or anything else. This is demonstrated from the five reports published by EEF in July 2017.

Two approaches that work are as follows.

Dialogic Teaching

“Dialogic Teaching aims to improve pupil engagement and attainment by improving the quality of classroom talk. Teachers are trained in strategies that enable pupils to reason, discuss, argue and explain rather than merely respond, in order to develop higher order thinking and articulacy. The programme uses video review, print materials and in-school mentoring to support teachers’ practice across English, maths and science lessons.

This trial found consistent, positive effects in English, science and maths for all children in Year 5, equivalent to about 2 months additional progress.”

This is consistent with other EEF trials focusing on cognitively challenging talk, such as ‘Philosophy for Children’, and ‘Thinking, Doing, Talking Science’. The consistent results across subjects and the lack of any subject specific content in the training suggest that the approach may improve children’s overall thinking and learning skills rather than their knowledge in a given topic.”

This EEF conclusion is important as it recognises that the successful approaches are not based on the learning of factual content but on stimulating and supporting the development of general cognitive ability. Put simply, the pupils made more progress because the teaching and learning methods used made them cleverer. It is important to note that pupil’s confidence and performance improved in all subjects, not just the ones directly relevant to what the ‘classroom talk’ was about. This is the claim of the long-standing ‘cognitive acceleration‘ movement led by Michael Shayer and the late Philip Adey, backed by a huge amount of peer reviewed research.

Thinking, Doing, Talking Science

“Thinking, Doing, Talking Science (TDTS) is a programme that aims to make science lessons in primary schools more practical, creative and challenging. Teachers are trained in a repertoire of strategies that aim to encourage pupils to use higher order thinking skills. For example, pupils are posed ‘Big Questions’, such as ‘How do you know that the earth is a sphere?’ that are used to stimulate discussion about scientific topics and the principles of scientific enquiry.”

Thinking, Doing, Talking Science appeared to have a positive impact on the attainment of pupils in science. Overall, Year 5 pupils in schools using the approach made approximately three additional months’ progress.”

This too was an initiative based not on ‘telling by the teacher and listening by the pupils’, but on the development of general cognitive ability through metacognition, pupil/pupil and pupil/teacher talk.

These are followed by three approaches that are not based on cognitive development and are much less effective .

The EEF and the Sutton Trust could have an important role in changing the ruling pedagogy of the English school system for the better, but they first need a rethink about the true nature and actual pedagogic implications of ‘The Attainment Gap’

I welcome comment and debate in response to this article.

Share on Twitter

Be notified by email of each new post.





Comments

agov's picture
Thu, 01/02/2018 - 12:51

"the EEF has not included CATs data in its analyses, so it is unable to test my assertions"

Doesn't that mean that until it does it remains entirely possible that the London effect is the effect that could happen anywhere that gets extra funding (aka London Challenge)?

At this point we only have claims (that happen to be currently politically convenient) that immigrants to London are high-attaining, after having had many years of claims that immigrant populations in London tended to have lower cognitive abilities.


Roger Titcombe's picture
Thu, 01/02/2018 - 14:56

I should have worded that better. There is no doubt from the GL Assessment data that on average, FSM pupils score significantly lower than non-FSM pupils. I did not need the GL Assessment data to know that. Having worked as a Cumbria head for many years and served on the head's panel that oversaw the CATs testing regime, that was very clear. The Cumbria LEA also made the CATs data for every school available to all the schools. The FSM profiles of the schools was/is publicly available so the pattern that I describe in my article is not only a clear fact, but one that was widely known by Cumbria heads. However, no Cumbria heads (including me) were keen to talk about it. The schools with the best exam results tended to to have the least FSM pupils and the highest mean CATs scores, but the schools wanted people to believe that their high position in the local league table was down to the special qualities of the school. In the case of the schools like mine with very high FSM numbers and low CATs scores, no head could tell the truth that the poorer exam results and lower league positions were explained by the poorer intake cognitive abity profile.

This pattern is shown in the chart included at the end of this article. ('Anyshire' is Cumbria)

https://rogertitcombelearningmatters.wordpress.com/2016/10/27/more-probl...

Every LA education officer and head in Hackney is also fully aware of this pattern, but OfSTED and the DfE pretend it doesn't exist, while the 'left establisment' doesn't want to believe it.

My reference to the EEF 'testing my assertions' is not an admission that they might be false, but a simple statement that EEF could not themselves come to this conclusion without using the key data, for which various KS1 and KS2 'prior attainment' is not a valid substitute, as I have explained many times.

Today's BBC News is once again pushing the same fallacy that average school attainment in the north of England is lower than in the south, because northern schools are poorer than those in the south, whereas the truth is that northern school intakes are on average characterised by lower mean cognitive ability than those in the generally more prosperous south.

As EEF pointed out, it is not clear which aspects of 'The London Challenge' account for the higher rate of cognitive development of its poorer pupils that enabled the London  'gap' to be less.

The 'ethic origin' argument is also unclear as you point out, because as the GL Assessment makes clear,  the mean CATs scores for different communities vary very considerably, with some being higher and others being lower. No one wants to talk about this either, with the 'left establishment' in denial and preferring to resort to 'discrimination' and prejudice to account for the lower attaining groups with no explanation at all for the high attaining Chinese.

I do not raise these issues to upset people, but because if IQ/cognitive ability is plastic as I assert (with evidence) throughout my blogsite, then schools should be focusing their teaching on exploiting this potential for development rather than chasing 'Outstanding School' and leage table glory, or in case of the high FSM schools, avoiding at all cost the 'floor target' trap door to closure and privatisation. 


Roger Titcombe's picture
Fri, 02/02/2018 - 13:03

Henry Stewart makes important points about the success of 'The London Challenge', here

http://23.254.179.97/2018/02/no-george-the-success-of-london-schools-was...


chris dunne's picture
Mon, 12/02/2018 - 15:48

Very interesting article, with which I mostly agree. Why wouldn't I, as the retired 21 year -long Headteacher of the inner London secondary school with the second highest FSM measurement in the country?

Two points of clarification:
(1) While I worked there from 1992 to 2013 Tower Hamlets also operated a system of allocating intakes to schools influenced by broad banding by cognitive ability. Many schools also chose to administer CATs tests at the start of Year 7.

(2) Professor Simon Burgess's study on the 'London effect' for Bristol university in 2014 provides definitive empirical evidence that the 'effect' is "entirely the result of ethnic composition". It goes on to make the salient point that London schools have on average two and a half times fewer the numbers of low-attaining students as schools elsewhere. I keep writing letters to the Evening Standard (and to be fair they keep printing them) every time they write leading articles saying that the league tables show how good London's teachers / schools are, with the implication of course that that those elsewhere are not.


Roger Titcombe's picture
Mon, 12/02/2018 - 16:24

Chris - Thank you for this and for drawing attention to Simon Burgess' Study, which was new to me. It can be found here.

https://www.bristol.ac.uk/media-library/sites/cmpo/migrated/documents/wp...

Burgess makes an important distinction between ethnicity and 'being a recent immigrant' in terms of positive contributions to the 'London Effect'.

It all adds to the evidence against the assumption that schools in the north of England MUST be failing because their aggregated attainment/progress is less (on average) compared to the relatively more prosperous south. There is no 'attainment gap' between FSM and non-FSM pupils when all the other complex but important factors are taken into account.

There is, however , still no sign of acceptance of this truth on the part of EEF, The Sutton Trust, The Social Mobility Foundation, the DfE or the NUT.


chris dunne's picture
Mon, 12/02/2018 - 17:12

I agree wholeheartedly about the unwillingness to accept the truth about 'attainment' Roger. It bedevilled me all of my life as a Headteacher. So much easier to fixate on 'Super Heads' or 'Academisation' et. al. than to look long and hard at what the empirical data tells you is really happening. And by doing so miss the opportunity to do more of the things that actually work!

Simon Burgess sums it up very well in the last three paragraphs of the report, just after pointing out that the "near unanimity" of agreement about the positive effects on results of the London Challenge has yet to have found any reliable basis in hard evidence. He says

"It sounds somehow uninspiring and disappointing that the London attainment premium is largely accounted for by demographic composition” rather than wholly caused by some innovative policy. I disagree. It can be seen as a story of aspiration and ambition. There is nothing inherently different about the ability and performance of pupils from different ethnic backgrounds. But the children of immigrants typically have high aspirations and ambitions, and place greater hopes in the education system than the locals do.

There are two important policy lessons here, albeit less straightforward ones than perhaps policy- makers might have hoped. First, integrated multi-ethnic school systems can be very productive, allowing the ethnic minority pupils to achieve the grades they seek, and (potentially) raising the scores of white British pupils as well. Here is a role for school leadership, in managing multi-ethnic school systems – it could have gone less well. Second, in parts of England where there simply isn’t a large community of recent immigrants, a focus on how to encourage pupils’ engagement with school, hard work and aspiration may pay strong dividends.

London has a right to be pleased with itself in terms of the excellent GCSE performance of its pupils. The argument here is that the basis for that success lies more with pupils and parents than it does with policy-makers."


Add new comment

Already a member? Click here to log in before you comment. Or register with us.