Pro-selection report inaccurate and incomplete, say academics

Janet Downs's picture
 8

The HEPI paper claiming grammar schools play a ‘significant role in supporting social mobility’ was incomplete and inaccurate, write three academics* on the Bath university blog.

 The Department for Education data used by the HEPI paper’s author, Iain Mansfield, was ‘provisional’, the DfE pointed out.  It warned ‘Caution should be taken in drawing definitive conclusions.’

 The three academics said comparing pupil progress in selective areas with pupil progression in non-selective areas was ‘naïve’ because selective areas aren’t representative of the whole of England.  Such an ‘inappropriate statistical approach’ would ‘upwardly bias the grammar school effect.’    The report’s conclusions, therefore, were ‘unlikely to be robust.’

 Mansfield’s conclusion that the presence of grammar schools improved social mobility ‘is unlikely to hold true’, the academics wrote.  This is because Mansfield did not consider whether there was any ‘offsetting impact’ on disadvantaged pupils in selective areas who don’t attend grammars:

‘…the vast majority of disadvantaged students in selective areas do not attend grammar schools, and the evidence suggests that these students suffer an education and wage penalty compared to their equivalents in comprehensive areas,’

The academics stress the importance of ensuring ‘appropriate data and robust statistical methodologies are used before drawing conclusions and making policy recommendations’ when discussing the ‘merits or otherwise of selective versus comprehensive education’.

My comments:

The DfE urged caution when its provisional analysis was used.  The data also had several limitations, the DfE said. 

But Mansfield did not use caution.  And while the DfE data supports his claim that 45% of grammar school pupils in highly-selective areas are from below median income households, Mansfield did not compare this 45% figure with data for non-selective secondary schools.   67% of secondary pupils in non-selective schools are from household below median income irrespective of whether the schools are in highly-selective areas or not.  

The provisional data, used with such confidence by Mansfield, shows grammar schools still take far fewer 'disadvantaged' pupils than comprehensive schools.

We must also remember what I have stressed before.  Judging a school system on one narrow measure - the proportion of pupils entering elite universities - devalues the work done in schools to meet the needs of all pupils not just those headed for the rarefied setting of Oxbridge.

My thanks to Katy Simmons for sending me a link to the Bath blog

Share on Twitter Share on Facebook

Be notified by email of each new post.





Comments

Roger Titcombe's picture
Fri, 11/01/2019 - 11:48

Janet is joining a flood of academic comment rubbishing the HEPI report. This blog on the UCL IoE website repeats most of Janet's points and adds a few more, as do the comments.

https://ioelondonblog.wordpress.com/2019/01/10/grammar-schools-and-acces...

I make a different point here.

https://rogertitcombelearningmatters.wordpress.com/2016/08/19/bringing-b...

Academic selection at age 11 lowers our national IQ.

Every new grammar school creates at least three similar sized secondary moderns. How can these schools still meet the tougher GCSE grade performance thresholds imposed by the government?

Only by abandoning any serious attempt to provide a cognitively demanding, broad and balanced education, through developmental teaching methods. Such empowering education will be replaced by training, and the teaching methods of behaviourism will dominate. This already happens in comprehensive schools that have an intake cognitive ability profile skewed towards lower CATs scores.

These teaching methods do not result in cognitive development and will not make our school leavers cleverer and wiser, which is what is really needed.

Even when the marketisation and competition model is finally abandoned along with the ‘Tyranny of testing‘ required to drive it, secondary modern schools will find it more difficult to provide a full, broad and balanced curriculum for all pupils.

The first Principal of Mossbourne Academy and the former Chief Inspector of Schools, Sir Michael Wilshaw, was very clear about the superiority of all ability comprehensive schools compared to a mixture of grammar and secondary moderns. This is from a Guardian article of 14 December 2013.

“In comments that put him on a collision course with the then education secretary Michael Gove, who has expressed support for grammar schools. Wilshaw said: “Grammar schools are stuffed full of middle-class kids. A tiny percentage are on free school meals: 3%. That is a nonsense. What we have to do is make sure all schools do well in the areas in which they are located.”

But will the degradation and impoverishment of the education available to 11 plus failures be more than made up for by ‘grammar school excellence’ for the more able? The following questions are crucial.

Is the quality of teaching better in grammar schools than in comprehensives?

Do grammar schools support the learning of bright children from poorer homes better than comprehensives?

There is no evidence that either is the case. As the former Head of OfSTED, Sir Michael Wilshaw should know.


Janet Downs's picture
Fri, 11/01/2019 - 12:11

Roger - the UCL website contains the same blog as the one in Bath.  

It's good to know that academics have attacked Mansfield's report.  Mansfield still stands by the data - he left a comment under the Bath blog.  But, as I point out above, even if we accept that pupils from households under the median income level are 'disadvantaged' (which may not necessarily be true), the same data shows 67% of pupils in non-selective secondary schools are under this level.


John Mountford's picture
Fri, 11/01/2019 - 15:40

I first came across the HEPI report in the IOE blog. The co-authors substantially rubbished Iain Mansfield's use of 'dodgy' data in reaching its main findings for the reasons you cite above, Janet.

Interested by the apparent contradictory nature of the ‘evidence’ from research into the allegedly clear benefits of grammar school education carried out by HEPI and, as was claimed to be the case in their blog, the fact that two vital aspects of that research do not ‘stack-up’ in the face of detailed analysis, I took the opportunity to look up HEPI and to read the report for myself. As a reasonably intelligent and interested citizen, if I did not already have a clear view about the inappropriateness of the grammar school selective process and its mainly damaging outcomes, especially for late developers and the disadvantaged, I would be unable to make up my mind who to believe in this battle of the 'academic giants'. This is of no small consequence!

Obviously, very curious to dig a little deeper in order to try to settle this battle over this really important issue, I decided to write to both the IOE and to Sir Ivor Crewe, Master of University College, Oxford and Chair of HEPI’s Trustees, in an attempt to expose the evidence in question to greater scrutiny. It is my hope that both parties will agree to engage. So far no response from either party.

As background, this is what the HEPI website says is its raison d’etre – “The Higher Education Policy Institute was established in 2002 to shape the higher education policy debate through evidence. We are UK-wide, independent and non-partisan. We are funded by organisations and universities that wish to see a vibrant higher education debate as well as through our own events.”

Nick Clegg wrote an introduction to the HEPI report. In it you can read Nick Clegg’s views at,
https://www.hepi.ac.uk/2019/01/10/grammar-schools-progressive-rhetoric-c...

As will be seen from reading Nick Clegg’s own words, the argument to retain the grammar school system is based on the assumption that ‘with all other things being equal’, it works - ” without these state-funded selective schools and with all other things being equal, Oxbridge would have far fewer students with Black and Minority Ethnic backgrounds. In addition, other selective universities would have far fewer students from the bottom half of the income distribution.”

In response to his comments, as reflected in this line of reasoning, I pointed out to him this makes the assumption that in order to tackle the issue of inequality of opportunity, the only thing opponents of the present unfair system want to do is end selection. I noted that this would be just a first and essential step in trying to create a fairer society, with a focus well beyond the school system.

We know, there are enormous improvements to teaching and learning that will only find their way into classrooms once we rid ourselves of the excessive testing and examination culture that currently impoverishes the education of most young people. Grammar schools contribute to the collective consciousness that promotes aggressive competition as a key outcome of the educative process.

I have received no comment in response. It might seem that I am expecting too much of this former politician and the various academics I have shared my views with. Why should they engage with the likes of me?

There is a fundamental problem with a nil-response, if that is what happens. My assumption is that both the authors of the HEPI report and its critics, from across a broad spectrum, including Professor Beck Allen (follow up on her remarks via the link from the IOE reaction), all believe that what they have put before the public by way of the report and its comments, is accurate, dare I say factual. However, they cannot both be right. The idea that policy may be made on the back of research that is so fiercely contradicted by others with comparable 'qualifications', is worrying. It undermines the call to look to evidence-based research for direction.


Janet Downs's picture
Sat, 12/01/2019 - 09:55

John - I don't think the HEPI blog you linked to was written by Clegg.  I can't find the name of the author.  It may be Nick Hillman, HEPI's director, but that's only supposition. 


John Mountford's picture
Sat, 12/01/2019 - 10:24

Janet, thank you for prompting me to check the facts. You are right, the piece was not written by Nick Clegg. Clearly in responding to a report I believe to have been liberal with the facts, I should have been more accurate myself.


Janet Downs's picture
Sat, 12/01/2019 - 10:39

Easily done especially in this case when the author didn't give a name despite using the first person.


Janet Downs's picture
Sat, 12/01/2019 - 10:11

Terry Wrigley has written about the HEPI report on Reclaiming Schools.


John Mountford's picture
Sat, 12/01/2019 - 10:43

It's interesting that Professor Wrigley has made more of the number crunching and comes to the conclusion, "We can deduce from the data (page 53) that if grammar schools were expanded throughout England, as the Prime Minister wishes, perhaps another 18 pupils living in the poorest fifth of the country might gain a Cambridge place. That is hardly a convincing argument for wrecking comprehensive schools across three quarters of the land."
Having taken another look at the HEPI site I am baffled to read the 'Who We Are' list to discover how many 'heavy-hitters' they have on board. In my mind, accepting that Iain Mansfield has taken great liberties in compiling his report and making specific claims that run counter to the facts, as I believe them to be, why does an organisation with such an impressive list of apparent education experts support, as it clearly seems to, the publication of findings reliant on the use of flawed data?? It does nothing for strengthening their case for offering evidenced-based policy advice, presumably a key reason for HEPI to exist in the first place!


Add new comment

Already a member? Click here to log in before you comment. Or register with us.