School intake governs academic achievement, says IFS report

Janet Downs's picture
 10
League table information based on the percentage of Year 11 pupils who gained five or more A* to C GCSE is “frequently misinterpreted as a measure of the quality or educational effectiveness of schools. Such an interpretation is invalid as no recognition is made for intake differences between schools in pupils’ academic abilities: the highest-scoring schools are largely those that already had the highest-achieving pupils when they entered secondary education.” So says a report from the Institute of Fiscal Studies (IFS) about Contextual Value Added (CVA) scores which factors in such things as socio-economic disadvantage. The IFS hopes that its report will encourage debate about the presentation of CVA scores by the Government and the media, and about how to increase public understanding of the uncertainty surrounding school league tables.

It’s a pity that the report will not be heeded because the Government decided to scrap CVA while the IFS report was being compiled.

A further IFS report listed problems with league tables:

1 They only focus on academic achievement measured in a limited manner.
2 Schools are held “accountable for outcomes over which they have little control”.
3 Because there is little “statistically rigorous evaluations of specific interventions”, schools face a deluge of “conflicting and poorly-grounded advice” about which practices to adopt.
4 Data used for accountability purposes is not the same as that required by parents when making choices about schools.
5 Schools may be “differentially effective for different types of student”. This factor has not been taken into account sufficiently.
6 Teacher performance is the most important factor in pupil achievement but the tables focus on school performance.
7 Non-cognitive skills (eg attitudes and behaviours) risk being neglected when the focus is on “a narrow academic measure of performance”.
8 League tables encourage “undesirable behaviours” by schools which could have a negative impact on pupils.

The Government should take heed of this report. However, similar warnings from the OECD about league tables and the undue emphasis on raw test results have been ignored. The IFS and the OECD are not telling the Government what it wants to hear so their research will be brushed aside.
Share on Twitter Share on Facebook

Be notified by email of each new post.





Comments

Melissa Benn's picture
Sat, 20/08/2011 - 10:09

Yes I saw this report mentioned this morning. It feels like we have come full circle, as this is what many of us have been saying for a long time - but some of the ' new schools' rhetoric has muted public knowledge of this single fact. Of course, as you note, good teaching can make a difference, but schools enjoy massive advantages from selected or advantageous intakes, which is why government admissions policies, allowing academies and free schools greater freedoms in this regard, will probably emerge as more significant than most of their other policies. Thanks for alerting us to this Janet.


Ian Taylor's picture
Sat, 20/08/2011 - 13:23

I am sure that Michael Gove could understand the argument about “Contextual Value Added” data and how it fairly compares pupil progress. If he wanted to.

However this is politics.

We constantly compare private schools with fees of £30,000 per annum to state schools with costs of £5,000 per annum, and ask why the state school is not able to give the same outcome as the private school. Even when the private school selects the top 5% of academic youngsters, who also have family wealth and advantage beyond the comprehension of most people.

The fact of the matter is that the people with wealth do not want their children mixing with the poor. Mr Gove knows this, but his policies dress this up in a different way which the middle classes can vote for.

Mr Gove wants a system which allows parents to be able to choose where their children will be, with others of similar wealth and values. If you cannot afford to pay for this then Mr Gove will give you a Free School where this happens. This is a vote winner, even though it is a philosophy of greed.

So all the arguments about fairness on this site are of no importance. Current education policy is about maintaining and encouraging privilege and class. Mr Gove is happy to allow us our fury over unfairness. It is not the poor that fund the government parties. Although repugnant to me, it is no surprise that governing elites want to maintain their position.

Alan's picture
Sat, 20/08/2011 - 13:36

A really useful report, thank you, Janet.

Regarding context, do you know of a study that has factored-in impact to secondary modern schools operating in selective counties?

I agree with Melissa's observation on selective schools having massive advantages, and can see why fair admissions are so important. But also, CVA has potential to reduce the 'selective effect size' if all factors aren’t included.

Statistical significance can be quite abstract if environmental interactions aren't included, don't you think?

Janet Downs's picture
Sat, 20/08/2011 - 15:21

I'm afraid I can't answer your question, Alan. The IFS published several reports and the second paper in my thread attempted to summarise the content of them all. I've only read the CVA report and the summary. Since the IFS published the report, the Government has decided to scrap CVA as I mentioned above. This is despite the IFS paper suggesting a different approach to calculating CVA which would make it easier to make comparisons about school effectiveness, and the OECD saying CVA was a step in the right direction.

To make the situtation worse, the Government is planning to judge schools on how many pupils they send to Oxbridge. This measure will again discriminate unfairly against secondary modern schools that still exist in Lincolnshire, Kent and other places where there are grammar schools. And how this measure is supposed to judge schools which have no sixth forms (again, like many in Lincolnshire) is unclear.

The Sutton Trust has branded the move as "extreme" and "unfair", and the deputy general secretary at the Association of School and College Leaders says the measure will be misleading as it will largely reflect the socio-economic background of the pupils in a school.

http://www.tes.co.uk/article.aspx?storycode=6109263

A further problem with this measure if it is implemented is that schools will have an incentive to push pupils to apply for Oxbridge whether or not it is in their best interest.

Janet Downs's picture
Sat, 20/08/2011 - 15:59

A Danish report published by OECD in 2005 looked at evidence from the international PISA, TIMSS and PIRLS surveys”. Its conclusions included:

“Dividing Students into Tracks increases Disparity, not Average Performance…

“Institutional Differentiation Means that Socio-Economic Background Matters More”

“Increased Reliance on Private Schools should be Approached Cautiously”

“Competitiveness in Educational Systems is connected with Personal Costs”

“Pre-Schooling is probably Beneficial for Later Academic Achievement”

“Differentiated Systems: Potentials in Better Teacher Support for Weak Students…
The majority of the countries with a particularly high degree of institutional differentiation in their educational systems are also countries in which students in PISA 2003 report a comparatively low level of individual support from their teachers.”

“Increased Instruction and Homework Time is not a Simple Solution”

“Education Systems Can Compensate for Different Socio-Economic Backgrounds” although it still "matters significantly for students’ academic performances."

“Certain Types of Student Assessments may facilitate student performance… there is
a tendency for schools in which teacher-developed tests are applied more frequently
to perform better, at least in some countries” (NB “teacher-developed tests”). Recommendation: “Policy makers and authorities should distinguish between testing for accountability purposes and testing as a part of the learning process.”

http://www.oecd.org/dataoecd/5/45/35920726.pdf

This report was published in 2005 and was based on evidence from PISA, TIMSS and PIRLS – a deluge of data. OECD documents for 2010 say many of the same things. Mr Gove was Shadow Secretary of State for Children, Schools and Families from June 2007 to May 2010. He had ample time to study the above document even though it is 218 pages long. He says he wants his policies to be evidence based but ignores evidence that has been around for years.

Alan's picture
Sat, 20/08/2011 - 21:24

A useful yet gloomy analysis of the changing education paradigm. There doesn’t appear to be a silver lining or ethnographic studies on the selective effect to secondary modern schools. The Lincolnshire Wolds hold back more than the wind.

I will read your linked documents more thoroughly, but it would seem that the stage is now set. Perhaps there is still potential for community hubs and extended school to bridge the gap.

Byron's picture
Mon, 22/08/2011 - 16:51

You pretty much said what i could not effectively communicate. +1

My site:
simulation rachat credits Rachat de Credit

Jack Hodges's picture
Fri, 30/12/2011 - 13:52

League tables aren't always the best way to represent how well a school is performing. There are many factors that should first be considered before placing them into tables. If Janet is right by saying schools can be judged simply by how many students they send to prestigious universities, this is deeply worrying.

http://www.schoolstickers.co.uk

Roger Titcombe's picture
Thu, 06/12/2012 - 10:47

This is really important as it invalidates the whole basis of the 1988 Baker Education Act, league tables and the rationale for competion between schools. The fact is that bright children tend to get high grades in exams, as they should. League tables reflect school intakes and vilify schools that take in lower proportions of brighter children when such schools may be more effective in achieving progress in individual children. We are very confused about ability and attainment, with the former being wrongly smeared with eugenic notions of 'ability conferred at birth'. Ability and attainment can BOTH be developed by good education (see the work of Philip Adey and Michael Shayer) and my work suggesting that ability can be depressed by bad education, at the same time as apparent attainment is inflated, and this is happening in the English education system as a result of league tables. As long as the left has an ability' hang up, it will not be able to confront the aruments of the right for a competive free market education system. What an irony that it takes the IFS to make the arguments that Labour should have made back in 1998, and what a tragedy that Blair and Blunkett built on the 1988 Act to lay the foundations for the current Gove destruction of the comprehensive education system.


Roger Titcombe's picture
Thu, 06/12/2012 - 10:51

This is really important as it invalidates the whole basis of the 1988 Baker Education Act, league tables and the rationale for competion between schools. The fact is that bright children tend to get high grades in exams, as they should. League tables reflect school intakes and vilify schools that take in lower proportions of brighter children when such schools may be more effective in achieving progress in individual children. We are very confused about ability and attainment, with the former being wrongly smeared with eugenic notions of ‘ability conferred at birth’. Ability and attainment can BOTH be developed by good education (see the work of Philip Adey and Michael Shayer) and my work suggesting that ability can be depressed by bad education, at the same time as apparent attainment is inflated, and this is happening in the English education system as a result of league tables. As long as the left has an 'ability’ hang up, it will not be able to confront the aruments of the right for a competitive free market education system. What an irony that it takes the IFS to make the arguments that Labour should have made back in 1988, and what a tragedy that Blair and Blunkett built on the 1988 Act to lay the foundations for the current Gove destruction of the comprehensive education system.
(typos corrected in previous post - sorry)

Add new comment

Already a member? Click here to log in before you comment. Or register with us.