Zombie PISA stats still being quoted – it’s going to take more than a ton of bricks to crush the use of discredited data.

Janet Downs's picture
 0
“Those results were embarrassing for Labour because they showed that our schoolchildren had slipped down the world rankings since 2000, falling from 8th to 27th in maths, 7th to 25th in reading and 4th to 16th in science.”

Toby Young, Daily Telegraph, 2 December 2013

No, it’s not a misprint.  That paragraph was written just over a week ago. 

It’s not as if Toby Young doesn’t know the OECD said PISA results for the UK in 2000 had been found faulty and shouldn’t be used for comparison.  It was in the second paragraph of Viewing the UK School System Through the Prism of PISA (2010).  Young must have read this document because he quoted from it at the time.  This is what OECD said:

“Trend comparisons, which are a feature of the PISA 2009 reporting are not reported here because for the United Kingdom it is only possible to compare 2006 and 2009 data. As the PISA 2000 and PISA 2003 samples for the United Kingdom did not meet the PISA response-rate standards, no trend comparisons are possible with these years.”

This warning was ignored by Michael Gove who prides himself on his “evidence-based” approach.  It was ignored by fellow ministers and Tory MPs.  It was ignored by most of the media.

But it wasn’t ignored by FullFact, this site or David Miliband who wrote to the UK Statistics Authority.

Using the faulty figures should have ended when the Statistics Watchdog censured the Department for Education (DfE) in 2012 for its misuse of the data. 

But apparently the views of the OECD and the Watchdog aren’t enough to kill these zombie stats.  No sooner are they beaten down than someone resurrects them.  Tory MP Chris Skidmore cited them in January; Lord Nash mentioned falling down international league tables in July; Tory MP Priti Patel wrote about how UK’s position in the tables slumped during Labour’s tenure in November and the CBI, who should know better, used the figures in Exhibit 3 in its 2013 Annual Report.

So, how will these zombie PISA stats eventually be staked through the heart?  Reasoned argument and links to evidence haven't worked.   Comments from the UK Statistics Authority haven't worked.   A reprise of the PISA Blues hasn't worked.

Toby Young once commented on this site that he wouldn’t dare mention the 2000 UK PISA figures here because I would be down on him like a ton of bricks.

The ton of bricks has not just fallen but has gone up and down like a steam hammer.  But still the data rises from the death bed.

So, perhaps we should start a Roll of Dishonour to record the names of anyone who still uses this discredited data.  There are five names on it already:

Chris Skidmore MP, Priti Patel MP, Schools Minister Lord Nash, the CBI and Toby Young.

If readers find any more examples, please let us know so their names can be engraved and garlanded with garlic.

UPDATE 11 December 2013

The OECD repeats its warning in the report published last week:

"The PISA 2000 and 2003 samples for the United Kingdom did not meet the PISA response-rate standards, so the observed higher performance in 2000 should not be used for comparisons."

It can't be clearer.  The 2000 results should NOT be used.
Share on Twitter

Add new comment

Already a member? Click here to log in before you comment. Or register with us.