Do you have the same depth of statistical analysis as schools ministers and writers of Department of Education (DfE) press releases? Find out by taking the following quiz.
You are warned not to compare international test data for the UK in 2009 with results from 2000 because the earlier results were found to be faulty after they’d been published. Would you:
A Ignore the warning, publicise the comparison as widely as possible and base your entire education policy on the premise that the UK is ‘plummeting’ down league tables?
B Refuse to publish the comparison?
C What warning? It's only a technicality.
DfE number crunchers say it’s not possible to compare Key Stage 2 writing tests with earlier years because the test was fundamentally changed in 2012. Would you:
A Prepare a press release and Tory party literature which compares the 2014 SAT results with 2009?
B Not make the comparison because it would be misleading?
C Who cares about technical footnotes from bean counters?
The Organsiation for Economic Development and Cooperation (OECD) warned the results of their Adult Skills Survey should be used with caution because of sampling difficulties. Would you:
A Use the poor showing of English and Northern Irish 16-24 year-olds in tests involving 23 countries to say that these young people were the most illiterate and innumerate in the developed world?
B Report the results accurately but ensure readers knew the number of countries involved in the survey and include the caveat?
C What sampling difficulties?
You find some surveys done for public relations purposes, such as those from Premier Inn to market hotels and UKTVGold to publicise TV programmes, which show English teenagers have a poor knowledge of selected historical facts. Do you:
A Write an article for the Daily Mail
saying survey after survey demonstrated ‘disturbing historical ignorance’ among young people?
B Laugh at the surveys which included such daft answers as the plague being a heavy metal band and wonder how anyone could be so gullible as to be taken in by such dodgy data?
C No-one will find out who did the surveys, will they?
The DfE had to admit in the High Court that achievement in academies was only ‘marginally higher’ than in non-academies. Would you:
A Continue to claim academies outperform ‘council’ schools?
B Accept the evidence that academies have little effect on results?
C The High Court case didn’t get much publicity, did it? This unfortunate admission can easily be buried by drip-feeding positive stories about academies, can’t it?
HOW DID YOU DO?
Mostly A’s: Your grasp of statistics matches the understanding shown by school ministers and those who churn out DfE press releases. Unfortunately, this is nothing to be proud of.
Mostly B’s: Congratulations – you have demonstrated an awareness of statistics far higher than the comprehension shown by school ministers and the DfE publicity machine.
Mostly C’s: Do you work as a DfE adviser?
INFORMATION on the stories
featured above is as follows:
of PISA 2000 figures.
2Gov’t ignoring DfE statisticians’ warning about not comparing 2014 Key stage 2 tests with those before 2012, see Private Eye
2 October 2014.
Adult Skills Survey.
given in High Court.
websites which investigate whether data is dodgy or not include Channel 4 Factcheck
and Full Fact
. Radio 4’s More or Less
looks at statistics in daily life.