Can you handle stats as well as schools ministers and the DfE? Find out by taking the LSN stats quiz.

Janet Downs's picture
 4
Do you have the same depth of statistical analysis as schools ministers and writers of Department of Education (DfE) press releases? Find out by taking the following quiz.

You are warned not to compare international test data for the UK in 2009 with results from 2000 because the earlier results were found to be faulty after they’d been published. Would you:

A Ignore the warning, publicise the comparison as widely as possible and base your entire education policy on the premise that the UK is ‘plummeting’ down league tables?
B Refuse to publish the comparison?
C What warning? It's only a technicality.

DfE number crunchers say it’s not possible to compare Key Stage 2 writing tests with earlier years because the test was fundamentally changed in 2012. Would you:

A Prepare a press release and Tory party literature which compares the 2014 SAT results with 2009?
B Not make the comparison because it would be misleading?
C Who cares about technical footnotes from bean counters?

The Organsiation for Economic Development and Cooperation (OECD) warned the results of their Adult Skills Survey should be used with caution because of sampling difficulties. Would you:

A Use the poor showing of English and Northern Irish 16-24 year-olds in tests involving 23 countries to say that these young people were the most illiterate and innumerate in the developed world?
B Report the results accurately but ensure readers knew the number of countries involved in the survey and include the caveat?
C What sampling difficulties?

You find some surveys done for public relations purposes, such as those from Premier Inn to market hotels and UKTVGold to publicise TV programmes, which show English teenagers have a poor knowledge of selected historical facts. Do you:

A Write an article for the Daily Mail saying survey after survey demonstrated ‘disturbing historical ignorance’ among young people?
B Laugh at the surveys which included such daft answers as the plague being a heavy metal band and wonder how anyone could be so gullible as to be taken in by such dodgy data?
C No-one will find out who did the surveys, will they?

The DfE had to admit in the High Court that achievement in academies was only ‘marginally higher’ than in non-academies. Would you:

A Continue to claim academies outperform ‘council’ schools?
B Accept the evidence that academies have little effect on results?
C The High Court case didn’t get much publicity, did it? This unfortunate admission can easily be buried by drip-feeding positive stories about academies, can’t it?

HOW DID YOU DO?

Mostly A’s: Your grasp of statistics matches the understanding shown by school ministers and those who churn out DfE press releases. Unfortunately, this is nothing to be proud of.
Mostly B’s: Congratulations – you have demonstrated an awareness of statistics far higher than the comprehension shown by school ministers and the DfE publicity machine.
Mostly C’s: Do you work as a DfE adviser?

INFORMATION on the stories featured above is as follows:

1Misuse of PISA 2000 figures.
2Gov’t ignoring DfE statisticians’ warning about not comparing 2014 Key stage 2 tests with those before 2012, see Private Eye 2 October 2014.
3OECD Adult Skills Survey.
4Dodgy surveys.
5DfE evidence given in High Court.

FACT CHECKING websites which investigate whether data is dodgy or not include Channel 4 Factcheck and Full Fact. Radio 4’s More or Less looks at statistics in daily life.
Share on Twitter

Comments

Robert Waring's picture
Mon, 06/10/2014 - 09:31

If I fail the test will it put me in good standing to be the next SoS for Education?


Janet Downs's picture
Mon, 06/10/2014 - 11:09

Robert - If you scored Mostly A's then you are qualified to become a SoS for Education. However, I suggest we raise the bar to avoid any accusations of dumbing down. If you scored Mostly C's then you could become a Special Adviser - a SPAD ie Statistics Propaganda Advice Doctor.

If you scored Mostly B's then it's unlikely you'll go far in politics or in certain sections of the media.

Brian's picture
Thu, 19/03/2015 - 15:57

Good to see that the DFE is up to date with its use of statistics and information to schools. Heads and governors have been notified of the DfE website which reports the latest list of schools where the pupil premium is used particularly effectively. It suggests that other schools might wish to contact these schools for support and advice.

One of those schools was visited by Ofsted during Autumn Term 2014. The report says:

'There were too few disadvantaged pupils in Year 6 in 2014 to compare their attainment with that of others. However, the progress of disadvantaged pupils through the school is similar to that of their classmates.'

The progress of their classmates is 'inconsistent' and overall requires improvement. The overall outcome of the inspection was 'Requires Improvement.'

The DfE criteria for inclusion in the 'Well done ' list for use of the pupil premium include consistently good achievement for disadvantaged pupils and Good or Outstanding in their most recent Ofsted inspection.

Thanks DfE, pleased to see you are applying your own criteria so rigorously.

Janet Downs's picture
Thu, 19/03/2015 - 16:21

Brian - this shows the danger of relying on data which comes from small samples. This is true when LAs with just a small number of secondary schools are slated because, say, 50% of them are 'less than good'. But if there's only a small number, then one more 'Requires Improvement' or one more 'Good' can disproportionately skew results.

This also affects, as you've pointed out, schools with a small number of FSM pupils or rural schools with a tiny number of pupils sitting tests. I know of one village school damned by Ofsted on the strength of its KS2 results when just 12 pupils sat the tests. These 12 included four designated SEN or who on School Action Plus.

Add new comment

Already a member? Click here to log in before you comment. Or register with us.