Stories + Views
DfE digs up more “survey’s” – but do they support Gove’s statement that teenagers have “disturbing historical ignorance”?
Today’s young people are uninformed about history – that’s what Education Secretary, Michael Gove, says. In a Mail article, he cited “survey after survey” which displayed “disturbing historical ignorance” among teenagers. But the Department of Education (DfE) could only find one survey when asked*. That survey, by TV Gold, turned out to be targeting all ages not just teenagers. And the survey’s questions contained inaccuracies.
Gove’s “survey after survey” implies more than one. So the DfE has dug deeper. What “survey’s” (sic) did it find?
First up is Lord Ashcroft’s poll (2012) which found a lack of factual knowledge about WW2 among 11-17 year-olds but showed that young people appreciated the human cost of war.
So far, then, one survey seems to support Gove. But what about the rest?
The DfE provided no link to its next citation: A survey of 2000 11-16 year-olds undertaken by Premier Inn. An internet search found that three newspapers, the Mail, Independent and Metro, had reproduced “shock-horror” articles citing the survey. The articles were remarkably similar which suggests the papers churned a press release without considering the reliability of the research. Premier Inn’s Facebook page (scroll down to 22 March) claimed the survey showed British teenagers were “clueless” about history and urged parents to explore historic sites. But the ridiculous answers suggest that a large number of teenagers were having a joke at Premier Inn’s expense. These included Delia Smith being one of Henry VIII’s wives and the plague being a heavy metal band.
Next up was a survey by Professor Robert Tombs for the think tank Politeia. Again, there was no link. An internet search found a Politeia study co-authored by Professor Tombs but nothing about a survey. The study could, of course, have contained a survey but no details were given in the press release and the study itself is not freely available.
An article in London Mums Magazine was the next source. This was based on research by an exam revision service.
The final survey was, again, not a survey but a 2010 Telegraph article about a poll done by Sea Cadets to discover the extent of knowledge about England’s maritime past.
So, the DfE “survey’s” included only one, Lord Ashcroft’s, which could be described as a properly-conducted poll. One was a marketing exercise taken at face value by gullible journalists. One wasn’t a survey and the final two were articles about surveys. An internet search found no details of either so we don’t know whether there was a representative sample, what the questions were or how the survey was conducted.
Gove says he is in favour of “evidence-based” policies. However, it’s unclear what value there is in “evidence” which includes surveys of dubious reliability – unless, of course, they’re chosen simply because they support Gove’s point-of-view.
*Freedom of Information request via WhatDoTheyKnow.com. Download here.
UPDATE 13 May 2013
The original post contained a typo: WhatDoTheyKnolw.Com. This should have read WhatDoTheyKnow.com. It has been corrected. My thanks to Simon for pointing out the error.
UPDATE 14 May 2013
1 Nick, commenting below on 13 May (1.10pm and 2.08pm) told us that OnePoll did the Premier Inn survey. OnePoll provides a panel which comprises different demographics (eg teenagers, although parental consent is required for those under 18). The surveys are posted on OnePoll’s website and panellists can choose which questionnaires to complete. They are paid for each completed questionnaire. OnePoll does not write the questions – that is done by the group commissioning the survey. The New Statesman has discovered that OnePoll also did the Sea Cadets and UK TV Gold surveys.
2 Kathy Bramley has left an annotation at the foot of the FoI response which says the survey cited on the London Mums website had not been found on the Education Quizzes site but had been sent in a press release pack to London Mums.
UPDATE 24 May 2013
Radio 4’s More or Less discussed the surveys on 17 May. It confirmed they were “non-rigorous polls … with the aim of turning the results into press releases designed to create publicity”. The programme’s presenter, Tim Harford, spoke to Michael Marshall (Merseyside Skeptics Society, BadPR) who explained how the PR survey industry operated:
1 These lightweight surveys are commissioned by companies to generate media stories.
2 The firms doing the surveys post them online for registered users to complete.
3 Registered users are paid for each completed survey.
4 There’s no monitoring of who is completing the surveys.
5 Those who answer the questions don’t necessarily care too much about accuracy.
The clip can be heard here at 11.36 mins.