“Robotically following the results of one trial…can be a very dangerous thing,” says author of "Bad Science"

Janet Downs's picture
“Robotically following the results of one trial can be just as foolish as ignoring the evidence and a little bit of knowledge can be a very dangerous thing,” said Dr Ben Goldacre, author of Bad Science and Bad Pharma, on the Radio 4 programme Bad Evidence.

The programme voiced concern that vast databases of evidence from randomised control trials (RCT) could lead to proposed solutions being picked off-the-shelf and implemented without considering whether such solutions were ethical, appropriate or socially acceptable. Professor Nancy Cartwright, of the University of Durham and co-author of Evidence Based Policy: A Practical Guide to Doing it Better, was worried that using evidence in this simplistic way could result in over-confident faith in particular strategies simply because they had worked in one trial. This excessive optimism overrode caution. Policies needed to be rolled out slowly and monitored carefully.

Dr Goldacre said that RCT results needed to be interpreted by people who were aware of their strengths and weaknesses. He asked Professor Martyn Hammersley, of the Open University, to give an example of when the results of a randomised control trial (RCT) had been “blindly over-interpreted” in a “naïve and arrogant fashion and implemented in a way that was harmful”.

Teaching reading and the reliance on phonics was the answer.

Professor Hammersley explained that there was useful evidence about the effectiveness of different ways of teaching reading but the Government had taken these as proof that particular phonics-based methods are the best ways of teaching reading and implemented these as policy. This was dangerous as looking at one RCT could lead to a “quick fix” approach which avoided looking at a range of different evidence. It was important to differentiate between evidence and “hunch” or “intuition”.

Dr Goldacre argued that the way forward is to make evidence better and more informative. The programme described one project, Literacy Catch Up, by the Education Endowment Fund (EEF), which was designed to trial and evaluate different ways of improving literacy at the primary/secondary transition stage. The EEF projects comprise four main categories: comprehension, decoding, reading for pleasure and writing. The project will also look at mixed approaches.

The EEF research will look at more than one approach to teaching reading including mixed methods. Yet the Government has already decided in a “naïve and arrogant fashion” to endorse phonics as “the best way to teach young children to read”. The EEF research may show that the Government’s hunch was correct or it may confirm the findings of a recent survey which showed that primary teachers “emphasised the importance of using a range of strategies to teach children to read, and not just synthetic phonics, because children learn in different ways.”

Bad Evidence was broadcast on 1 January 2013. At the time of writing there are 6 days left to listen. The section featuring Professors Cartwright and Hammersley begins 21.55 minutes into the programme.

Share on Twitter Share on Facebook

Be notified by email of each new post.

Add new comment

Already a member? Click here to log in before you comment. Or register with us.