Department for Education “misunderstands/misuses evidence" says academic at ResearchED conference

Janet Downs's picture
First in a series of threads based on lectures at ResearchEd 2013. The one below summarises the first part of a presentation by Professor Coe, Durham University, on evidence-based education.

Problems with evidence:

1 There’s an evidence continuum: evidence based – evidence “garnished”

2 Evidence can be found which supports any position in education.

3 Ofsted’s approach in asking schools to collect evidence to show Pupil Premium (PP) spending is narrowing the gap is the wrong way round. Schools should start with a question, collect evidence and then answer the question – not start with the answer (PP spending reduces the gap) and find evidence to support the answer.

4 Research can be misunderstood – the Department for Education “misunderstands/misuses evidence (eg making much of small changes with small samples; citing relative gaps as percentage difference”.

Problems with research:

1 Quality of education research is variable – a lot isn’t very good.

2 Some is good but the difficulty is in knowing the difference.

3 It’s not easy to access academic research (eg in academic journals; behind paywalls).

4 Some academic debates are irrelevant: Prof Coe went as far as to say it is “(mostly) pointless”.

5 Peer review doesn’t work very well in education research.

Small positives:

1 Performance indicators for universities now require research to have an impact. It requires “public benefit”. (My question: how do researchers know what will have an impact when they start researching?)

2 Funding available for Education Endowment Foundation (EEF aka Sutton Trust) which provides a toolkit summarising cost/benefits of different educational interventions.

3 Increased interest in evidence-based education.

4 The internet and social media provides opportunities for “instant critique, debate, interaction”.

Key messages from the EEF

1 Some popular interventions aren’t as effective as thought. These include: setting by ability; after school clubs; performance-related pay; raising aspirations; smaller classes and teaching assistants. Professor Coe admitted these last two had caused more controversy than the others.

2 Some interventions looked “promising” eg effective feedback; meta-cognitive and self regulation strategies; Peer tutoring/peer-assisted learning strategies; homework.

BUT it wasn’t just a case of choosing the “best” interventions from the EEF toolkit and applying them in schools because:

1 There are problems with research evidence: it might be too thin; the studies “might not reflect real life”; context is important – what works in one situation mightn’t work in another.

2 There are problems with implementation: it’s not known how to ensure teachers put interventions into practice “in ways that are faithful, effective and sustainable”; and teachers might be implementing a strategy but not doing it correctly.

Professor Coe cited Assessment for Learning (AfL) which had been used for years but hadn’t been carried out properly. What was being done in its name was often tokenistic and was not proper formative assessment (ie assessment that informs practice). He recommended Old Andrew’s blog on AfL which summarised how a sensible idea became “a checklist to be mindlessly followed.”

UPDATE 17 September 2013

The original title of this post was Academic upsets Ofsted Chief Inspector in presentation on evidence-based education at ResearchED.


Professor Coe’s presentation can be viewed here and the powerpoint presentation can be downloaded here.

TES summary is here. Sir Michael Wilshaw, Ofsted chief inspector, told TES Professor Coe’s views on Ofsted are “tosh and nonsense”. He made his remarks in a TES interview which is advertised as being at However, I can’t find the interview. If anyone finds it please can you post a link in comments. Thank you.

Share on Twitter Share on Facebook

Be notified by email of each new post.

Add new comment

Already a member? Click here to log in before you comment. Or register with us.