A misleading implication and dodgy use of data – recent ministerial comments about education
According to the dictionary, ‘transformed’ means ‘to change (something) completely and usually in a good way.’
That’s what I thought Education Secretary Nicky Morgan meant when she told the Tory Conference about ‘‘1000 failing schools transformed under the leadership of strong sponsors.’
But I was wrong. Morgan's ‘transformed’ applies to ‘all schools that have become sponsored academies since May 2010’, the Department for Education says (see here, scroll down). Morgan was referring to all schools turned into sponsored academies not just those which had improved, the DfE claims. To hammer the message home, the DfE added, ‘The Secretary of State was not saying that 1000 schools had all received an improved Ofsted rating since becoming a sponsored academy.’
But improvement was what was being implied. It appears, however, Morgan was only referring to schools which changed their legal structure. That's probably because all 1000 sponsored academies haven't improved. 1000 haven't even been inspected yet. And 200 of those were academies set up under Labour (see here).
There’s increasing evidence that becoming a sponsored academy does not necessarily bring improvement. The opposite appears to be true. Henry Stewart’s analysis shows sponsored academies are more likely to stay Inadequate or become Inadequate. And no-one has yet responded to Henry’s challenge to show evidence about sponsored academies’ superior performance.
DODGY USE OF DATA
Does taking a week’s holiday during GCSE years reduce the chance of getting ‘good’ GCSEs by a quarter? That’s what schools minister Nick Gibb claimed on the Today programme. Radio 4’s More or Less investigated (listen here, from 15.40).
The programme found pupils with 100% attendance during these years passed at least 5 GCSEs A*-C including Maths and English. Those who had 5-6 days off for any reason had 8% worse results.
8% is not 25%. Where, then, did 25% come from?
The investigators found Gibb had chosen the figure which best suited his argument. He defined ‘good’ GCSEs as those in the English Baccalaureate. 43% of pupils who never missed a day got the EBacc. The pass rate dropped to 33% for those who missed a week for any reason. That’s 25% lower. The interviewer said:
‘If you squint at the numbers in the right light, you can get the 25% figure to stand up.’
However, the programme highlighted an important statistical rule: correlation isn’t causation. It’s wrong to assume going on holiday for a week causes lower GCSE results – the data couldn’t sustain this claim. Other factors could come into play such as pupils less likely to gain ‘good’ GCSEs being more likely to drift off.
More or Less concluded they would not give Nick Gibb an A* for his use of data.
Nicky Morgan implied by her use of a particular word that schools are improved by becoming sponsored academies. Nick Gibb used a ‘more obscure’ figure to underpin his argument that a week’s holiday in GCSE years reduces the chances of getting a ‘good’ GCSE by a quarter. Both demonstrate how words and statistics can be used to mislead.