Coalition reforms caused dramatic rise in achievement in global reading test, says DfE. Not so.

Janet Downs's picture

English 10 year-olds are now joint eighth in the world in reading as measured by the international PIRLS test.  

The Department for Education (DfE) says:

‘…this year’s results are the country’s best to date and a dramatic improvement on the 2006 results, when England was ranked 19th out of 45 countries.’

Dramatic improvement indeed.  Except the DfE has omitted to mention the 2012 results when English pupils leapt from 19th  in 2006 to joint 10th in 2011 when the tests were taken..  

What PIRLS 2016 actually shows is that England climbed from joint 10th  in 2011 to joint 8th in 2016.  An improvement, yes, but not such a dramatic improvement as the DfE claims.

Schools minister Nick Gibb was quick to claim responsibility:

Today’s results put the success of our increased emphasis on phonics and continued focus on raising education standards on a global scale.'

This was followed by an endorsement by Mark Lehain, Director of Parents and Teachers for Excellence and founder of Bedford Free School, said:

This adds to the already substantial base of evidence that systematic synthetic phonics are the most effective way of starting a child’s lifelong love of reading…’

Except that the ‘substantial base of evidence’ shows the systematic teaching of any method of phonics is an effective way of teaching reading.   And the EEF toolkit, regularly praised by ministers, found there was not enough evidence for the sole use of synthetic phonics.  

Lehain gushed on:

There is also a wider point about the reforms of the past seven years. A great deal of bravery was required to really shake up the system and implement them, but here is yet more evidence that this was the right thing to do.’

 But there is no evidence that reforms since 2010, however brave Lehain believes them to be, are responsible for the rise in ranking of English 10 year-olds in PIRLS 2016.

The great leap forward, remember, happened in 2011.  And instead of praising the results (published in 2012), ministers downplayed them at the time. 

Would it be true to say, then, that reforms put in place by Labour were actually responsible for the improved performance of English 10 year-olds in international reading tests?

The answer is ‘Possibly not’.  Correlation isn’t causation.  And teachers teach reading not politicians however knowledgeable they claim themselves to be.

What is known is that phonics was already embedded in the teaching of reading before Gibb pushed synthetic phonics and his phonics screening test.  

Omitting the PIRLS results for tests taken in 2011 to imply Coalition reforms are responsible for the improved performance of English ten year-olds in reading is a statistical sleight of hand.  Yet another example of how the DfE uses data to misrepresent and mislead.

CORRECTION 6 Decemeber 2017 08.25.   I have altered the above to make it clear that PIRLS results published in 2012 referred to tests taken in 2011.  The original article used 2012 throughout which didn't differentiate between when the results were published and when the tests were actually taken.

Share on Twitter Share on Facebook

Be notified by email of each new post.


Roger Titcombe's picture
Tue, 05/12/2017 - 14:37

While Janet makes some important points I must question the validity of all of these international comparisons. I discuss this here.

To explain the problem, consider a hypothetical example of a primary school that streams its pupils for all subjects including reading. This deplorable and discredited practice is becoming more common in Free Schools and Academies.

Consider that such a school enters all of its Y5 pupils for this PIRLS reading test. It would be safe to assume that the top stream pupils in this school would be more cognitively developed than those in the bottom stream and that they would therefore do better in the PIRLS tests. Clearly, for PIRLS rankings to be valid, the test cohorts have to be standardised for mean cognitive ability and they are not.

As for pupils in streamed groups within a school, so for pupils from different counties where mean cognitive abilities/IQs differ substantially. Sorry if the reader is uncomfortable with this, but it is true.

My article addresses this issue using a statistical approach that has been verified by a number of international expert statisticians since it was published nearly a year ago. Despite its being one of my more recent articles, it has received the second highest number of all-time viewings.

The effect of taking cognitive ability/IQ into account is dramatic. In the latest PISA maths tests this correction drops the UK from 27th to 49th place. The leading East Asian countries also suffer large demotions.

I note that in the PIRLS reading tests, Hong Kong and Singapore are (as usual) in the leading positions, so it would be safe to assume that these too would be substantially demoted if a cognitive ability/IQ standardisation is applied.

This is just another reason why DfE claims for the success of its education policies should treated with extreme caution.

Roger Titcombe's picture
Tue, 05/12/2017 - 17:14

Sorry - typo. I mean countries not counties.

Janet Downs's picture
Wed, 06/12/2017 - 08:11

Whether international test results are valid or not, politicians use the results to support or attack the education system in the UK (more likely England alone).  We saw how Gove misused the 2009 PISA test results to claim the UK had plummeted down league tables in ten years.  This, as we know, was not true as the claim was based on data known to be flawed.  Yet his claim underpinned his education reforms.  It's also a myth that survives even today.

Now we have Gibb claiming the latest PIRLS result vindicate his policy re phonics.  The Times front page today had a headline 'Best reading in schools for a generation'.  I can't read the whole article as it's behind a paywall but 'a generation' suggests ten years.  This takes us back to 2006 and would ignore 2012 when the leap in performance actually happened.

(If anyone has access to the whole Times article, I'd be grateful if you could summarise it in comments with particular reference to 2006.  Thanks.)

Janet Downs's picture
Wed, 06/12/2017 - 09:29

It's not often that my views and those of G Heller Sahlgren,  the  Research Director at Centre for Education Economics, agree. But he tweeted yesterday:

'England's PIRLS scores improved by 13 points between 2006 and 2011 and 7 points between 2011 and 2016. To draw the conclusion that the Conservatives' curriculum reforms are behind the latest improvement is bonkers.' (My emphasis)


Add new comment

Already a member? Click here to log in before you comment. Or register with us.