When the January 2014 school performance tables were published, they were directly linked to a press release
containing a Ministerial statement by the then education secretary, Michael Gove, praising Government education policies. But the text containing the link didn’t make this clear thus giving an illusion of impartiality. This could ‘undermine confidence’ in the independence of school-level exam data if they’re linked to such press releases, UK Statistics Authority* said.
The headline of the press release put a positive spin on analysis done by the Department for Education (DfE) which purported to show a quarter-of-a-million fewer children were taught in ‘underperforming’ secondary schools than in 2010 thanks to measures the Government had put in place. But the analysis appeared to predict GCSE performance ‘retrospectively’ using two floor standard thresholds, the watchdog said, and wanted to know how the DfE came up with its figures.
The watchdog asked the DfE to explain the methodology behind its analysis but the department didn’t do so. In any case, the watchdog concluded, the stats didn’t meet ‘accepted professional standards’
. This was because:
1When schools become academies the results from predecessor schools may not be available for all four years. The DfE didn’t explain how these schools were treated.
2The analysis didn’t appear to take into account the uncertainty of results caused by the changing characteristics of cohorts within a school year by year.
3The analysis included the total number of pupils in each school and ignored the fact that only one year group takes GCSEs at any one point in time.
4The watchdog recognised there was ‘some evidence’ that performance is affected by targets. However, the DfE hadn’t attempted to take this into account and didn’t provide information that a specific target could affect results and this might be cumulative.
The Statistics Authority concluded:
‘The Ministerial statement linked to the secondary performance tables in January 2014 was not clearly distinguished from the statistics and included analysis that did not appear to be impartial or meet professional standards.’
In other words, the press release was decidedly dodgy.
It’s not the first time the UK Statistics Watchdog has criticised the DfE (see here
The press release was churned by, among others, the Daily Telegraph
, Sky News
, The Independent
, and TES
. The 250,000 fewer children in 'underperforming' schools was regurgitated by, among others, Michael Gove in his 'Facts are Chiels that Winna Ding' speech
and by education secretary Nicky Morgan
in the House of Commons.
See if your handling of statistics is better than the DfE by taking the LSN quiz
*UK Statistics Authority, Assessment of Compliance with the Code of Practice for Official Statistics: School Level Examination Statistics for England can be downloaded here
. Scroll down to number 289, 25 September 2014.