Is Michael Gove right to praise externally marked exams as the route to "rigour"? The most "rigorous" research shows he's wrong!

Francis Gilbert's picture
 6
Michael Gove is giving a widely-leaked speech today at the Independent Academies Association in which he will praise tests. According to the Telegraph he will complain that teachers often over-value white pupils' work, and under value "ethnic minorities", saying:  "External tests are fairer. With external testing there is no opportunity for such bias - the soft bigotry of low expectations - and tests show ethnic minority students performing better. So external tests are not only a way of levelling the playing field for children of all backgrounds they are a solvent of prejudice.” This is an insidious comment and one worthy of attention. Yes, externally marked exams may get rid of the issue of racial stereotyping -- which has only been anecdotally suggested by the Ofqual report in the GCSE marking fiasco -- but this doesn't mean that exams are "reliable" per se. Racial stereotyping is a whole issue unto itself and conflating it with summative assessment is unhelpful to say the least because it begins to hint that many teachers are racist.

Gove really needs to read Assessment for Schools -- fit for purpose? a commentary by the Teaching and Learning Research Programme, which looks forensically at the issue of external testing. The whole report is illuminating, but this section reveals that externally marked tests are not reliable measures for assessing achievement overall.  The report says: "Research suggests that we should treat national test results in England, as measures of pupil performance,  with caution. As noted above, Dylan Wiliam estimated in 2000 that at least 30 per cent of pupils could be  misclassified in these tests. In December 2008, Paul Newton suggested a figure closer to 16 per cent. In  March 2009, the Qualifications and Curriculum Authority published research based on tests taken in 2006 and  2007. This analysed the number of times markers agreed on the level to award to pupils’ answers in the now discontinued key stage 3 tests in English, maths and science. The extent of agreement varied from 95 per cent  in maths to 56 per cent in English writing, the latter suggesting that markers disagreed on the “correct” level to award in nearly half of these cases."

The report also reveals that there is real problems with the validity of much external assessment -- that is, are they actually testing the right skills and knowledge? Gove believes that children don't learn enough facts. He says: “Only when facts and concepts are committed securely to the working memory – so that it is no effort to recall them and no effort is required to work things out from first principles – do we really have a secure hold on knowledge." The problem here, is whose facts and concepts? Michael Gove's vision of what are important facts may well be different from many other people's; are the "facts" he believes in more important to learn than other people's? Here, we come up against the central problem with the way he has gone about implementing much education policy; he hasn't consulted anyone other than people who agree with him -- as a recent Guardian article on the advisors involved with his National Curriculum review shows. No real consensus that embraces the full diversity of the nation has been drawn. As a result, many parents, teachers and students feel that Gove is imposing his vision upon us.

And how misguided and ill-judged this agenda is! According to the Telegraph, he will say: "success in exams also gives pupils a sense of achievement and “happiness” that motivates them to work harder and achieve more in the future." Once again, it's worth quoting the Teaching and Learning Programme's research, its report states: "More generally on pupil motivation, the most extensive review of research in recent years (Harlen W., & Deakin Crick R. (2002) A sytematic review of the impact of summative assessment and tests on students’  motivation for learning. In: Research Evidence in Education Library. London: EPPI-Centre, Social Science Research Unit, Institute of Education, University of London) on the effect of the tests found that those that were seen as “high stakes” de-motivated many children. Only the highest attainers thrived on them, with many showing high levels of anxiety. After the introduction of national testing in England, the research found, self-esteem of young children became tied to achievement, whereas before there was little correlation between the two. This effect was found to increase the gap between low- and high-achieving pupils, with repeated test practice tending to reinforce the poor self-image of the less academic. The review also found that pupils tended to react by viewing learning as a means to an end – the pursuing of high marks – rather than as an end in itself."

The trouble is that Gove is anything but rigorous. He uses the word all the time, but if he had cared to consult the most "rigorous" research, he'd find out he was wrong.
Share on Twitter

Comments

Michael Dix's picture
Wed, 14/11/2012 - 18:43

Reliability of tests can be called into account on two fronts. There is the variation in marking - only hearsay but one of my teachers joined her KS2 English marking team a few years back in the days when all markers had samples to do beforehand. The experienced team, with official guidance, marked the same sample level 3, level 4 and level 5!

There is also the fact that a child, on a different day, may produce a different set of answers or piece of work. On the government's own data analysis for schools site RAISEonline, this is shown in whisker diagrams. This also brings into question the use of examination data to judge schools, and the fact that tests are high stake also affects their reliability as measures of pupil's achievement.

Rebecca Hanson's picture
Sat, 17/11/2012 - 17:39

Then there's also the issue that externally marked tests can only assess a much smaller range of skills than can be assessed by someone who has direct contact with the child.

So there is less rigor, because the progress of the child is less thoroughly assessed, because less of it can be assessed.

Adrian Elliott's picture
Thu, 15/11/2012 - 09:03

Francis and Michael. You are both correct; an excellent piece Francis. I was teaching throughout the days of the so called 'tough' exams. Of course, most kids got what they deserved (and what one had predicted for them) but there was always a significant minority who fell through the net of the 'one off' exam, either as a result of an unexpectedly poor performance on the day or, just as likely, maverick marking.

And the latter goes back decades . In the early '60s the JMB asked its Englsih examiners to take part in a experiment. It distirbuted old scripts with the previous examiners comments and marks erased and asked examiners to re-mark them. There was an astonishing discrepancy in a large number of papers -Bs becoming Es and vic-versa etc.
And this despite the fact that the Board had returned the scripts to exactly the same examiners who had originally marked them.

As an A level History examiner twenty years later, I recall a new examiner joining us at our post exam moderation meeting. He was actually the chief examiner from another paper who wanted to broaden his experience.When we went round the table presenting our marks from the first moderated script (always a tense moment) all the regular team
awarded it a D, giving it a mark in the low 50%s. The new guy gave it 84%. It was a cringingly embarrasing moment but it did make me wonder whether I should switch to teaching his paper!

As you say,Francis, Gove has no interest in research. He is still a Murdoch hack at heart - right wing, not in the old fashioned Tory manner but in a Tea-party,Fox news sense.

Francis Gilbert's picture
Thu, 15/11/2012 - 13:23

In Finland, teachers are empowered to “tailor-make” bespoke assessments for pupils; assessments that genuinely reveal their abilities. Exams are a very blunt and imperfect instrument. I think we need to separate off assessing teachers and pupils; we should not be assessing teachers on their pupils’ exam results, but on the progress pupils make; as we have seen, external exams are a very imperfect measure of progress. We need to return power to the teachers, the people who know the pupils best; the increasing centralised control over teachers has meant a stripping away of autonomy, and that has meant assessment systems have failed to assess properly.

James Blythe's picture
Fri, 16/11/2012 - 08:19

Surely a 'tailor made' bespoke assessment for a pupil is a school report or do these, along with the attendant increase in sales of blotting paper (Yaroo!), not exist, anymore?

Janet Downs's picture
Fri, 16/11/2012 - 08:57

James - there used to be something called a Record of Achievement (c 1998) which assembled teachers' comments, a pupil's self-assessment, exam certificates, work experience report, sporting and other certificates and so on into a specially-produced folder (they were maroon and had National Record of Achievement, or similar, embossed on the front). These were "tailor-made" combining internal assessment with external certification. At the time employers were being asked to request them at interviews.

However, they seem to have disappeared. I think that's a pity - such a record could contribute to graduation at 18.

Add new comment

Already a member? Click here to log in before you comment. Or register with us.