Stories + Views
The myth of measurement — the problem with high stakes exams
“A school that is ridden by examination incubus is charged with deceit. All who become acclimatized to the influence of the system, pupils, teachers, examiners, employers of labour, parents, MPs and the rest, fall victims and are content to see themselves with outward and visible signs, class lists, orders of merit, as being of quasi-divine authority.” (Edmund Holmes, 1911)
I’m reading a brilliant book at the moment, An Introduction To Assessment by Patricia Broadfoot which mounts the most convincing case I’ve read against standardised tests. Broadfoot’s thesis is that for all standardised tests’ apparent objectivity, they are inevitably subjective tests because of the narrow range of skills and knowledge that they assess. Skills such as problem-solving, personal effectiveness, thinking skills, and willingness to accept change are very difficult to quantify in a standardised test. She makes the argument that standardised tests arose in the 19th century as a form of centralised social control — a more effective form than even a system like feudalism. Napoleon in France and Frederick the Great of Prussia used standardised tests to set the agenda as to what was taught in schools. She goes on to show that far from testing intelligence most standardised tests are actually assessments of social class and only serve to exacerbate social inequalities. She points out that the 11-plus is one of the most socially divisive exams in history with just 5.8% of children on Free School Meals attending grammar schools. Her quotation of Mary-Lee Smith’s summary in 1991 of the problems with tests is worth quoting in full:
“The teacher may not have covered what is tested in the examination.
The test may be too long for students to concentrate on it.
Students may guess if the formate is a multiple-choice one.
The test may be incorrectly normed for a given population of students.
Perhaps one summative grade is offered which conceals substantial differences in the quality of performance in understanding contributory aspects of achievements.
The test may measure performance on a given day which has no connection with long-term retention by the student.
Students may be bored and disaffected and not engage to the best of their ability with the test or examination.
They may find the questions confusing or ambiguous.
They may not be able to apply their knowledge because of the limitations of handwriting or other mechanical abilities.
Many achievement tests merely measure endurance or persistence rather than performance.
Some students who are divergent thinkers may read too much into the test terms.
Some students become frightened and ‘freeze up’ in the testing situation, especially those who have no self-confidence or some kind of emotional or family disturbance.”
Furthermore, research in the psychology of assessment has consistently demonstrated these factors all have an effect on performance that:
- small changes in task presentation;
- in response mode;
- in the conditions under which assessment takes place;
- in the relations between assessor and assessed and within students on different occasions