PISA and TIMMS

rogertitcombe's picture
 0
Whenever Gove and others draw attention to international tests that show declining performance of English pupils especially in maths and science, those in denial on LSN draw attention to differences between PISA and TIMMS.

My starting point, as a science teacher, is looking at the sample questions. They can be found at:

for PISA: download PISA 2009 key findings Volume 1. What students know and can do.

Maths and Science samples start on pages 122 and 137

for TIMSS: select Grade 8 Released Items (bottom of page)

The highest PISA level is Grade 6. You will find a grade 6 science question at p141. It is about the Greenhouse Effect. I can't copy it so you will have to look at it. There is no doubt that the question challenges at the highest levels. If you are thinking Bloom it does the lot. I particularly like the structure of the question. Some complex experimental data are provided togerther with 'explanations' from two students. The question requires the explanations to be compared and evaluated both for internal consistency and scientific understanding. Wow! It is hard.

The highest TIMMS level is 'reasoning'. I looked at item SO52165. It should be possible to find this on the website.

The comparison with PISA difficulty is obvious. The TIMMS question is in the 'accessible' short answer format familiar to English school students from KS2 and GCSE science questions. There is no way this 'reasoning' is on a level with PISA Level 6. It is bumping along the Bloom basement 'lower' order thinking. If you don't like Bloom then any other analysis will draw a similar conclusion.

But there is an even bigger problem. The question is completely misguided in principle. The author clearly does not understand the science that the question purports to test. The context is energy transfer from potential to other forms simulating hydro-electric power generation or a water mill of some sort. This situation is only meaningful in terms of work and power, not just forms of energy. The maximum power (watts) that can be extracted from the system = weight (in newtons) of water flowing per second x height dropped. The problem is that the speed of rotation of the paddle wheel does not necessarily indicate power output, unless the rotation is doing some work. If it is not doing any work and rotating at constant speed then there is no useful power being developed by the paddle regardless of how fast it is rotating. If the paddle is doing work then the power output = torque x rotations per second. High torque, low speed of rotation can produce the same power as low torque, high speed of rotation, so part C is misguided. In the set up in the diagram all the energy ends up being wasted as heat in the friction and turbulence of the system, which is extremely complicated, and no clear relationship with the speed of rotation of the paddle wheel is likely.

To come to the mark scheme, increasing the flow rate of the water and raising the height of the tank above the paddle will result in more potential energy per second (power) lost by the water, but the wheel turning faster does not necessarily equate to this power. The design of the paddle, number and size of the blades are complex factors that would affect the efficiency of the arrangement, but as it isn't doing any work then it has zero efficiency anyway.

To make this question work then the paddle should have been shown driving a small model generator connected to a filament lamp. Increasing the flow rate of the water would then make the lamp glow brighter, but the effect of messing about with the paddles would be very uncertain.

So this question is deeply flawed and should not have got into the test. Clearly success or failure on the question does not validly indicate scientific understanding. A really bright student would have written 'this is a bum question' and got no marks for their correct answer!

So this does not inspire confidence in TIMMS.

I invite other subject specialists to compare PISA and TIMMS in their subject areas.

Like Henry Stewart and many other contributors to LSN, I deplore the constant denigration of teachers and comprehensive schools by Gove. But this does not mean Gove is wrong about declining standards.

The point is the decline is not the fault of schools, but of the league table driven, marketised competitive system in which schools are compelled to operate.

The main causes of the decline in understanding across the full ability range are: high stakes concentration on some levels and grades at the expense of the rest; resort to behaviourist cramming teaching methods and general curriculum degradation through substituting time filling easy vocational studies for learning that stimulates cognitive development.

Most on LSN accept that these negative developments have taken place, but appear to be denying they have caused any damage.

The English education system is seriously broken and I agree with Fiona Millar and Henry Stewart that Stephen Twigg's latest proposals would go a long way towards addressing some of the problems. However they do not address the core problem - the competitive customer/provider model driven by school league tables that have no valid connection to school quality.

Until Labour addresses the core problem its educational proposals cannot be taken seriously.
Share on Twitter
Category: 

Add new comment

Already a member? Click here to log in before you comment. Or register with us.