National IQ and PISA: update

Roger Titcombe's picture
 2

Since I published this article I have received a lot of email correspondence from internationally respected academics. The general thrust of this has been to confirm the general validity of my approach. However Professor Richard Lynn informed me that my national IQ data were out of date and provided me with the update published in the appendix of his book, ‘ Lynn & Vanhaned Intelligence: A Unifying construct for the Social Sciences,’

https://www.amazon.co.uk/d/Books/Intelligence-Unifying-Construct-Social-…

I have to emphasise that I am using these IQ data for the purpose of interpreting the 2015 international PISA test results. Others may use the data for other purposes and come to conclusions that I do not support. However, Lynn’s IQ data now come fully referenced as to sources and include updates resulting from the Flynn effect, along with confirmation that the Flynn increase in IQs over time has ceased for pupils in the UK, as noted in Section 5.10 of, ‘Learning Matters’ where I explain this in terms of the degradation of the English education system caused by marketisation.

Most usefully the updated IQ data are given for various ages. In my corrections I either use the median values or those not above but as close to age 15 as possible. This removes the weakness in my previous analysis of using adult IQs as a proxy for the average national IQs of the students taking the PISA tests.

I fully understand that the details of my analysis depend on the quality of the IQ data. This is not uniform, as Lynn makes clear that the evidence base for some countries is much more limited than for others. I note that Lynn has been criticised for underestimating black African IQs. I can easily see why many educationists find the whole IQ field such dangerous territory that they keep well clear. However in relation to my analysis it should be noted that for all countries if IQs are revised upwards then the effectiveness of the education system is revised downwards.

Lynn’s IQ data comes with this foreword.

These IQ have been obtained from the administration of tests of intelligence and the IQs have been calculated in relation to a British mean of 100 and standard deviation of 15. All IQs have been adjusted for Flynn effects, i.e. secular increases in IQ. Flynn effect adjustments up to the year 1980 are 3 IQ points per decade (Flynn, 1987) for all tests except the Progressive Matrices, for which they are 2 IQ points per decade reported for Britain by Lynn and Hampson (1986). The same adjustments are made for children from 1980 onwards, but for those aged 14 years and above no adjustments have been made because for these, IQ ceased to increase in Britain (Lynn, 2009).

Where data for more than one study in a country have been reported, the mean of the two studies is given, while where there are three or more studies, median IQs are given in the last row for each nation as the best estimates of the national IQs derived from intelligence tests. IQs of multi-racial societies are calculated by weighting the IQs of the races by their proportion in the population given in Philips (1996). Descriptions of many of the studies and how the IQs are calculated are given in Lynn (2006). 

Not everyone may be familiar with scatter charts and their interpretation, so in this update I am presenting the conclusions in the form of a list of the top 30 countries in 2015 PISA maths on the basis of my national IQ mediated outcomes. I also produce the data for the East Asian countries that top the PISA ‘raw results’ table, but this time in order of the raw results, but also including the place in my  IQ mediated league table. Finally I give the IQ mediated UK and USA data as these are of obvious interest.

The scatter chart in my earlier article has been updated using Lynn’s latest IQ data and can be viewed alongside my ‘league table’ list for those (like me) that like scatter charts. As will be noted from what follows, the top country is Poland, closely followed by Ireland.

How I obtained my national IQ mediated list.

The first step was to rework the scatter chart using the corrected IQ data provided by Professor Lynn to produce the updated regression line. This produced a correlation between national IQ and PISA maths scores of 89 percent (compared to 88 percent previously). This shows a very strong link (remarkable even) between IQ and performance in the PISA maths test. Note that as in my previous article I have used IQ percentiles rather than standard IQ scores (mean = 100, SD = 15). The IQ percentile is the proportion of the population with that IQ score or less, so an IQ score of 100 represents the 50th  percentile. IQ Percentiles can be found from IQ scores by consulting published tables. This link  also contains a good explanation of IQ.

However correlation is not causation. This is very important. For example, there is a strong inverse correlation between pupil attainment in GCSE and socio-economic measures like Free School Meals (FSM). This has led virtually the entire UK educational establishment and the media to the conclusion that social deprivation causes low school attainment. This has led a whole industry from the Sutton Trust to Alan Milburn’s ‘Social Mobility Commission’ to create a class discrimination/parental skill deficit construct of an ‘Attainment Gap’, that is wholly false.

So more than correlation is needed. There has to be a credible mechanism, backed by evidence, that it is primarily high IQs that drive high attainment in school exams and in particular the PISA tests that are designed to test deep learning and sound reasoning where there is no specified knowledge content.

What does IQ measure? It is the ability to come to valid conclusions about the meaning of observations/evidence/patterns and logical propositions. In short it is general reasoning power. So is it credible that a student with well developed ‘reasoning power’ will do well  on the PISA maths test? Of course it is.

If you think I am making a meal of something obvious, it has been suggested that my method lacks validity because students in East Asia that become good at maths acquire a high IQ in the process and it is this that explains the high IQ of East Asian students, rather than a high IQ makes it easier to become good at maths. Since I am a strong believer in plastic intelligence, this notion cannot be dismissed out of hand. Indeed it seems likely that the national education systems at the top of my league table do promote cognitive/IQ development. The problem is that the East Asian teaching methods, being based on rote learning from direct instruction are not effective in promoting cognitive development  however much they may appeal to Daily Mail readers and the educationally illiterate politicians in charge at the DfE. This is being increasingly recognised by educationalists in those countries.

We must be clear about what a national IQ means. It does not mean that all East Asian students are cleverer than English students. The results of IQ tests produce the classic Gaussian Normal Distribution of continuous variation, as do the measurements of all individual human traits (eg weight, height) and competences.

Having established the validity of the regression line as a description of how success in the PISA maths tests varies with IQ, we can move on to what to do with it.

If you have studied GCSE maths you will know the general formula for a straight line graph:

y = mx + c

m is the gradient/slope of the line and c is the value where the line crosses the axis. Excel works out this value for you and it is displayed on my updated scatter chart. In terms of the quantities on my scatter chart the formula becomes:

predicted score = 3.1293 x (IQ percentile) + 344.95

Note that the numbers for m and c arise entirely from the data in the chart. When I updated the chart with the revised data these numbers changed, but only slightly.

By substituting the national IQ data into the formula I obtained the predicted average PISA maths score of countries whose students have that average IQ. However the correlation is not 100 percent. Assuming that some national education systems are more effective than others the actual PISA scores are above (more effective education system) or below (less effective system) the predicted score. The underlying assumption is that had all the national education systems been equally effective then the actual PISA scores would have been the same as the predicted scores and would therefore lie on the regression line. For this to be true a large sample size is needed to allow for individual students being ill on the day/being distracted by a personal crisis, etc. The PISA system claims to provide appropriately large samples of students that are representative of the full national student population.

It is also important to recognise that a given high national PISA score could be as much down to high performance of the less cognitively able students in the sample as by the average or the most able. In other words it is not possible to conclude from the national PISA scores that high/low attainers are more/less effectively taught in one country rather than another, although deeper digging into the data does reveal such patterns, which is another reason why the PISA tests and their analysis are such a rich resource for educationalists.

To compare the effectiveness of national education systems all that is left is to subtract the predicted score from the actual score to produce a residual.

Residual = Actual Score – Predicted Score

The formula can be entered into the Excel worksheet, which then then does the calculations.

So here is my IQ mediated PISA maths national education system league table in reverse order of the residuals.

The numbers after the country are IQ, actual score, predicted score, residual

 1. Poland, 92, 504, 437.9, 66.1

2Ireland, 93, 504, 445.1, 58.9

3. Vietnam, 94, 495, 452.9, 42.1

4. UAE, 83427, 385.3, 41.7

5. Portugal, 95, 502, 460.4, 41.6

6. Slovenia, 96, 510, 468.6, 41.4

7. Lithuania, 92, 478, 437.9, 40.1

8. Finland, 97, 511, 476.7, 34.3

9. Estonia, 99, 520, 493.0, 27.0

10. Denmark, 98, 511, 484.8, 26.2

11. Russia, 97, 494, 476.7, 17.3

12. Qatar, 83, 402, 385.3, 16.7

13. Greece, 92, 454, 437.9, 16.1

14. Lebanon, 82, 396, 380.9, 15.1

15. Canada, 100, 516, 501.4, 14.6

16. Belgium, 99, 507, 493, 14.0

17= Romania, 91, 444, 430.7, 13.3

17= Italy, 97, 490, 476.7, 13.3

19. Germany, 99, 506, 493.0,13.0

20. Montenegro, 87*, 418, 405.3, 12.7

21. Trinidad & Tobago, 87, 417, 405.3, 11.7

22. Switzerland, 101, 521, 509.9, 11.1

23. Netherlands, 100, 512, 501.4, 10.6

24, Israel, 95, 470, 460.4, 9.6

25. Spain, 97, 486, 476.7, 9.3

26. Australia, 98, 494, 484.8, 9.2

27. France, 98, 493, 484.8, 8.2

28. Czech Republic, 98, 492, 484.8, 7.2

29. Cyprus (Greek), 91, 437, 430.7, 6.3

30. Thailand, 88, 415, 411, 3.3

* estimated IQ

The next 10 countries are in order of their actual maths score

48. Singapore, 109, 564, 572.1, -8.1

39. Hong Kong (China), 106, 548, 549.9, -1.9

45. Macau (China), 1o6, 544, 544.9, -5.9

47. Tapei (China), 106, 542, 549.9, -7.9

62. South Korea, 106, 524, 549.9, -25.9

9. Estonia, 99, 520, 493.0, 27.0

15. Canada, 100, 516, 501.4, 14.6

8. Finland, 97, 511, 476.7, 34.3

6. Slovenia, 96, 510, 468.6, 41.4

19. Germany, 99, 506, 493.0,13.0

As noted in my earlier article, there is an issue in China in relation to the average IQ levels in each area of this vast and diverse country. The Lynn data do not provide regional breakdowns. In this update the IQ figure has been revised upwards from 100 to 106, which explains why China comes out better in my IQ mediated table.

The Global Education Reform Movement (GERM) countries.

49. UK, 100, 492, 501.4, -9.4

53. USA, 98, 470, 484.8, -14.8

Regional IQ variations probably apply in most countries and almost certainly in the UK. In England, school attainment is much lower in the former industrial northern towns that have suffered economically from de-industrialisation following globalisation. For example I have CATs data for Barrow-in-Furness that produce a mean CATs score for the town of 92. Other northern towns will be similar. In the London Borough of Hackney, the subject of my study reported in Part 4 of, ‘Learning Matters’, I give a figure of 97 for the mean CATs score.

Scotland and Wales have even higher proportions of de-industrialised, poverty blighted towns than England. It is therefore highly likely that the UK average IQ of 100 is made up of a higher figure for England balanced by lower figures for Scotland and Wales. This being the case the Scottish and Welsh governments would be very foolish not to take this into account in evaluating the PISA scores for those countries compared to England. There is no indication that the devolved governments are doing this. My advice to the Scottish and Welsh governments would be to introduce universal Y6 CATs testing in all schools. It would certainly be a great mistake to believe that the English SATs regime is a good model to be copied.

The IQ mediated league table differs massively from that published by PISA based on the raw test scores. This is to be expected. Consider a school that has Cognitive Ability Test scores (CATs) for all it pupils. Assume this school teaches maths in four ability sets. Set one contains the CATs top quartile (score of 110 or greater). Set four contains the bottom quartile (score of 90 or less). The Head of Maths wants to rate the effectiveness of the teachers of each set. Quite obviously he could not do this on the basis of the average GCSE grades obtained in each set. He/she would have to take account of the average CATs scores by using a method like that described in this article.

The current grammar school debate in England is degraded by the often deliberate failure to recognise this statistical fact.

So what do the the most effective national education systems based on my analysis have in common? PISA published a very thorough analysis of the 2012 tests. A huge range of different approaches to pedagogy were evaluated. A similar analysis of the 2015 results will doubtless soon follow. In England the Education Endowment Foundation (EEF) is funded by the government to evaluate pedagogic interventions and initiatives. The problem is that the government largely ignores its findings in favour of its free market dogma.

All of the education systems towards the top of my list are therefore worthy of study. This is not a job for me, but here is some information about the top four.

In top position is Poland. After the 2012 PISA round the Daily Telegraph published an article extolling the Polish education system quoting Education Secretary  Nicky Morgan as claiming that the English education reforms were based on the Polish success. Nothing could be further from the truth as noted by PISA education guru Andreas Schleicher in the same article.

Andreas Schleicher, the OECD director of education and skills, stated: “The UK has pretty much been flat in terms of learning outcomes at least until 2012, despite a very significant increase in spending.

He said that the UK should be doing much more to check what difference education reforms make to children’s lives.

The OECD’s research also warns that the UK has some policies such as grouping pupils by ability in class, and giving families choice over schools, that could “hinder equity” – meaning they may not help to create an equal school system for all pupils.

The Polish education system has been completely reformed in recent years. The new system is described in detail here. Does if follow the English model of the creation of an artificial market in schools driven by large scale statutory testing of pupils primarily to drive league table competition between schools? It does not.

Second comes Ireland. I described the Irish education system  in my earlier articleAgain there is almost nothing in common with the English marketised model.

Third we have Vietnam. This may come as a surprise, but not to Andreas Schleicher, according to this BBC article.

Not just rote learning

These students are expected to leave education not just able to recite what they have learned in class, but to apply those concepts and practices in unfamiliar contexts.

In Vietnamese classrooms there is an impressive level of rigour, with teachers challenging students with demanding questions. The teachers focus on teaching a few things well and with a great sense of coherence that helps students to progress. Teachers in Vietnam are highly respected, both in society as well as in their classrooms. That may be a cultural trait, but it also reflects the role that teachers are given in the education system, which extends well beyond delivering lessons in school and embraces many dimensions of student well-being and support.

Teachers are expected to invest in their own professional development and that of their colleagues, and they work with a high degree of professional autonomy.

Respect, autonomy and high pay for teachers. No mention of testing driving marketisation. Time for a visit to Hanoi, Justine Greening. Why not take Michael Gove with you to learn where he went so wrong? The educational performance of Vietnam is especially impressive given the comparatively low level of public spending by the communist government and lack of infrastructure investment, which must be linked to the lack of an efficient tax system and the highly entrepreneurial nature of its informal and largely unregulated economy. The school system makes do with cramped buildings and at least in primary education, a two shift system in which the school week of six days is made up of half day shifts, so doubling the number of pupils that can be enrolled in each school. Education, however, has a very high priority.

Fourth is UAE. This really is a surprise. Once again we have a country that has recently reformed its entire education system and certainly not on the UK or East Asian model. UAE school students have an average IQ of 83 (13th percentile). Compare this with Singapore (73rd percentile). Yet the UAE students achieved a PISA maths score of  427. What PISA score was being achieved by the very low proportion of Singaporean students with an IQ of 83? What would be their chances of passing the high very high stakes national exams in the Singapore and Chinese systems? All students in these East Asian countries appear to have desperately pressured school lives. This is not good for cognitive development.

This is from a publication of the Embassy of UAE in the USA

Education reform focuses on better preparation, greater accountability, higher standards and improved professionalism. In addition, rote instruction is being replaced with more interactive forms of learning [my bold], and English language education is being integrated into other subjects, such as math and science. The Abu Dhabi Education Council (ADEC), the Dubai Education Council (DEC) and the UAE Ministry of Education (MOE) are each tasked with education reform, while preserving local traditions, principles and the cultural identity of the UAE.

Education at primary and secondary levels is universal and compulsory up to the ninth grade. This takes place in a four-tier process over 14 years:

  1. 4 to 5 year-olds attend kindergarten
  2. 6 to 11 year-olds attend primary schools
  3. the preparatory stage caters for children aged between 12 and 14 and
  4. 15 to 17 year-olds attend secondary schools.

A note of caution is needed here. The fourth place of UAE  is very much down to the very low IQ score of 83. This is from a single study.

So what can we say about the UK (49th) and USA (53rd) systems?

There is clearly very little to be positive about that is for sure. Even more depressing is that the frantic pace of reform is to be stepped up with more testing, more Academies and Free Schools, more faith schools with their own enhanced sectarian admissions rules and now the imposition of selective grammar schools. It would be hard to come up with proposals to make the national education system worse.

The most important message to the DfE is the key role of cognitive ability in driving higher attainment. This needs more of the well-proven developmental pedagogy that the ideology of marketisation is replacing with knowledge-focussed rote learning and behaviourism, enforced by ever more draconian and abusive systems of harsh discipline.

The potential for raising standards through exploiting the potential of ‘plastic intelligence’ is explained here, and the dire consequences of further attempts to ‘close the gap’ are set out here.

This article describes how cognitive developed can be enhance through fundamental changes to school culture.

And finally if we are looking to the long term,  this article explains how Chinese and other East Asian countries came to have such high national IQs.

I am very happy to discuss any issues related to the validity of the arguments/data in this article either through ‘Comments’ or privately at:

Roger Titcombe@yahoo.co.uk

Share on Twitter Share on Facebook

Be notified by email of each new post.





Comments

Roger Titcombe's picture
Mon, 26/12/2016 - 21:26

Important correction in the following paragraph

As noted in my earlier article, there is an issue in China in relation to the average IQ levels in each area of this vast and diverse country. The Lynn data do not provide regional breakdowns. In this update the IQ figure has been revised upwards from 100 to 106, which explains why China comes out better in my IQ mediated table.

This should be:

As noted in my earlier article, there is an issue in China in relation to the average IQ levels in each area of this vast and diverse country. The Lynn data do not provide regional breakdowns. In this update the IQ figure has been revised upwards from 100 to 106, which explains why China comes out worse in my IQ mediated table.

 

 


Roger Titcombe's picture
Thu, 02/03/2017 - 13:50

The following highly relevant comment was posted by 'Y Yn' on my website in response to the my article, 'The Growth Mindset misunderstood'.

https://rogertitcombelearningmatters.wordpress.com/2017/01/30/the-growth...

"Look at any exam-intensive guide in China or in Asia and you find that it uses simple reverse engineering techniques on exam questions over the years (TOEFL, AP, …) giving the user the method of identification, the requisite base knowledge, the correct approach/solution to any minutae change in variable in problem formulation but almost nothing in terms of open-ended real-world questions of unknown complexity, unknown scope and unknown consequences (which mirrors real life)."

This is a profound issue. In a marketised education system driven by parental choice of schools, on what basis can such choice be made? Inevitably it comes down to aggregated exam results on the assumption that the 'best' schools will get your child the best possible exam results and so launch said offspring into an adult life of boundless choices and prosperity. Progress 8 is an attempt to make exam-based school performance indicators more valid. This is like searching for the pot of gold at the end of the rainbow - you think you are almost upon it only for
it be revealed as an illusion. See my articles:

https://rogertitcombelearningmatters.wordpress.com/2016/10/23/why-market...

https://rogertitcombelearningmatters.wordpress.com/2016/10/27/more-probl...

with other educational commentators lining up in increasing numbers to condemn this latest attempt to design a 'success label' for schools as, like all its predecessors, not fit for purpose.

My correspondent Y Yn explains why the goal is unattainable. It is because high stakes exam-based school indicators will always result in corrupted teaching, learning and exam systems that feed entrepreneurial 'sales pitches' for 'innovative' approaches to passing exams.

Y Yn describes these as, "simple reverse engineering techniques on exam questions over the years (TOEFL, AP, …) giving the user the method of identification, the requisite base knowledge, the correct approach/solution to any minutae change in variable in problem formulation but almost nothing in terms of open-ended real-world questions of unknown complexity, unknown scope and unknown consequences (which mirrors real life)."

In other words the task of helping students to understand difficult concepts gets transformed into the deep study of past exam papers in order to memorise the appropriate 'model response' to any question. This is the basis of the 'ARK', knowledge-based model that is spreading throughout the UK and US exam systems. It works by achieving exam success that does not require deep understanding and does not develop cognition (general intelligence).

The approval of Far-East education systems based on the model spells long term disaster for true innovation and creativity and societal loss of the ability to rationally analysise and solve the new problems faced by humanity. As my PISA studies show, the Far East systems achieve success based on the high mean IQ of their students, despite the failings of their education systems.

We get Trump and Brexit. The only antidote is the long-term reform of our education system so as to prioritise the development of intelligence, wisdom and deep understanding and reject all the perverse errors and elephant traps of marketisation.


Add new comment

Already a member? Click here to log in before you comment. Or register with us.