Despite the title, it is about, learning from mistakes. The ‘black box’ reference is literal. It refers to the black box data in crashed aircraft that provide the evidence needed to learn from mistakes that caused or contributed to the crash. It addresses a vast range of bad event contexts from airline safety through deaths and harm caused by medical errors, the mental contortions of politicians faced with dire negative consequences of their decisions ( eg invading Iraq), the unwillingness of police and prosecutors to accept the innocence of those wrongly convicted of serious crimes when subsequently vindicated by solid evidence (often DNA), to wrong-headed decision making in business, scientific research, sport and more.
It is a powerful articulation of the vital, perhaps universal, role of mistakes in all deep learning.
Syed identifies two cultural attitudes to mistakes. He refers to these as ‘closed loop’ and‘open loop’ responses to failures. This is not to be confused with the use of the same terms in engineering feedback systems.
Syed’s ‘closed loop’ is where information doesn’t lead to progress because errors and weaknesses together with their consequences are misinterpreted or ignored. An ‘open loop’ is where feedback from a failure is rationally and thoroughly analysed leading to learning that is acted upon.
An example of the former is the historic medical practice of ‘bloodletting’, which frequently led to the death of the patient either directly through the loss of blood or indirectly because the blood loss so weakened the natural systems that the patient succumbed to the true agent causing the illness. Despite continuously poor outcomes, ‘bloodletting’ continued as a medical practice for more than 1500 years. As late as 1942, a medical textbook considered bloodletting appropriate treatment for pneumonia. The ‘closed loop’ culture resisted any proper analysis of outcomes. If a patient survived the doctor would say, “Boodletting cured him”. If not, the explanation was also clear. The patient must have been extremely ill if the wonder cure of bloodletting did not work.
‘Open loops’, where mistakes are analysed and learning takes place, have long been the modern scientific assumption, but as Syed shows through many examples, especially in healthcare, this often does not take place in practice.
Nature uses ‘open loop’ feedback in response to failure. The clearest and most ancient example is the mechanism of ‘natural selection’ that drives the process of evolution of all life on earth. Failure is the death of individuals prior to reproduction. The feedback from this failure directly drives the changing make up of the gene pool of the population of the species.
Open loop feedback is genetically hard wired into the mechanisms of perception in animals. The responses from sensory nerves are processed by the brain resulting in both conscious images (vision) and subconscious responses such as by the autonomic nervous system: a part of the nervous system that regulates key involuntary functions of the body, including the activity of the heart muscle; the smooth muscles, including the muscles of the intestinal tract; and the glands.
This is well understood in the human sense of vision, where we do not perceive the visual image that accurately reproduces the physical optical image projected by the eye lens onto the retinal nerves. Instead, the brain processes this raw image so as to make the best guesses from optical ambiguities and perceived threats to produce conscious images in the mind that are optimal for the success of the individual. This process relies on algorithms produced by millions of years of the evolutionary feedback from failures. This feedback is somehow processed in brain tissue in ways that are a very long way from being understood.
The achievements of science and technology are the result of the adoption of the ‘scientific method’, which is directly driven by the principle of learning from failure. Scientific understanding is not the celebration of experimental confirmations, but of the failures of attempts at refutation. A new scientific theory results not in a rush for experimental confirmation, but in attempts to refute it, especially by those that believe it to be true. This is the famous principle described by Karl Popper.
Technological achievements are often a combination of scientific theories tested by such Popperian failures and actual physical failures and crashes (literally). The windows in jet aircraft are rounded in shape rather than square as a direct result of the crashes of the pioneering Comet jet airliner in the 1950s. These identified the role of window shape in the development of fatigue cracks.
Feedback from failures can also drive technological innovation through cumulative marginal gains that result from ‘small step’ iteration techniques that work when there is either no well understood theory or else the complexity of the system is too great to be theoretically analysed. Syed gives some powerful examples of this in sport, design and business.
Part 2 of ‘Black Box Thinking’ is about ‘Cognitive Dissonance‘, which is one the main mechanisms of not just failure to learn from mistakes, but in explaining why decent professionals so often adopt defensive, cover-up, self delusional attitudes when they make mistakes. Cognitive Dissonance is almost always a factor in human ‘closed loop’ responses to mistakes and failures.
Cognitive Dissonance (inner mental conflict) is a universal human psychological response that mitigates against open disclosure and analysis of one’s own mistakes and failures in order, consciously and/or subconsciously, to protect self esteem and reputation. Syed shows that the higher the status of the person involved, the greater the degree of personal cognitive dissonance and the greater the self-delusion and public obfuscation. He quotes by way of example the public statements about Saddam Hussein’s Weapons of Mass Destruction (WMD) by the British Prime Minister Tony Blair prior to and after the invasion of Iraq.
Syed only briefly addresses the issue of ‘cognitive dissonance’ in school-based learning, which is the subject of my book , ‘Learning Matters‘. In it I describe the ‘closed loop’, common sense learning theory of ‘behaviourism’ in contrast to the counter-intuitive ‘open loop’ learning theories of ‘developmentalism’. This is also explained on my website.
Syed’s approach has further informed my thinking in terms of the universality of ‘learning from failure’. My book, Learning Matters sets out an alarming analysis of the marketised English education system in which I argue that the result of competition between schools is driving a destructive form of market choice that reduces educational standards, while at the same time securing a ‘closed loop’ national justification in the form of continuously rising exam results that hide the resulting decline in deep understanding and failure of development of the cognitive function of our learners. This is further addressed on my website here and here.
Syed is right to draw attention to the positive features of market forces in which competition and market failures drive technological advances and product improvements.
However I argue that marketisation in the area of public services is more likely to result in the degrading of quality through the emergence of flawed performance indicators that drive corrupting perverse incentives. This is fully explained in my book.
A direct mechanism is the imposition onto schools by the market schools of ‘closed loop’ behaviourist teaching methods adopted by ‘business culture’ school leaders. These are then forced onto teachers and then by teachers onto their pupils. Such approaches emerge because market choice requires ‘performance indicators’. In the case of consumer products and services these are often obvious and simple to identify and to quantify. In public services this is usually not the case because the ‘performance indicators’ are not selected by voluntary customers, but imposed by governments upon the public that needs and has already paid for the service through taxes. It is the distinction between customers; andpatients (state provided hospital services), school students (state provided education) andpassengers (state provided transport). Imposed ‘performance indicators’ like for example, the proportion of pupils obtaining ‘C’ grade GCSE passes in a school, always generate unforeseen consequences, that often turn out to be seriously degrading of quality.
In Part 5 of ‘Learning Matters’ I give several examples of ‘Open Loop’ developmental approaches to teaching and learning. These are being displaced from our schools by the pedagogy that fits the ideology of artificially imposed public service marketisation. This is called the Global Educational Reform Movement (GERM), an international ‘closed loop’ educational recipe that originated in the US and which has been adopted in England (but not other parts of the UK) and uncritically accepted by the British media. Most regrettably of all, GERM ideas, which do not stand up to evidence-based analysis, have been taken up not just by the Republicans in the US and the Conservatives in the UK, but also by the Democratic Party in the US and the Labour and Liberal Democratic Parties in the UK. Sweden is currently recovering from the failure of their adoption. The current phase of the English education system is finally being informed by the failure of the system of imposed marketisation to raise real educational standards rather than the illusion created by the inflation of the artificially imposed performance indicators.