Daniel Kahneman on Making Intelligent Decisions in a Chaotic World | Intelligence Squared

Welcome everyone to Union Chapel and Intelligence Square. It's great to be back with all of you in the hall, not on a computer, however there are people watching online, so hello, thanks for joining the fact that "we". That we're all here tonight is proof enough, haven't we already made at least one smart decision in a

chaotic

world

, plus I haven't arrived with a suitcase full of wine or even a karaoke machine? We will all be courteous to security staff and cleaning staff. Our decision

making

so far has been impeccable, but tonight's conversation may reveal that perhaps not all of our

decisions

are as sound as we would like to believe, our self-confidence. our experience can be misplaced and our judgments can be wrong so without further ado let me introduce you to our speakers Daniel Kman probably doesn't need much introduction, he is a Nobel Prize winner and author of the hugely influential international bestseller Thinking Fast and Slow and his His day job is as a professor of psychology and public affairs at Princeton University.

Professor Caraman, of course, is here to talk about his new book Noise, a Defect in Human Judgment, and I'm pleased to say that one of his colleagues is also joining us. -authors Olivier Bourney, professor of strategy and business policy at H Paris, was previously a senior partner at McKenzie and is also the author of You're About to Make a Terrible Mistake, Well, I Hope We're Not About to Get It Right. Now before we really get going, I just want to remind you that I want all of you to participate both in the classroom and at home online.

There is a chance that there will be a lot of questions and answers towards the end, after about an hour. There will be microphones down here in front and up there, please come to the front. I'd love to hear from all of you and if you're online, there's kind of an area at the bottom of the screen, so to speak. You can ask questions in the box just below the screen, write them down we want to hear from you, start thinking and, uh, I'll make sure there's enough time for everyone's questions, let's get started, so noise, basic definitions, what is noise Well, noise has many meanings. but the noise we are talking about is judgment noise and judgment noise is variability that should not exist, it is when people make judgments that should be identical because they are about the same topic but they turn out to be variable and there is a lot of noise and our statement about it is that wherever there is Judgment there is noise and there is a lot more than you think and you know that is what the book is really about and there is more noise than people expect to see and what it is like. that, unlike what we consider bias, to make a clear distinction between noise and bias, we have to look at what we have to talk about what judgment is and the way we think about judgment is that we think about judgment as a species measurement. where the measuring instrument is the human mind and like other measurements, you deal with a subject, an object subject and you assign that subject's value on a scale and this is what, Jud, the type of judgment that what we talk about is, we really talk about judgments about specific topics now in measurement there is a theory of measurement which is a basic theory that you know works for all sciences and, of course, there is a theory of measurement errors, so when you measure, you take a physical measurement, let's say of a line with a very, very fine ruler and you take repeated measurements of the length of the same lines, then what's going to happen is that you won't get the same result every time if the ruler is fine enough, and that variability is noise, but there is another type of error and that is that each measurement is likely to be an error, an overestimation or an underestimation, and the average error is called bias and this is actually those are the meanings of bias and noise that we use in discussing judgment so bias is an average systematic error and noise is just a variability of judgments and it's a variability that shouldn't exist so Olivier, why are we all so much more familiar with the idea of ​​bias?

Is it the kind of sexy cousin of loud, well, it's sexier, it's more visible, so one way to think about the definition that Danny gave, which is perhaps more visual, is to imagine that you have a team of shooters shooting at a target, everyone uses the same rifle and they are all shooting at the same target and you look at the target after everyone shot and if you see that they all hit the target, you say great, now they are accurate if you see that they all hit the same place, but that the place is not the target you immediately say that this requires an explanation there must be a reason there is something wrong with the rifle or there is something wrong with the target or someone moved the target or maybe the wind is blowing too hard immediately you look for a c is for That prejudice is sexier as you say well because prejudice is something you can explain, you can point your finger at the reason why all these people made the same mistake if, on the contrary, you see your five or six shooters shooting everywhere you just shrug your shoulders. and say: Well, they're not great shooters, but that doesn't make you want to explain it, so noise is more random by definition and less easy to detect when you see it.

I'm still curious why we haven't thought about it. This more systematically the way you do it in the book is that, in a sense, people in general have too much confidence in their judgment when

making

decisions

. Well, you know, it would be an exaggeration to say that people haven't thought about it. People have looked at reliability of judgment for a long time, which I think is justified in writing the noise in writing the book was with the pre, well, the prevalence is just a lot of noise and that would possibly use an example for that. Absolutely, why should we worry about that?

I would like to tell the story of how the study began and how the study began several years ago, when I was doing consulting work at an insurance company and I had the idea to perform what we would now call a noise audit. but it's actually a pretty standard exercise of constructing problems that were typical underwriting problems, pretty realistic, and where multiple insurers assign a dollar value to the same risk, and of course you wouldn't expect these people to agree, but I asked the executives if they took two. below the random writers on how much you expect them to vary in percentages and that question turns out to have an answer and without asking you I think we know what number you have in mind because we have asked it question from a large number of people and the number is approximately 10 % people think that 10% is a reasonable difference between people we know that they will not agree perfectly because it is a matter of judgment and judgment is defined by the possibility of reasonable disagreement but we expect a limited amount of noise, it turned out that The average difference between two insurers was 50%, five times greater than what the executives expected and that's really the reason for writing the book, is that there was a lot more noise.

Than people expected and this was a complete surprise for his organization and Olivier soon joined and we worked together on that issue and it turns out that it is not only in the insurance company but wherever you look at the trial you find noise and a lot and also There is a misunderstanding that explains why people have paid less attention to it. The misunderstanding is the misconception that it doesn't matter because on average it cancels out the right noise, it's essentially a zero average error and people tend to say well, we. We care about bias because bias is shared error, but we don't care about noise because on average it cancels out, of course, that's wrong.

If you think about underwriters, if on average they set the price correctly, but sometimes they set the price too low and sometimes it's too high, well, when it's too low, the company will lose money by paying too much in claims and when it's too high, You will lose money by losing business to a competitor, so both mistakes are costly, but when we think about the noise. We tend to think that average errors, on average, errors cancel each other out, they don't cancel each other out and that's what we want to draw attention to, so this is a business context and we can go back to some more examples, but in a business context. context, there is money at stake essentially what you are saying is a financial cost oh absolutely it is a mistake, it is a mistake what is actually a notion that I should have mentioned before measurement theory is that in measurement theory measurement noise and biases are equally important, in fact there is a formula, the math doesn't matter, but the formula is that a measure of overall inaccuracy is the square of the bias plus the square of the noise and that's just to give you a idea that bias and noise are independent sources of inaccuracy and, in principle, are equally important in practice.

We believe there is more noise than bias in the judgment. It has a cost, as you point out, but it's not just the business cost. Another example that we analyze is that of Criminal Justice. and when you give the same cases, this is an old study that was done in the US, we are not aware of it being done anywhere else including here, but I'm sure the results would be very similar if you give the same cases in simplified cases. vignettes to multiple experienced judges you will find a lot of variability in their judgments for some of those cases you will find some judges saying you are in prison and others saying you are in prison for some of these cases you will find some judges saying 15 years in prison and others saying there is no no prison on average for a sentence of seven years you have almost four years of difference between two judges again looking at the same case we are not talking about individualized sentences you take care of the details of each situation that is already in the case we are talking about different judges looking at the In the same case, this is not a commercial cost but it is a great problem of equity and one of the great consequences of noise is injustice, something that I found fascinating in that particular example was that it had been recognized in an adequate study, mandatory guidelines were established and then they were taken, there were so many things that the judges were so unhappy that they became advisors, it is a little more complicated, in fact, it was a long battle. led by a judge called Franco in the US and then by Ted Kennedy when he was a senator, which led to the establishment of guidelines that define quite precisely what the sentence should be, with some freedom to adjust to the situation and the possibility of deviate from the guidelines if justified, etc. and then, for technical reasons that have nothing to do with what we're discussing here, the Supreme Court turned those guidelines into advisory guidelines and the noise reappeared, not because the justices were unhappy with them, but because the justices They weren't happy, but but.

They were not happy, and as soon as they could regain their discretion, they used it to the fullest extent possible. Talking about noise with judges is a pretty interesting exercise because, you know, the idea is that this is a justice system. determined as I am for the accused and the idea that there is noise in that system is acceptable to the extent that I was actually invited to speak to a large group of judges, but what the organizers asked me to speak about was the noise in Medicine, so the drilling noise in Justice is quite complex, but actually the noise in medicine, sorry, go on, but what is the noise in medicine?

There is a lot of noise in medicine, about as much noise as wherever there is Judgment, there is noise, etc. this is true for diagnosis this is true for treatment in some cases it's actually quite shocking that they sent us a data set from Harvard Medical School on the diagnosis of epilepsy from EG records and the average correlation between two doctors in judging cases is not exactly zero, but it is not much greater than zero and is negative. I think, if I remember correctly, I think it was a small negative number. No no? I don't think I remember it as slightly positive, but Cas in neither.

In the case it was very small, adding to that opinion that medicine is an art more than a science, but so far I want to say that these are decisions that are made repeatedly, so if you go back to the judge or the doctor, in fact, they are making similar decisions. decisions over and over again. Does it apply to one-time decisions? Can we talk about noise in singular decisions? Well, we ask ourselves that question because, obviously, when we measure noise, when we define, no, we define it as variability between multiple decisions. that should be identical, that definition does not apply to a decision that is made only once, but when we look at the things that cause noise that we have not talked about yet and the psychological mechanisms behind the noise, it becomes very clear that they are also present .

When you make one of those life-or-death decisions that changes the

world

, you know, Brexit or no Brexit, is there noise in that decision? Well, if you believe that it is a judgment that a human being is making, there must be one because the same causes that createnoise out there, we come to think of those singular decisions, those unique decisions, as repeated decisions that happen only once, if they were repeated, you would actually see that there is noise in them, you can't see it, but that doesn't mean that there is no noise , so let's talk about the causes of noise.

I mean, why do three judges facing very similar cases make three different decisions or three doctors looking at the same x-ray come to three different conclusions? Well, there are several sources. of noise actually thereThere are three main sources of noise and one source of noise is the constant differences between noise differences between judges in the trials they make, so some judges will be harsher than others, so on average their sentences the sentences of one judge will be higher than the other this is well known this is quite recognized there is another source of noise that does not surprise us people either and this noise is made within a judge who is the same judge uh in a good mood or bad mood results in a sunny day or a rainy day and a hot day or a cooler day will pass, let's make slightly different judgments, we call that occasion noise, but then there is a third source of noise that turns out to be the most important and I would call it a difference almost in personality, if you will, and to see it in the decisions in the context of Justice, if you take two judges and show them 20 cases, the new type of noise is that they will not order the cases in terms of severity in the same way, so you may have a judge who is particularly surprised when you know the defendants are young or when the victims are older and there are differences in taste, you may have a judge who is strongly affected by someone was similar to a member of their family and that would be consistent and we call it pattern noise and it turns out that pattern noise is the most important, most interesting and most difficult type of noise, so pattern noise it's not like group thinking, it's where a group works, but that also affects the noise, so group, let's clear up the tapping noise first, okay, let's clear up the noise P, yes, p, and that touches on something you mentioned earlier that when people look at the world or at any problem in the world uh you have the feeling that each of us had the feeling that we see the world the way we see it because that's how it is so we have the feeling that we are in contact with reality and if the world is as we see it, we expect other reasonable people to see it the same way and it turns out that they don't to an extent that is radically surprising, so what we have to assume here is the idea of that People are really more different from each other in the way they view any problem in the world than any of them would expect and that is the basic phenomenon of no, and we also have to accept the idea that that is not good news when we say oh, we are all different, we are all unique, we are all diverse, we have different points of view on everything we normally celebrate, that's what we say, that's beautiful, that's diversity, that's where creativity comes from, that's where innovation comes , but the question is when you go. to a doctor and he says you have this and then you go to another doctor and he says you have that don't you say oh beautiful creativity Innovation diversity no you say one of you must be wrong maybe both of you by the way so when I think there is a correct answer, that's how we define judgment.

That type of diversity is what we call pattern noise. The fact that people project their own history, their own sensitivities, and their own biases onto a situation is what creates this pattern noise. The good news is not what people call human agency, it's what people call projecting their personality into their judgment and of course if you expect them to do that that's beautiful, if you choose who to marry that way congratulations , that's how I would do it. I recommend you do that if you're making a hiring decision on behalf of the BBC, we might have a problem, so let's go back to the previous point, so if you have a meeting and the first person stands up and says, I definitely want to do it. it's this way, you know, I definitely think we should only employ people who wear red shirts and the second person stands up and says if they're more likely to make the same decision, how do people affect each other?

How does that create noise? Groups tend to create noise just as you said: when you want to minimize or reduce noise, then you want people's judgments to be independent of each other, so you would want witnesses to the same crime to discuss their testimony. before they give it and this is precisely what happens in a meeting in a meeting people influence each other and the first person who speaks has a disproportionate influence on the others and that is an element of noise because the first person who speaks is not necessarily the The best is not necessarily the most accurate.

So are you saying in a sense that we are not willing to disagree? Well, there's certainly a lot of conformity and this is where you know otherwise the meetings could last a lot longer, but. and how they are too long but there is convergence and in that sense there is more convergence than we would like. Ideally, the ideal form for a meeting would be one where people come prepared, each with their own opinion and you establish the degree of divergence in their opinions and now you discuss it and now you try to achieve convergence, but the automatic type of convergence that happens in a debate is actually not a noise reduction mechanism when we were working on this. topic, we interviewed an organizational psychologist who worked with college admissions officers and he told us the following story: He had arrived at an admissions office at a large university where admissions officers were reading applicants' application essays, graded and then give the essay with the grade to another admissions officer, who would make a separate judgment, so our friend recommended, of course, that they make those independent judgments by hiding the grade given by the first officer from the second officer so that These two judgments could be different. and the response he got was oh, that's how we used to do it, but we disagreed so much that we adopted the current method and this is a perfect summary of the balance that organizations make between the consensus that they must achieve at some point and the disagreement.

What they do have, they tend to suppress disagreement as best they can to achieve consensus and be able to make decisions, and what we are saying here is that if they want consensus to be achieved on something that is closest to the best possible answer, You may want to delay when that consensus is achieved and make sure you get independent opinions early in the process. It is interesting that we are talking about convergence and people who want to reach an agreement, when certainly in politics what we all talk about at the moment is polarization, well, we talk about judgments and we try to avoid value judgments and differences in values, for what the assumption is that when you have judges or subordinates, they speak on behalf of an organization, the organization speaks through individual officials and you want an organization to speak with one voice uh and and the noise is a failure it's a cacophony but when it's about values ​​or when it comes to politics in a democracy you certainly want to allow for differences and divergences of values ​​and and you certainly don't want to impose noise reduction oh, that's interesting, there's no noise reduction in politics Hum, so let's think about some of the things that this has thrown up, that convergence in a sense, hiding well, is a way of avoiding the fact that there is noise, why do you think?

Institutions such as educational establishments or companies are reluctant to face the idea of ​​noise beyond wanting to move forward and make any decision if it is good or bad, well, I think that first of all they are not aware of the amount of noise there is, I think everyone already knows it. Whenever we show the results of a noise audit to an organization or simply share the results of noise audits performed at other organizations, people are very surprised. We could hear the shock in the room when Denny mentioned the insurance company's 50% figure. very surprising, so I think the obvious answer, I mean the first and most important answer, is that they just don't know and that's part of the reason we wrote the book, but in many fields they are using the use of algorithms.

Actuarial work, which is obviously in the insurance field, you know, humans used to do all the risk calculations and now a proportion will be done by a computer, but as far as I know, they still ask humans to take the final judgment because we don't do it. I don't want to give away that that power, uh, clearly depends on the topic. I mean, there's a long history of about 70 years of comparing people's judgments with formulas and, more recently, with algorithms generated by Ai and humans not. It is very good in those comparisons and the main reason for human inferiority is actually noise because algorithms and simple or complex rules have the advantage of being free of noise and that gives them a real advantage when it comes to accuracy , but it's not like that?

Obvious counterpoint to that algorithms can make things worse because they will have inherent biases of whoever wrote the code, although there is no reason why those biases should be worse than the biases of humans again, it depends on what the comparison is. . The point is you see algorithm bias is a big topic, it's being discussed everywhere and it should be because it's real, but the B of algorithms is there because algorithms are trained on data which is past decisions and the past judgments of humans, so what? what algorithms do is that they are the mirror of our own biases, they are no worse, which makes the bias more visible when you actually have an algorithm, it's two things first, you can run a million decisions on the algorithms and see How many biased decisions do you know? there are things you can't do with a human being, second, the algorithm is noiseless, as Denny pointed out, it is very consistent in its biases, so if you train your hiring algorithm and tell the algorithm, these are all the people that have been promoted at my company find out what it takes to be successful at my company and the algorithm comes back and says you have to hire men, not women.

Is the algorithm sexist? No, you're the reason you hadn't realized because you're loud in your sexism, even the most sexist recruiter will occasionally hire a woman, the algorithm won't do that, the algorithm will say, well, it's pretty clear that here to be successful you have to be a man, so the absence of noise makes the bias more visible. It doesn't actually make it worse, it also makes it easier to solve. If you want to solve it, you can design an algorithm that is free of such biases, so that the algorithms can make very good decisions. Do we want them to do it?

That is a very different topic. Yes, when you know that algorithms are biased, biased algorithms are poorly constructed algorithms, and usually the bias comes in somewhere in the definition of what it is that you're using as a success criterion, so e.g. , Amazon tried to develop an algorithm to replicate their high decisions and then that algorithm turned out the way they built it: they looked at people who had been hired or had been rejected and looked for what set them apart and then they discovered that there was bias bias. gender because people had, in fact, preferred men to men. women in your hiring and that went into the algorithm, that is a flawed criterion, that is, when you want to hire properly, you should not follow previous hiring decisions, then you are guaranteed to replicate the bias, but when you do it correctly and then the bias is not a necessary characteristic of the algorithm uh, although, as Olivier pointed out, the biases of algorithms will always be more visible because they will be more detectable because they are not masked by noise, while the biases of humans are very common. away from the noise I'm starting to think you think experience is overrated, is that a concept?

We shouldn't even talk about experience in the field if you want us to say oh, we've had enough experts, we're not going to say no, there's an interesting question when you think about experience, how do we evaluate experience and that is that a lot of the judgments when We look at experts are not actually verifiable, that is, there are judgments that we make as long-term forecasts that cannot be verified. And yet one might think that people who specialize in these types of judgments know how to distinguish a good professional from a weaker professional, and it turns out that there are ways for people to emerge as experts.

We call them experts in respect because they are experts. because they are respected not for the quality of their performance because their performance cannot be evaluated so there were expert astrologers and that is very useful to remember that some ERS astrologers had the respect of their peers and were taken more seriouslythan others and operated as experts and what creates expert respect is self-confidence eloquence

intelligence

I mean many properties that we want but they do not guarantee good performance good performance requires some verifiable criteria in some way some feedback on the accuracy of the drug made now if you are a chess player if you are a weather forecaster if you are an investor your experience can be measured against objective benchmarks and we can decide after a while whether you are an excellent weather forecaster or a bad weather forecaster because we see how precise you are, these are not the respectful experts, they are the experts, you can see the results, yes, so a good part of the book is actually a kind of instruction manual and by that I mean it tells how to detect and reduce the noise give us some advice what if I decide this is something really bad I want my decisions to be more efficient to be more precise what do I do so we've talked about algorithms and the basic idea of ​​algorithms is, as Denny pointed out, wherever there is judgment There will be noise, if you want there to be no noise, eliminate human judgment, let's leave that aside because there are many situations where it is impractical or undesirable, whether for good or bad reasons, to use algorithms and you will want to use human judgment when Going to use human judgment, the approach or set of approaches that we suggest. reducing the noise in human judgment is what we call decision hygiene and the reason we use this strange phrase is because the analogy of washing your hands is appropriate here we, when we wash our hands, we don't say oh, this was this germ that I removed and this is this disease that I have avoided we do not know what problem we have avoided we only know that it is a good prevention we are putting the process under control and decision hygiene puts the decision process puts the judgment process under some type of control , how do you do that?

We've talked about adding independent opinions, that's a good way to introduce hygiene into decisions, making sure these opinions are independent, if you add opinions through discussion, as we pointed out, you actually do more harm than good to another. One way would be to structure your decisions, but another way would be to use relative judgments instead of absolute judgments. Wherever possible, there are a whole host of things that, together, help introduce a little more discipline into the decision making and I should point out here that when we talk about noise reduction or decision hygiene that we had when writing the book, we had it in mind for organizations that the book is not self-help for individuals.

I am quite skeptical about the ability of individuals to improve their own thinking, but organizations have procedures, they have processes, it is possible, although this is not typical, there are few organizations that I know of that have designed processes to reach judgments or decisions, but we believe that more organizations should have designs for making decisions and designs that embody or incorporate the principles. of decision hygiene so that organizations have an opportunity that people don't have in the beginning you described that insurance agency underwriters understand realizing how much noise there was, but okay, so you identified it, but is it an expensive process and hard?

Do you think there will be a reluctance to identify the noise? Well, you probably know this, partly because noise is so abstract that since you can pinpoint an error to a single error, it looks like a bias and is easy to explain, since Olivier mentioned noise earlier, there is not a single error unless It may be a complete outlier, but normally you can't point to an error and say this is noise, you need more than an error, you need the variability of the errors, this gives noise that kind of abstract. character that makes it quite difficult to conceive and it is true that when you try to mitigate the noise in an organization you will surely face resistance and you will face resistance that is partly fully justified because there is the danger of bureaucracy existing. of imposing procedures that become mechanical and that make people less involved in the judgments they make, so there really is a balance between design decisions and spontaneous decisions and it is up to individual organizations to find their way in that balance and before even reaching the noise.

Reducing just awareness of noise is something that some organizations will resist, and many organizations will resist because noise embarrasses them. We've talked about a problem with noise, which is that it is expensive. We have talked about another one, which is that it creates injustice. There is a The third reason why noise is harmful is that it damages the credibility of the organization if at one insurance company you realize that you could get a quote for $100,000 and the next customer with the exact same need could get a $2,200,000 quote that doesn't reflect very well on the organization and it's very tempting to move on and talk about something else and I can tell you this is what happened at the insurance company, they didn't know about the problem and once they I think they forgot the problem very quickly, but does that perhaps reflect that going back to this idea of ​​human agency, that the power to exercise discretion can be much less efficient, but in reality within a company within a hierarchy it is a Necessary job? -off I'm the best, I know what I'm doing and I'm going to tell you, it's very tempting.

We had an interesting conversation yesterday with a senior HR executive who said, "You know, we read everything." That's what you're talking about and it's clear that it would improve the odds of hiring the best people, but as you point out, hygiene is a little tedious, it's a lot more fun to hire people I like and if that's what you're going to do, it's fine. good. it's your business, if I were your boss I'd probably raise some questions but I'm not so do whatever and there's another reason that really comes up and that judgment is typically, and very often, very fallible e.g. in hiring with real precision. in hiring is impossible because future performance is not completely predictable.

Many things will happen at work that can be known in advance and that are not characteristic of the individual and that will determine the individual's performance to the best of his ability. What we do in recruiting is pretty poor, but you can do worse than that, and that's what people do, but what you get in response is quite often good, if all you can promise me is that the performance will be quite Poor thing, I might as well trust my God. and that is a mistake