Lawfare Daily: Prophecy, Prediction, and Power with Carissa Véliz

[Intro]

Carissa Véliz: We use numbers when we don't trust people, and we forget that people come up with the numbers, and the numbers are not as objective as we would like them to be. And big tech is just the latest iteration of that.

Tyler McBrien: It's the Lawfare Podcast. I'm Tyler McBrien, managing editor of Lawfare, with Carissa Véliz, an associate professor at the Faculty of Philosophy in the Institute for Ethics in AI at the University of Oxford.

Carissa Véliz: It's very easy to have the temptation of, "I just want to know. I just wanna be prepared." But that kind of attitude terrifies us. It pushes us into asking prophets to essentially tell us what to do, and, and what we're saying is, "I'm scared. I don't know what to do. Just tell me what to do." And that's very dangerous because it's giving away our power.

Tyler McBrien: Today, we're talking about Carissa's new book, “Prophecy: Prediction, Power, and the Fight for the Future, From Ancient Oracles to AI.”

[Main Podcast]

So Carissa, I think it's no understatement to say that this is a very relevant, timely, dare I say prescient book in some ways. I, I mean, I'm just thinking about w- the World Cup, for example. There's, every other ad seems to be for a prediction market. Animal oracles have been known to predict the outcome of sporting events like this. There's Paul the octopus a few years ago. So I wanna start, though, with, with your interest in the topic. How did you land on this really wide-ranging, you know, far-reaching idea for a book and especially how it relates to, to your, your past work?

Carissa Véliz: It was a confluence of things. So on the one hand, realizing that the kind of AI that we use, machine learning, is nothing but a prediction machine, and realizing that we haven't done enough work on what exactly is prediction and, and especially the ethics of prediction. It's kind of astonishing, if you think about it, that we've been using prediction for thousands of years, and there are thousands of books about prediction and academic journals and, and, and yet there is, there wasn't one book on the ethics of prediction, so realizing that there was a, a big gap there.

But also, I advise governments and companies around the world on, on issues about digital ethics, and realizing that even without AI, so many decisions that are very important having to do with justice depend on prediction, and sometimes in very questionable ways. And that led me to look into the history of prediction, which in turn led me to the origins of Western philosophy as a kind of reaction to a very kind of mythical culture that was heavily invested in prediction. And so it was all of those things coming together and thinking, "Yeah, there, there's a book here."

Tyler McBrien: Yeah, and it's, it's honestly a, I think a paradigm-shifting book. At least it was for me. You know, I, I think Western democracies, especially with more of a liberal tradition, like to delude themselves into thinking that we're rational thinkers and we've moved past a belief in prophets and, and oracles. But not so.

But before we jump into the history, I think it's helpful to lay out the, the characteristics of prediction and, and what we're talking about here when we say prediction or forecasting or fortune telling. So could you lay out a bit about the, the bounds here?

Carissa Véliz: Yes. So a prediction is a statement about the future. And one of its characteristics is that it seems like it's a description about the future. So something like tomorrow it will rain, or this candidate will win the election, or we will be using AI in this and that way in the future. So it's a kind of statement that very much sounds grammatically like a description, and it's supposed to be, to tell us something about the world in the future.

But actually, when you analyze it in terms of philosophy of language, you realize that it's a very misleading kind of assertion. Because even though it sounds like a description, facts, by their very nature, belong to the past. And so a prediction can never be a fact, not even in very scientific context. So, at best, a prediction can be an educated guess or a kind of hypothesis that you then either confirm or, or deny.

But more often than not, when you look at how we use prediction in the public sphere, you realize that it's often a kind of marketing, a kind of wishful thinking, and very, very often, too often, a kind of power play in disguise. Because the person making the prediction makes it sound like what they're saying is a fact when they're in, they're trying to bring about that future because they often have a, a kind of financial or other kind of interest in it.

Tyler McBrien: Yeah, and we see this, of course, with market manipulation in the prediction markets. I wanna just briefly dwell on the term “speech act” here to pick up on the last thing you said. I think some listeners will be familiar with that term in international relations theory, but how does the, how does the concept of a speech act play into this?

Carissa Véliz: The concept of speech act comes from the philosopher J. L. Austin, who in the 1960s gave a talk at Harvard that then become a book, became a book, that is called “How to Do Things With Words.” And the main argument of the book is that there are sentences that don't describe the world, but rather act in the world in a, in a particular kind of way.

So when a civil servant marries two people, they're not describing the state of the world, they're, they're marrying two people. And when you tell your child that they have to clean their room, you're not describing the state of the room, you are issuing an order or a command.

And many predictions, even though they sound like descriptions, are actually veiled commands. And even ones that have to do with apparently objective things. So even if I tell you "Tyler, it's going to rain tomorrow," what I'm actually saying is, "Take your umbrella," or, or, "Cancel that event," or, "Postpone that."

And even more so when it comes to social matters, because we are so, such suggestible creatures that when somebody tells us something about the future, we tend to be affected by it, and it creates different expectations. So if I tell a student, "I think you're terrible and you're gonna fail," it's a very different kind of act than if I tell them, "You're brilliant, and you're gonna do great, and, you know, don't worry about it." And both of those are very likely to have an effect on that person.

Tyler McBrien: And it depends on who's doing the speech act, right? So if, if we've already elevated the Oracle to this mythic status and believe them, then I assume what they say will have more of this power. Is that a fair read?

Carissa Véliz: That's exactly right. And we see this with financial institutions. So when a financial agency says that a country's gonna be in trouble, you bet the country's gonna be in trouble. Even, even if nothing else, because that statement will make investors flee.

And the same thing when it comes to a bank making a, an assertion about the future, or these days tech executives are very prominent prophets that tend to say a lot of things about the future. And they get a lot of airtime on the press and with politicians, and they get to influence the future in many ways, not only through the gadgets that they design, but also through the assertions they make about the future

Tyler McBrien: And of course, this is not a new phenomenon. So I wanna transition into some of the history in the book, because I think that's some of the richest anecdotes that, that came out.

So, in looking at the, the first part of the book going back to, to Rome and other periods of history, well, first of all, I'm just curious how you scoped this. I mean, there's, humans have been predicting things or expressing anxiety about the future since we've been human. So how did you kind of decide where to, to bracket the history, and, and can you walk us through some of some of the history in the book?

Carissa Véliz: Yeah, so as, as soon as I realized that there was a connection to ancient philosophy, that was a, a clear anchor for me. First, I started reading ancient philosophy, but also history of ancient Rome and ancient Greece. And of course, the Oracle of Delphi is a very important figure, and I started there.

But there was also this quirky situation that I recount in the book that I was in an Oxford event. There are a lot of events in Oxford, and I ran into the librarian of the Bodleian, which is the, the library at Oxford, and he's, his name is Richard Ovenden, and he asked me about what I was writing about, and I didn't really wanna say, but I was, you know, said something about prediction, and he was like, "Oh, you, you know, we have an exhibition coming up in the Bodleian about prediction." And the exhibition was about ancient oracles. And so I went to the exhibition, and one of the things that stood out was, first, that prediction has always been a business. Prophets are merchants of predictions. And the second thing is that we always ask for the same things. Whether it's now or 2,000 years ago, we care about our family, our health, the political situation in our country, and our business. That's, that's basically it. That's it.

And so that also became a kind of n- narrative thread of like, okay, if prediction is a business, like, how did it change? And how did, how did we go from predictions based on qualitative issues like, you know, animals, entrails, and, and, and, you know, the manifestation of Apollo to predictions that are quantitative, that are most familiar to us today? And that led me to the history of insurance and the history of statistics, and it was a fascinating book to research. Quite hard because predictions span so much, so it took a lot of re- research. But it was a, a wonderful kind of journey.

Tyler McBrien: Yeah, I mean, the book is so many things. To cherry-pick from a, a passage of yours, you say it's a technology book, a business book, it explores the history of prediction. It's a political treatise, a book about climate change, a defense of literature. But then you say at the end, most of all, it's a book about how to live well, which I think the, the structure of the book and, and, and the free-ranging quality also speaks to just the free-ranging quality of the, of the topic itself.

I wanna, I wanna drill down on the, the tech criticism, the big tech, big data criticism at the heart of the book especially because it connects well to your past book on privacy and surveillance technology. Talk about, more about that transition from qualitative to quantitative prophesizing. What, and, and if you could just explain a bit about what's distinct about this critique of big tech versus others that listeners may be familiar with in the discourse.

Carissa Véliz: Yeah. A quick comment about, about how broad the topic is. It's also bec- about philosophy. I think one of the drawbacks of specialization in, in academia is that we end up with a lot of people who are extreme experts in this very, very, very narrow thing, and very few people who can connect the dots, and sometimes that leads us to lose perspective and to have inc- incredible gaps in knowledge, like no knowledge of the ethics of prediction.

And I think philosophy is particularly well-suited to be a kind of discipline that can connect the dots, and so that's part of the hope of the book. But also, I find myself in some instances giving talks to audiences that are very specialized, and I'm not just saying this, but one of my favorite audiences is lawyers because we have a particularly, I don't know, k- kind of akin minds of, like, liking premises and conclusions and, and, and logical thinking.

Tyler McBrien: Wow, you're really pandering to the audience here as well.

Carissa Véliz: But it's true. It's a really kind of pleasant audience to talk with. However, sometimes lawyers get stuck in the laws that we have and not in the laws that we should have or in the ethics that are behind the laws. And so getting the bigger picture of not only data protection, but, like, what is privacy and wh- and what is connection to, to prediction, I think, is very important to understand the role that you're playing in society and the meaning and the weight of your job. And, so that's also part of the hope of the book.

And then, yeah, going back to this transition of v- you know, from qualitative predictions to quantitative predictions, one of the interesting aspects of it is that it's a, it's a way of justifying our decisions. And, you know, in ancient Greece and ancient Rome, going to a priestess was a perfect way to justify your decision, but when Christianity kicks in and then we have this process of disenchantment of the world, suddenly going to a priestess doesn't quite seem to cut it.

And then we have a situation in which bureaucrats are in a kind of limbo because they don't have divine right to justify their position anymore, and neither have they vote- been voted in. So they have to justify to citizens, like, who the hell are you and w- and what, what do you do?

And I had always assumed that the development of statistics had mostly been a mathematical development by mathematicians, and it wasn't. It was mostly a practical solution to the problem of justification, and very much pushed by, by bureaucrats. And so essentially, if, if I were to distill it into a sentence, we use numbers when we don't trust people, and we forget that people come up with the numbers, that the numbers are not as objective as we would like them to be.

And big tech is just the latest iteration of that. It's just the latest iteration of prophets who need to justify their decisions, and who come up with often very opaque methods and very opaque machines that partly create that function of shielding them from accountability and saying like, "Well, it wasn't me. It was the algorithm." Then the algorithm is the subjective thing that is based on prediction, and predictions are knowledge. And that's completely wrong. No, predictions are not knowledge. They're more, they belong more to the, to the sphere of power. And at the end of the day, human beings are the ones who are the responsible agents here.

Tyler McBrien: What has been the reaction from big tech? I could almost see this ironic scenario where people reading this might try to turn you into a prophet. You know, that this is n- the new answer or something like that. But yeah, what kind of, what kind of reaction have you gotten from some of the subjects of your book?

Carissa Véliz: It's been very exciting because you never know. You, you spend years writing a book pretty much on your own, and then you never know how it's going to be received. And it's been very nice to see excellent reviews in The New York Times, in The New York- New Yorker, Wall Street Journal, Financial Times, Economist, is exactly what an author wants to see. And from people around the world and very different professions from lawyers to people who work in risk management and, of course, philosophers, but also teachers and businesspeople.

From big tech, there's been pretty much silence so far, which is not surprising. However, big tech is very big, and within big tech, there are people who are interested in different points of view and who are eager to engage.

But I'm more excited about the response I've, I've received from smaller alternative businesses who are excited about doing things differently, who want to innovate, who don't have the, this paradigm of domination and, and that makes me more excited. And also from young people who are the future. But you're right that there is this temptation, especially with the press, to end, end, especially end interviews asking me about, "And what is the future?" It's like, it's such a habit.

Tyler McBrien: So because this is the Lawfare Podcast, I'm curious how these ideas apply to the law, another discipline very known for its attempts at least to, to make order out of the world.

Carissa Véliz: I've been thinking a lot about the law in this project, partly because very important figures in the history of statistics and, and thinking about statistics and numbers were very interested in the law, like Laplace and Condorcet and, and others. And I'm worried about how much we're importing probabilistic thinking into law because I think that law should be more about principles, about justice and truth, and probabilistic thinking is in tension with all of those.

So for example, in the UK, you might have legal insurance, but your insurance is only going to cover your case if they think that you have a 51% chance of winning. And of course, you can understand why they do that, and it makes financial sense, but the implication is that the bad guys don't have to make it hard, much less impossible, to fight them. They just have to make it slightly unlikely, and it's very rare for somebody to come after them. And for them, they know this. It's a calculation.

And so we are designing society in a way to disincentivize people pursuing justice, and that's kind of a recipe for things not only not improving, but getting worse. And similarly, when, for example, police use predictions about how a jury will see a case instead of allowing the jury to actually see a case and make a decision for themselves, we are going against justice.

And similarly, when we use algorithms to determine questions about sentencing and bail, we make it very hard for justice to prevail because whereas if, if you make a decision based on criteria that are clear and contestable and based on evidence, there can be a rational discussion, and the person being accused can defend themselves and navigate that system. But when you deny someone an opportunity based on a prediction, there's no way to argue against that because predictions are not facts, because they're not verifiable. They're not falsifiable.

So we're kind of creating this Kafkaesque world. And so something that I would like to ask the lawyers of the world is to encourage their clients, their political representatives, policy makers to try to design things in a way that push people towards being more principled and, and incentivizing people to pursue justice.

Because part of what makes human beings human beings and part of what makes the world a better place is people fighting battles that are unlikely victories. If we only fight the battles that we know we're gonna win, it's a recipe for disaster. We need people to know that even though they have a 1% chance of winning, they should still fight that battle even if they lose, because it's the right thing to do.

Tyler McBrien: I'm curious what you would say to someone pushing back on an argument that… And I'm, I'm taking your argument to the extreme. So say that that the current, our current algorithmic predictive machines are simply just another iteration of the Oracle of Delphi and is it, is it not though that, that we're just sort of trying to hone these predictions or using data to improve them with still some small recognition, some more than others, that the future is inherently unpredictable, we're just trying to narrow the universe of possibilities?

Yeah, I, I, just this, this idea that, you know, you write that “to be resilient, prepare, don't predict.” But, but why, why couldn't, you know, people do both, for example?

Carissa Véliz: I encourage people to do both. So one of the examples I give is, you know, I, I look at my weather app every day and I, you know, I can't live without it. And that is very much a, a prediction.

So, the way I would respond is showing that, yes, the, the picture is more complex than that. It includes more phenomena. And of course, prediction is a very important part of science. But when done well, it is put in a scientific context. Prediction by itself does not equate science, with, with science.

And so for example, one of the best uses of AI that I know of is using it to try to find new materials or new m- antibiotics or, or try to find, it's actually trying to predict how molecules are going to interact with one another. And that's a brilliant use of it because it narrows the field of possibilities as you, as you put it, and then from that narrow set, which saves you thousands and thousands of hours in the lab, you actually test those molecules that seem promising in the lab and then y- you have the, the usual process of randomized control trials and peer review and all of that, and that's what makes it scientific, not the use of AI.

AI can be used in very unscientific ways, and so what I worry about is that with the pretext of using prediction in helpful scientific ways that mostly have to do with predictions about things and not people, because predictions about things don't affect things. So if I predict that it's gonna rain, it's either gonna rain or not, and it doesn't matter what I predict, and that's different from making predictions about people, that we use that excuse to use prediction in very unscientific ways that have to do not with scientific advancement, but with the advancement of authoritarian tendencies.

And so one of the things you, you said is, is very important because it gets sold as that, as, well, we're trying to minimize uncertainty. And the assumption there is that it's a good thing to minimize uncertainty And it depends. So it's, it's a good thing to minimize uncertainty when it comes to the weather, sure. But just think about what it would mean to you to know exactly what you will be doing and where you will be in a year. It would actually be a, a, a tragedy because it would mean that your destiny is sealed. There's nothing to do, and you can- you just have to accept it. You, it would mean that you live in a police state.

So, you know, in the extreme, one way to predict exactly where somebody's going to be tomorrow is to jail them. That's a very effective way of predicting the location of someone. We see this phenomenon throughout history, that when we become better at predicting the behavior of human beings, it tends to be a very bad symptom because it means that we are creating that behavior, not that we are discovering a script that's already written, but that we're creating it.

Tyler McBrien: Yeah, I think another great example that you bring up in the book you write, "Democracy needs uncertainty to thrive. It's only when we don't know the outcome of a future election that we have democracy." Which I think is such a perfect encapsulation of this connection between prediction and power, and yeah, the best way to predict the, the future is to control it, and that's what we see also in, in prediction markets as well.

So, so I am, I'm interested in this, this idea of being comfortable with uncertainty and even delighting in it and running toward it. So could you speak a bit about some of the latter chapters of the book that touch more on, you know, how to live and how to live well?

Carissa Véliz: One of the things that makes us most vulnerable to charlatans and to prophets of all kinds is our fear of the future. And of course, it's entirely understandable, and I feel it just like the next person. It's nerve-wracking not to know what's gonna happen in a month, and there are moments in history where things seem particularly uncertain, when geopolitics is hot, and when scientific advancements are changing things. And it's very easy to have the temptation of, "I just want to know. I just wanna be prepared."

But that kind of attitude terrifies us. It pushes us into asking prophets to essentially tell us what to do. And, and what we're saying is, "I'm scared. I don't know what to do. Just tell me what to do." And that's very dangerous because it's giving away our power.

And another perspective is to be excited about uncertainty, to realize that the minimization of uncertainty when it comes to social realities would be horrific news, and that uncertainty means that the future is not written, that we can intercede, that we can partly build it and mold it. And so instead of being afraid of it, saying, "Wow, I ha- I have an opportunity here. I'm given a blank page or a, or, or a partially blank page in which to write in," and that's incredible. That, that is fantastic news.

And so what I would like to see more in the public sphere is rather than have prophets telling you what the future is and just telling you, "Just deal with it and adapt to it and say as I do," to say, "Look, I've thought about this feature and I think it would be really cool. Do you wanna build it with me?" And to have that attitude of collaboration, of this is, the future is something that we build together, that we decide together.

Maybe a, a couple of two or three elements that I particularly enjoyed writing about is one, having this curiosity and attitude of curiosity towards the world, towards other people, and how that is a kind of antidote to the anxiety we feel about uncertainty. If you think about life as a good book and as an adventure, you get that sense of I wanna keep on reading. I don't wanna find out what happens at the end. I want the story to go on not to end too quickly.

And a second element that I enjoyed writing about, and I think is more important than meets the eye, is humor. I think comedy is a very important antidote to anxiety, but it's also a very important element of democracy, is a way for us to question power, to be irreverent towards power, to create freedom. But it also is very conducive to creativity and to all kinds of, of positive thinking.

And then maybe the last one is I end the book by suggesting 10 guidelines that might be practical. Because I want people, whether it's an ordinary citizen or a lawyer or a policy maker or a business person, to go away knowing roughly how this applies to their own life. And maybe my favorite guideline is to try to seek more exposure to serendipity. The more we allow algorithms to decide who we meet, where we eat, what we watch, where we go, the less autonomy we have, the le- the, the more we narrow our agency.

And so it's good to try to get exposed to ideas that you maybe wouldn't have thought about. So go to the library or a bookshop and ask for recommendations. Talk with strangers. Yeah, read widely, and just take strolls along the beach because you never know what the tide might bring.

Tyler McBrien: It's funny, the especially the humor point, it had me thinking that the, the people most associated with prediction, at least today, tech CEOs, insurance people, actuaries, aren't typically people known for their sense of humor. So I think there's definitely a strong connection there.

And, and I think it's right. I mean, there's, it's human nature to be anxious about the, the future and, and a lack of control. But it's also human nature to, to hate a spoiled ending. I, I completely agree. I mean, the, the, if you knew the end of something, it's, there's a lot less reason to, to undertake it.

As I was reading your book well, several books came to mind. One was the book “Charlatans,” which is a recent book by Kiko Toro. And, and so of course the, the charlatan's use of, of prediction was front of mind. But another person I interviewed on the podcast recently is a professor named Joey Kellner, who writes about how when societies are on the verge of collapse or in a very uncertain time, that the people living in them look for answers in very unlikely places, in fortune tellers, in astrologers and these types of things. Is this something that you looked at in, in terms of the, the nature of the unpredictability of a society and then an increase in this faith or seeking of, of fortune tellers?

Carissa Véliz: Yeah, and the worst part is that it tends to be a vicious cycle because the more things seem uncertain, the more people panic and then go to astrologers and prophets of all kinds, and the more they lose power. And then the more they create also uncertainty because when we try to control reality, it's like a teenager that rebels and we create even more uncertainty than, than what we had before.

And so one historical example is in ancient Rome when the republic was faltering, a lot of people went to astrologers and astrologers started predicting the coming of an emperor. And that was the death of the republic. When people started believing more in the future that was an empire than in the present that was a republic, in a way they had already given up on the republic. They had given up on their right and their duty to build the republic that they were supposed to defend and instead they just accepted the imperial chart of someone.

Tyler McBrien: I'm thinking a lot about the, the reaction to AI's ubiquity these days. So, you know, you see this flattening of aesthetics and the antidote to that is people are returning to very sketchily hand-drawn aesthetics. There are, you know, memes out there of, of images that AI could never create. So this sort of celebration in the, the oddness of the human experience. Did you find any, any good examples of this? Of, of people, you know, revolting against the, the, the tyranny of prophecy?

Carissa Véliz: Well, one excellent example is ancient Greece and philosophers revolting against the Oracle of Delphi and so on. So I think it's not a coincidence that philosophers like Anaxagoras and Socrates and Aristotle were accused of impiety. There's a, there's good reason for that.

But I think pretty much every hero that you can think of in history, whether it's Rosa Parks or Marie Curie or anyone that you can think of, was someone who defied their odds, who got told, "You're never gonna succeed." And when we, we think about the victories of humanity, like for example, the Universal Declaration of Human Rights or universal suffrage or, you know, whatever victories we've had, all of those at some point seemed impossible, and the people defending them got told that it was impossible. So I remember reading about the economics of slavery and, and how many economists thought that it was just literally impossible to abolish slavery, and of course they were wrong.

So, every important victory is a kind of defiance of the odds. And when it comes to algorithms, you're right. I think we're seeing people rebel in, in certain ways, and one example that comes to mind in the UK a few years ago, when the pandemic kicked in, students couldn't sit their exams, and so the government decided to use an algorithm to predict what they, what their grades would be. And you can imagine how this wasn't very attractive to the students, and it was, I think, one of the first times that we saw a protest in which the chant was, "Fuck the algorithm," by students.

Because mostly when we encounter injustice and that it's perpetrated through an algorithm, people mostly don't realize it was an algorithm or, or they're so isolated that they can't go out and protest easily. You can't prove that the algorithm was wrong easily. But because this was a very collective and very public case, people got angry. And, and in fact, the government had to backtrack, and we ended up pretty much changing things so that we accepted many more students than, than we would otherwise have because, because people protested.

Tyler McBrien: So for the students out there who are relying on a, on a, on a spectacular final exam grade to save themselves, do it. Defy the odds. I am curious what you're thinking of working on next, or if you have another book project in the works, especially because the, the connection between your previous book and this one is, is so clear. I'm, I'm curious, you know, where else are you taking your research agenda, essentially?

Carissa Véliz: I do. I'm already writing, but I'm not gonna tell you about it. You're gonna have to find out. Wait and find out.

Tyler McBrien: And I will be okay with that uncertainty.

Carissa Véliz: Thank you.

Tyler McBrien: Carissa, thank you so much for joining me. This was a really, really great conversation.

Carissa Véliz: Thank you so much, Tyler. It's been a pleasure.

[Outro]

Tyler McBrien: The Lawfare Podcast is produced by the Lawfare Institute. If you want to support the show and listen ad-free, you can become a Lawfare material supporter at lawfaremedia.org/support. Supporters also get access to special events and other bonus content we don't share anywhere else. If you enjoy the podcast, please rate and review us wherever you listen. It really does help.

And be sure to check out our other shows, including Rational Security, Allies, The Aftermath, and Escalation, our latest Lawfare Presents podcast series about the war in Ukraine. You can also find all of our written work at lawfaremedia.org. The podcast is edited by Jen Patja, with audio engineering by Cara Shillenn of Goat Rodeo. Our theme music is from Alibi Music. As always, thanks for listening