1.5 – Artificial Instincts: Playing Poker

Transcript
Subscribe
All AI: Hype vs. Reality Podcasts

Does AI have intuition? It’s tough to say. Some experts predict AI will take the guesswork out of stock markets, come out on top in high-stakes competitive games, and figure out what we’re thinking before we think it. But will it? And how soon? AI’s capacity for augmenting human intelligence is already helping humans making more educated guesses in several different arenas, by looking at wide swaths of historical data and trend analysis. But when it comes to poker, can AI trust its gut?

What you’ll hear in this episode:

  • The difference between poker and chess
  • How intuition moves through a neural network
  • What exactly is intuition?
  • Is intuition just a logical algorithm?
  • Will there ever come a day when there’s no reason to watch sports?
  • The interesting way AI is learning to play Pictionary (with your help)
  • A brief overview of Game Theory
  • Jessica tries her hand at Hold ‘Em.

Guest List

  • Dave Graham is the director of emerging technologies messaging at Dell Technologies and specializes in AI and social transformation.
  • Tuomas Sandholm is the Angel Jordan Professor of Computer Science at Carnegie Mellon University as well as co-director of CMU AI. Sandholm led the team that developed the AI program that beat the human champion in heads-up, no-limit Texas Hold’em poker. Sandholm is also founder and director of Electronic Marketplaces Laboratory and founder and CEO of Strategic Machine Inc., Strategy Robot Inc., and Optimized Markets Inc.
  • Aniruddha (Ani) Kembhavi is a senior research scientist in the Perceptual Reasoning and Interaction Research team at the Allen Institute for Artificial Intelligence. Kembhavi was the research lead that developed a pictionary playing AI.
  • Raphael Fiorentino is the CEO of Butterwire, an AI run investment firm.
  • Victor Kristof is one of the co-founders of Kickoff.ai, a startup that uses advanced mathematical techniques to ‘encode’ the current strength of teams and project outcomes.
  • Geoffrey Hinton is an engineering fellow at Google, an emeritus professor at the University of Toronto and the chief scientific adviser of the Vector Institute, which researches machine learning. Hinton is considered a pioneer in the branch of machine learning referred to as deep learning.

Jessica Chobot: I’m Jessica Chobot and this is AI: Hype vs. Reality, an original podcast from Dell Technologies. And I’m at Carnegie Mellon University to play poker against a world champion AI, and get this, I’ve never even played poker before, so this is going to be real good.

Jessica Chobot: Hi, Jessica, nice to meet you.

Tuomas Sandholm: Hi, Tuomas Sandholm, nice to meet you.

Jessica Chobot: So you’re working on something called Libratus. So what is that about?

Tuomas Sandholm: Yeah. Libratus says the AI that became the first and only superhuman AI for heads up no-limit Texas hold’em poker.

Jessica Chobot: Why would Texas hold’em poker be the main crux out of all the games that are out there?

Tuomas Sandholm: Yeah, so unlike, let’s say chess or go, it’s a game of imperfect information, because when you have to make a decision, you don’t know the state of the world and your opponent or opponents might know things that you don’t know.

Jessica Chobot: I’m going to pretend like I know what we just talked about, but it sounds great and I’m ready to go up against the AI.

Tuomas Sandholm: Oh this is perfect, we have four cards of diamonds. If we get one more diamond, we’ll have the nut flush. On the other hand, and if we have … if we don’t get the diamond, we really have nothing here.

Jessica Chobot: I say, just go for it.

Tuomas Sandholm: Go for bet?

Jessica Chobot: Go for it, yeah.

Tuomas Sandholm: Okay. Three quarter pot.

Jessica Chobot: But hold on. Before we find out who won that poker match, let’s unpack the hype around AI and intuition.

Jessica Chobot: Hey, I will take all the guesswork out of doc market investment and will correctly predict the outcome of any match after defeating humans in the most complex board games. AI will conquer ESports. AI will figure out what we’re thinking before we think it. All of this, of course, per usual will happen any day now.

Jessica Chobot: So to sort the hype from the reality, I’m here as always with Dell Technologies, emerging tech expert, Dave Graham. So Dave, how does the AI that I played chess against on my PC, like 10 or 15 years ago compare with the AI’s that are out there right now?

Dave Graham: 10 to 15 years of cumulative data analysis, that kind of increased compute capability, the technology advancements, so on and so forth, allow computers to process information even faster than they ever did before. We also have 10 to 15 years more chess acumen that we are able to add into the fold. And again, remember AI is constantly learning. We’re constantly feeding it data. So as we sit there and play these games, AIs are observing. So here are some of the thoughts of deep learning pioneer, Geoffrey Hinton and what he sees as the difference between playing games with brute force versus intuition.

Geoffrey Hinton: So after they’d managed to beat Kasparov at chess, people said, you’re not going to be able to do the same thing for Go. There’s too many alternatives to consider. You’re not going to be able to play Go unless you have the spacial intuition. So a Go master, will look at a board, and he’ll just know certain places that are good to play it. He’ll just have a strong intuition about that, and he wouldn’t bother to explore all the other places, whereas a brute force machine would explore everything, and there’s just too many alternatives. And the key was getting neural nets that had intuition. So the neural nets were trained to mimic the moves that a master would have made to begin with, and then the system played against itself, so it got even better. But the key was having intuition about where is the sensible place to consider, so you don’t have to consider everything. And that required a neural network. That had intuition, it wasn’t logical reasoning at all.

Jessica Chobot: Well, so that’s really interesting to hear, because I feel like, to me that just still sounds like logical thinking. That doesn’t sound like a feeling, and I’ve always defined intuition as being based in feeling.

Dave Graham: So let’s start out with the basic definition of what intuition is. Right? By definition, it is the ability to understand something immediately without the need for conscious reasoning. So, I’m able to act without having to rely on anything else, right? So a neural network will do this by virtue of, it receives some input and then it triggers off an action, right? It’s reaches something we call a threshold, and when that threshold activates, then it activates the next neuron in that series, and continues to go on that neural network, and it continues to go along that path. An input is received, I recognize a spot on the board, and I know that I can place my object, my player, my chip, whatever it ends up being. And when I do that, an action will result. Intuition leading to learning, that ostensive is the nature of things there.

Dave Graham: On the other hand, logic would dictate, I would try to examine every single option before I make a play, before I make a choice, right? And that it would evaluate every opportunity. This is what you see a lot in chess, everybody prognosticating and thinking about it, if I place X here, Y will result, right? You know, that kind of operation. So, this is truly placement without an understanding of what the end result could be, just that you can do it. So this differs to your concept of a gut feeling, right? That’s an emotive response. That’s basically letting the entire environment press down on you at one point and saying, I feel this way, therefore I’m gonna act in a way that’s neither rational nor based around anything to do with logic. It’s literally operating out of an emotional center.

Jessica Chobot: Well, okay, so speaking of gut feelings, one of the decisions that we might make with our gut is how we invest our money. So, have you been in one of these meetings where if financial advisors shows you a bunch of pie graphs and charts and then asks you what your risk tolerance is at all?

Dave Graham: Unfortunately yes.

Jessica Chobot: So, unless you happen to be really up on money markets, I think a lot of us then go with our gut, or like you were saying earlier, kind of an educated guess based on what knowledge we come to the table with and maybe what our advisor is telling us. And then, even if you do know the markets, like a professional stock trader and broker, I would assume that they also use their intuition as well, just at a more educated level, because they have probably more data that they can base that off on. So what is AI being used for then in the world of investment?

Dave Graham: So this type of work is actually being done right now with AI investment companies using bots and other types of technology. I heard from Raphael Fiorentino, the CEO and founder of Butterwire, an AI app for stock market investors. Because he doesn’t believe in just unleashing AI on the stock market, he’s come up with a different approach, what he calls IA or intelligence augmentation.

Raphael F: AI, is about the machine trying to be smarter than you could possibly hope to be. AI is being smart at making you smarter. So if you go along these lines, developing machines that help you get smarter faster and therefore make better informed decisions with your investment, then you stand a chance to claw back this huge knowledge deficit that exists between a segment of the investing population, a small one, and the vast majority who’s not. And I mean, and it’s tackling a really crucial issue, which is your pension, right? There isn’t a single professional investor that can absorb more than a few percent of the relevant knowledge that you can get at your fingertips on the Bloomberg terminal. So, it’s going to start looking and linking things that you did not know were relevant.

Jessica Chobot: Okay. I can, I can get behind that. I think that’s well stated. All right, so when we talk about following our gut, something else that seems somewhat non scientific is sports. I mean thinking that one team is going to win instead of another just because we might prefer or love that team. So when there’s so many different factors, so many variables like injured players, weather, coaching tactics, can AI really parse all of that stuff out?

Dave Graham: I think to a certain extent it can. Again, you’re looking at correlation. There is a tendency to be able to say, well we believe that since the Patriots won six out of the last, I don’t know what, 10 Super Bowls, there’s a good chance that they’re probably going to win the next year as well. In addition, I did speak with Victor Kristof. He’s a machine learning PhD student in Switzerland, and his team has creativekickoff.ai, a platform to predict the outcome of football matches and when he says football you mean soccer because he is in Europe. He says the software is all about turning intuition into a formula, like if you have a feeling that one team is going to win instead of another.

Victor Kristof: But it’s just a feeling, and now we actually have a quantitative way of putting a number on this outcome. We have a database of about 55,000 matches that spans over a hundred years of football data. We can take into account the uncertainty that is present in the data, which enables us then to have more accurate predictions, more accurate probabilities. There is some sort of predictability in those games, and that’s a bit what our model is trying to capture. We’re not predicting whether team A, or let’s say Barcelona will win. We say that Barcelona has 65% chance to win over Real Madrid. There is an inherent part of randomness in football matches and I think that AI will never be about to predict 100%, with a 100% accuracy, football games and fortunately, because otherwise there is actually no point in watching football anymore.

Jessica Chobot: This is so confusing to me. I’ll tell you this, this particular episode of the podcast series is the hardest one for me to wrap my mind around because it sounds like based on what everybody we’ve heard talk about, that maybe this podcast shouldn’t be about AI learning intuition, but AI proving that intuition doesn’t actually even need to exist, because it’s all just accumulated data and really we should get rid of intuition and just call it educated guesses, I guess.

Dave Graham: Yeah, a lot of it’s all predicated on data, how we feed data into these systems, how we get data out of these systems, how we appreciate that data, how we let it be used.

Jessica Chobot: Well that depresses me because I always make the wrong decision, so that must mean that my data is really, really poor.

Dave Graham: But you’ve learned from those wrong decisions, hopefully.

Jessica Chobot: Well, putting this to the test, I’m actually going to go play Texas hold’em against an AI opponent, and I’ve never even played poker. Before I get to that, I’m going to find out about AI and intuition in another game, which I do dominate at, which is Pictionary. So I will see you next time and we will have a very, very competitive game of Pictionary.

Dave Graham: Absolutely. I’m looking forward to that.

Jessica Chobot: All right, see you later. Dave.

Dave Graham: All right, bye.

Ani Kembhavi: My name is Ani Kembhavi. I am a senior research scientist at the Allen Institute for Artificial Intelligence in Seattle.

Jessica Chobot: Ani is part of the team that created Iconary, an online AI driven version of Pictionary. And the first problem they ran into, when building the game was the most basic, and AI can’t really draw freehand. So Ani and his team developed a library of icons that the AI uses, so things like cakes and mountains and pencils. But even with icons, the AI still needed a ton of training, which it got from watching humans play Pictionary.

Ani Kembhavi: What’s interesting is that we have gathered a lot of games of human players playing a closed set of phrases. So what’s interesting is that the majority of human players use a sort of small set of intuitions to depict this phrase. You know, each person has had different upbringings, has had different exposure. The very basic set of institutions, especially in this drawing and guessing game seem to be drawn from a sort of a similar source of common sense knowledge.

Jessica Chobot: And that’s the key to how Pictionary works. Even though we all have our own intuitive sense of how to play, we share to some extent a common set of images of icons, and that’s what the team at Allen is trying to teach their AI. But here is the thing, they couldn’t use the same kind of reinforcement learning that taught AIs to play chess and Go, where they just played each other until they get really good, because Pictionary is so subjective.

Ani Kembhavi: Because you can imagine if two agents play this game, one is given a phrase like a dog eating a bone, and the other one has to guess the drawing. You can imagine that these agents might get better over time, but they might decide that they’re going to use a cat icon to depict dog, and a hamburger icon to depict bone. And so, they’ll start developing a … some sort of a language where they can understand each other and get better, but then when you deploy one of the agents with the human, it won’t work out.

Jessica Chobot: Then again, just getting the AI to play against a human brings another big challenge. Even though the AI can see the phrase, family birthday party, and use the icons for people, cake and candles, what if the human partner in the game guesses happy birthday?

Ani Kembhavi: And so, when the human partner makes a wrong guess, how do you adapt its drawing to guide your human partner towards the right guess? This feedback that happens between the drawer and the guesser is one of the most challenging things we face from a modeling standpoint.

Jessica Chobot: And so Iconary was unleashed on the internet a few months ago to help it grow and learn.

Ani Kembhavi: One of the main reasons was we wanted to get a lot of people playing it because the more people played with Allen AI, the more data we get to push back into the system and help Allen AI improve its algorithms.

Jessica Chobot: So the more we play Iconary with this AI, the better it will get, but can we really say it’s developing intuition? At this point in the show, I’m wondering what that even means. Anyway, you can play the game yourself, either draw or guess at iconary.allenai.org.

Jessica Chobot: And now we’re heading back to Carnegie Mellon University for that game of Texas hold’em poker. I’m going to put the hype around AI and intuition to the test. Can AI replicate human intuition, or is AI’s decision making all based on numbers? The whole thing starts with Professor Tuomas Sandholm explaining how his Ais are different than any other poker playing AI.

Tuomas Sandholm: These AIs have never listened to any human, or read any poker book, and they’ve never seen a single human play poker ,or a single other AI play poker. So they generate the strategies just from the rules of the game.

Jessica Chobot: So like trial and error for it?

Tuomas Sandholm: No, it’s more sophisticated. It has these algorithms, and this is where we would normally call the AI. The AI is the algorithms that just take as input, the rules of the game. Then, they figure out how to play and then they output a strategy and the strategies. And the strategy is then what plays poker.

Jessica Chobot: Okay.

Tuomas Sandholm: Some people would call the strategy the AI, but that’s kind of a fuzzy line. So that’s why I say, “We write AIs to write AIs to play poker.

Jessica Chobot: In case you didn’t catch that, and I have to admit I didn’t the first time around. Here’s how it works. First Tuomas and his team feed all the rules of poker into an AI, which analyzes them to fully understand the game, not by playing it just by the rules. Then that AI creates another algorithm using game theory to come up with a strategy. That’s the AI I will actually play against. In other words, Tuomas never tells the AI how much to bet or what hands are best. The second AI, the game playing AI, figures that out for itself based on all of the work the first day I did using game theory.

Jessica Chobot: And then when you’re saying game theory, I mean maybe this is silly of me to ask, but like what exactly is game theory?

Tuomas Sandholm: So, this is actually a good question.

Jessica Chobot: Oh okay, good, because I felt really stupid asking it.

Tuomas Sandholm: So, if you think about learning from experience, that is what we in AI called machine learning. This is not machine learning. This is a different subfield of AI. This is computational game theory, and here game theory is all about what’s going to happen in the future. We can reason about the rules of the game, how the players should play into the future.

Jessica Chobot: Sounds kind of like reading between the lines in layman’s terms, that the AI does so well because it’s a bit of a wild card when it comes up against human players that are used to playing in a certain way. I, however am also a bit of a wild card, because I have never played poker or cards honestly except for maybe Go Fish, in my life. So do you mind teaching me real quick and then going up against the AI, and see how I do?

Tuomas Sandholm: Happy to do that?

Jessica Chobot: All right, great, thanks, because I was nervous that you’d say no. All right, well this looks very nice sitting here. We’ve got green felt on the table, got some chips. I have no idea what those mean other than they’re supposed to represent money. I’ve got a deck of cards, and I think we’re ready to go.

Tuomas Sandholm: The goal at the end is to make the five card hand out of the seven cards you have, ie, The two private cards, and the five public cards.

Jessica Chobot: Pulling from the public cards.

Tuomas Sandholm: So we start by one of us being the big blind, and the other one being the small blind.

Jessica Chobot: I got a lot of stuff to smash into my brain right now.

Tuomas Sandholm: Okay. So now we’re going to get two private cards each.

Jessica Chobot: Okay, I’m not going to make you listen to the entire very long poker lesson, but a few key things that I learned. First, poker is really complicated. All the different kinds of hands you can have, and which ones are more powerful or whatever than others.

Tuomas Sandholm: Then better than that is four of a kind, so four of the same number.

Jessica Chobot: Oh my God, this is so hard, okay.

Tuomas Sandholm: And bigger than that is a straight flush, where it’s both a straight and a flush.

Jessica Chobot: Second, you have no idea what the other player has in their hands, so you’re making your decisions based on your gut instinct. And third, how can something so nuanced and subtle be played by an AI? Well, it’s time to find out. I’m sitting down with Tuomas at his computer and on the screen is a pretty typical top down view of a poker table with cards being dealt out.

Tuomas Sandholm: Okay. So now we’re the first to move, and we have a pair of sevens. That’s kind of-

Jessica Chobot: Sevens are always the worst card, I think because it’s so right in the middle.

Tuomas Sandholm: What we really like to do here is what we’d probably like to get the other guy all in right away.

Jessica Chobot: So you’re putting in all your chips?

Tuomas Sandholm: Well, now here’s the thing. If we put in all of our chips, it’s probably gonna fold, unless it’s clearly ahead, so I wouldn’t.

Jessica Chobot: What if you just decide to act crazy? And does it get-

Tuomas Sandholm: Yeah, well the games theory is that even if the opponent plays in a crazy way, we’re still safe, unlike let’s say, machine learning based approaches. So, it’s okay. I mean you can’t really … you can throw it off by crazy play, throw his beliefs off, but you can’t gain any value from that.

Jessica Chobot: So just to be clear, the AI is not reacting to how I play. It isn’t learning about me or even about poker game after game. Instead, it approaches every new game with the same strategy, the strategy it developed using game theory and the rules of poker.

Jessica Chobot: Do 777, seven, seven, seven.

Tuomas Sandholm: You want to do seven, seven, seven?

Jessica Chobot: Yeah, because …

Tuomas Sandholm: Okay, we do that seven, seven, seven. Okay, and we’ll see what’s going to happen. Okay, it folded.

Jessica Chobot: Ha ha ha, see, I knew. I called your bluff. I think we’re actually doing pretty good here, which is nice.

Tuomas Sandholm: Yeah, we are. We are ahead, and that’s not surprising in that there’s a lot of luck in poker. You really need to play thousands of hands before you know who’s better.

Jessica Chobot: I also thought it was interesting that we were trying to trick it by lying to it, but it’s not-

Tuomas Sandholm: It didn’t go for it.

Jessica Chobot: There’s nothing … it doesn’t go for it, and there’s nothing that it’s reading as far as our poker faces or not. It’s just strictly running by the numbers.

Tuomas Sandholm: Yup. Okay. So now, ace, three-

Jessica Chobot: Well, ace is good, but three, yeah.

Tuomas Sandholm: But they’re suited. It’s actually … it’s good.

Jessica Chobot: Okay.

Tuomas Sandholm: You have a decent chance or flush, and if it ends up being in the flush, we’ll have what’s called the nut flush, which is the best flush, because we have the ace.

Jessica Chobot: All right, I’m taking your word for that.

Tuomas Sandholm: Yup, so do you want to do Pot, or do you want to do 260?

Jessica Chobot: Let’s do 260.

Tuomas Sandholm: Oh, this is perfect. We have four cards of diamonds. If we get one more diamond, we’ll have a nut flush.

Jessica Chobot: Do you want to trick it by making it continue to play by going pot?

Tuomas Sandholm: Normally, it would it be three quarters pot here.

Jessica Chobot: Oh, three quarters?

Tuomas Sandholm: Yeah, in this late of betting round. On the other hand, if we have … if we don’t get the diamond, we really have nothing here.

Jessica Chobot: I say we just go for it.

Tuomas Sandholm: Go for bet?

Jessica Chobot: Go for it, yeah.

Tuomas Sandholm: Okay, three quarter pot.

Jessica Chobot: Obviously I’m playing by the seat of my pants here. I barely have any idea of what’s going on, but I have to admit what’s really throwing me is that there is no emotion involved. There is no use in trying to play any psychological games, which maybe would cover up the fact that I’m totally making it all up. Instead of being forced to think and play like the computer, just numbers.

Tuomas Sandholm: Well, it called. So now, we have one more card coming. If you think about it, there are nine more diamonds out there.

Jessica Chobot: I say go for it, just …

Tuomas Sandholm: Bet?

Jessica Chobot: Yeah, let’s bet. Let’s bet.

Tuomas Sandholm: Okay, three quarter pot would it be the standard size.

Jessica Chobot: Okay, come on.

Tuomas Sandholm: All right. Oh, it folded.

Jessica Chobot: It quit?

Tuomas Sandholm: It folded, very good, because we ended up … in essence, you had nothing there.

Jessica Chobot: Okay.

Tuomas Sandholm: But we basically bluffed it off the hand.

Jessica Chobot: But if we continued to play, would it get better at kind of assessing the situation, and learning off of how our habits are?

Tuomas Sandholm: This AI is game theoretic, so it doesn’t need to do that. It’s already optimized its strategy. So if we played a thousand hands, we would be losing like crazy.

Jessica Chobot: Yeah.

Tuomas Sandholm: And it’s not because it’s learning how to play. It’s already playing better than we are. It just got unlucky there.

Jessica Chobot: I just want to point out, because this will probably never happen again, that I played against a super intelligent poker AI and I beat it. Yes, it was just beginner’s luck, but whatever, I still won and I’m claiming the title of AI Poker champion. It’s time to step away from the table and enjoy my winnings. So,, obviously we’re playing poker here, but I’m assuming you didn’t create this AI strictly for poker playing, so what other applications does it have?

Tuomas Sandholm: That’s exactly right. So we have been working on this for about 17 years now, and it is definitely not for poker. Poker is just the main benchmark application. Rather, the technology’s game independent and we’re actually taking it into a variety of real world applications. So in my startup company called Strategic Machine, we’re taking it to various business and entertainment applications. And in my startup, Strategy Robot, we’re taking it to various military and intelligence applications.

Jessica Chobot: And so all of that sounds really interesting and potentially top secret. Anything that you can do a deeper dive on?

Tuomas Sandholm: Yeah. In terms of the high level issue in how optimization and planning happens today, in all of these applications, if at all, sometimes it’s just human gut feel and people make decisions, but if there’s optimization, today it’s assuming a strategy for the adversary. For example, in pricing, it assumes that the other competitors don’t change their pricing, and I get to be the only one who’s optimizing pricing. Of course, in reality, as I changed my prices, my adversaries are going to … competitors are gonna change their pricing, and so on. So, this allows you to think ahead what will be the strategic response of the opponents. So think about negotiation, pricing, various investment banking, and trade execution situations, video games, coming up with good, smart AI opponents for video games, instead of these simple, boring AI opponents that you see in video games today.

Jessica Chobot: You are now speaking my language. So thank you very much for your time today. It was super, super interesting.

Tuomas Sandholm: Thank you for coming.

Jessica Chobot: Yeah, I had a great time. Well that didn’t go nearly as disastrously as I thought. So after playing poker against the AI, and speaking with the experts, can AI replicate human intuition? The answer? Not really. What it’s really doing is using data analysis to make the best choice. But will AI help us make better decisions? Absolutely. Based on facts though, and not gut instinct. But the most important thing to remember from today is this, I won. That’s AI hype versus reality from Dell Technologies. And to watch that game of Texas hold’em yourself. Check out delltechnologies.com/hypeVreality. Next time on the podcast, just how worried should we be that AIs can identify us in a crowd? We’ll find out.