Playing Cepheus

A short follow-up regarding Cepheus, the heads-up fixed-limit hold’em playing program developed by the Computer Poker Research Group at the University of Alberta I was writing about yesterday -- i.e., the program that is said to have essentially “solved” heads-up LHE insofar as (the researchers claim) “a human lifetime of play is not sufficient to establish with statistical significance that the strategy [employed by Cepheus] is not an exact solution.”

I mentioned how they’ve put Cepheus online for the curious to play against. Today after queueing up for a long time I managed to get a game against the program. We played 100 hands of 10/20 LHE, after which I managed to finish up 105 units -- just about five big bets.

I ran hot early on, so hot it almost seemed like things were rigged in my favor as I built up a lead of over 250 through the first 30 hands. Then things evened out between us and after 49 hands we were dead even, and for a hand or two after that I was down briefly. But I won three big pots in a row to zoom back up over 200, and ultimately never lost the lead again.

I played tight-aggressive throughout, becoming a little more conservative during the last dozen hands or so as I wanted to preserve my lead. Both Cepheus and I were mindful of position, with Cepheus raising almost every single button and folding otherwise (i.e., never limping). Meanwhile I also mostly raised or folded my buttons (folding more than Cepheus did), though I limped occasionally, too.

Cepheus would three-bet me fairly often before the flop when I did raise, and probably bet when checked to around 80-90% of the time (I don’t have a log of the hand histories, so can’t say for sure). After about 75 hands I had just begun to become aware of the fact that Cepheus hadn’t seemed to have check-raised me on either the turn or river, then the program did it twice within just a few hands, both times successfully earning extra bets as a result.

In the first case I was playing from the button with K-7-offsuit with the king of clubs, and the flop had come all clubs with the ace to give me a nut flush draw. Cepheus check-called me there, then a king fell on the turn and that’s where Cepheus check-raised me. The river was a blank, and Cepheus won the hand with K-Q.

The second instance also involved Cepheus connecting on the turn -- that’s a screenshot of that hand above (click to enlarge). I also managed to check-raise a couple of turns after making hands to get extra value.

Obviously the tiny sample established practically nothing regarding either Cepheus or myself. I will admit that toward the latter part of the session I felt my attention flag just a touch, enough to remind me of the difference between myself and my non-human competitor. If we’d gone on, say, to play 1,000 hands or more I imagine it would have been very difficult for me to continue to make correct decisions (not that all of the ones I made were correct).

If you happen to play Cepheus, let me know how it goes and what impressions you get from the program.

Cepheus and the “Solving” of Heads-Up LHE

Yesterday I was writing about the “New York Times 4th Down Bot” and related efforts by those studying stats to present math-based models for dictating how best to coach a football game. There was a related story last week regarding the team of researchers at the University of Alberta who have been working for several years on “solving” heads-up fixed-limit hold’em, with their new study published in the January 2015 issue of the journal Science “announc[ing] that heads-up limit Texas hold’em is now essentially weakly solved.”

I remember many years ago -- way back in the summer of 2007 -- speaking with Jonathan Schaeffer of the University of Alberta about Polaris, the computer program developed by his team of researchers that comprised the university’s Computer Poker Research Group (CPRG) in the dept. of Computing Science which he chaired. An LHE match between Polaris and two pros, Phil Laak and Ali Eslami, took place that summer in which the humans won, and for PokerNews I spoke to Schaeffer about Polaris and the CPRG’s goals.

“One of these days -- within 5 to 10 years -- two-person, limit hold’em will be solved,” he said to me. Here we are about seven-and-a-half years later, and it sounds like those continuing the work there at Alberta have fulfilled Schaeffer’s prediction.

The new heads-up LHE-playing program is called Cepheus and is in fact available to play against online, although when I went to the site there were “too many in queue” and I was invited to come back later to try.

With my academic affiliation I have been able to get a copy of the Science study and have read through it. Simply titled “Heads-up limit hold'em poker is solved,” it begins by pointing out that while certain “perfect-information” games like Connect Four and checkers have been solved, others like chess have not even if much-celebrated events like Deep Blue’s victory over Garry Kasparov has led some to suggest it has. “Defeating top human players is not the same as ‘solving’ a game,” the study’s four authors point out.

The study then notes how only “perfect-information” games have been solved thus far, making their claim regarding the “imperfect-information” game of heads-up limit hold’em groundbreaking. Reference is made to the work of game theory pioneer John von Neumann and the element of bluffing that distinguishes poker, to Michael Craig’s The Professor, the Banker, and the Suicide King (which details the high-stakes LHE games between by Andy Beal and “the Corporation”), as well as to the previous work of Schaeffer and his team at Alberta.

An explanation of what it means to “solve” a game follows, with the distinction “weakly solved” referring to a game in which “for the initial position(s), a strategy has been determined to obtain at least the game-theoretic value, for both players, under reasonable resources.” From that definition, the researches extrapolate that as far as heads-up LHE goes, it is safe to say that the game is “essentially weakly solved... if a human lifetime of play is not sufficient to establish with statistical significance that the strategy is not an exact solution.”

From there comes further refinement of what is meant by an “imperfect-information game” then an explanation of the programming of Cepheus and the “solution” ultimately found. This admittedly is where your humble scribbler feels especially humbled, not being versed in the various fields of the researchers.

Back in 2007 there was already lots of talk about “poker bots” and online poker, so I had to ask Schaeffer about Polaris and how it might relate to the online game. “I want to be clear,” he told me. “We do not play online poker. None of our software is enabled to play online poker on any of the sites.”

Nor has being able to “solve” heads-up limit hold’em ever been the endgame for the researchers at Alberta. They conclude their study by noting how “the breakthroughs behind our result are general algorithmic advances that make game-theoretic reasoning in large-scale models of any sort more tractable.” In other words, as often gets pointed out by those who study game theory as it applies to recreational games or sports, the findings there have value in other realms involving human decision-making.

As a longtime fan of LHE, I’m curious to learn more about Cepheus, including how it was created and what it can do. If you are also curious, here’s a good piece on the FiveThirtyEight site summarizing the team’s work and placing it in a broader context by Oliver Roeder called “Computers Are Learning How to Treat Illnesses By Playing Poker and Atari.” And I’m about to go listen to the latest episode of the Thinking Poker podcast on which Andrew Brokos had on as guests two of the study’s authors, Michael Bowling and Michael Johanson of the CPRG at Alberta, to talk about Cepheus.

In the meantime I’m going to keep queueing up to try to play Cepheus. Last night Vanessa Selbst was on PokerStars playing at some some 8-game play money tables, and I actually sat at one for a few orbits. Feeling equally intimidated to sit across from Cepheus, I think.