An Academic Study of Online Poker Forums

This morning I found myself trying to read a kind of thumbnail history of poker forums written by Wendeen Eolis for Poker Player Newspaper. I say “trying” because like other pieces by Eolis, her less-than-clear writing style can make reading her a bit of a challenge, I’m afraid. Indeed, the title of this one -- “Online Poker Forums: Winding Through The Maze” -- has kind of a double application, referring both to the complex world of the forums and to the piece itself.

The article is actually the first of two parts, and despite the unclear organization and occasional Faulkernesque unwillingness to end a sentence, the article does manage to remind us that poker forums have been around for a long time -- more than two decades, in fact, with the once-prominent rec.gambling.poker (RGP) site among the pioneers.

I only bring up Eolis’s piece, actually, because it made me think of another article by Killian O’Leary and Conor Carroll published late last year in the Journal of Gambling Studies called “The Online Poker Subculture: Dialogues, Interactions, and Networks.” I had meant to write something about this study some time ago, and so am glad to have an excuse today to share it.

The study does a good job of explaining not just how online poker has evolved into an important “eco-system” over the last 15 years, but also how poker forums have come to affect and shape the functioning of what the authors refer to as the Online Poker Subculture (OPS).

As the authors point out, poker forums constitute one category or “platform” for interaction within the world of online poker, along with news sites (PokerNews, BLUFF, etc.), reporting/tracking sites (PokerTableRatings, etc.), and the online poker sites themselves. The forums are their focus, however, and they end up uncovering some interesting findings as they develop their ideas regarding how people tend to interact within these forums and how those interactions follow certain expectations regarding subcultures, generally speaking.

The methodology employed by the researchers was to follow procedures of “netnography” which if I understand it applies techniques used by anthropologists or ethnographers when analyzing a web-based group or subculture. In other words, they were essentially “lurkers” looking in on the forums of Deuces Cracked, High Stakes Database, and most primarily 2+2 in order to learn more about them.

They share a lot of interesting ideas and ways of describing how, say, a site like 2+2 functions and the influence the forums have over the OPS and even the poker world at large. As a way to make my own post more readable and also avoid going through the entire study point by point, let me just list a few of the findings presented in the article and comment briefly on each.

Collaboration and Competition

It is common to hear poker forums characterized as an antagonistic, combative environments, but what the authors of this study have found is something different, namely, “an ethos of collaboration/co-operation” within the forums that involves “conforming to the norms of OPS etiquette.” In other words, people often genuinely communicate and work together on the forums, as evidenced by the individuals sharing information in order to uncover insider cheating scandals as well as small groups discussing how best to play a particular hand.

That said, there also exists “a competitive hierarchy of status” in the forums. “The more one engages and participates in online forums the higher [one is] elevated within the subculture[’]s hierarchy,” they observe, noting for example how things like join dates and post counts greatly affect one’s influence when it comes to posting. There’s also a pressure to “enact and adhere to the ideals and ethos of the OPS” since “members are and have been in the past ostracized for non-conformity.”

Thus, the forums in particular show how the online poker subculture “distinctively enacts a contradiction, in that within a context of individually driven selfish motives (i.e., everyone playing to win), collaboration and cooperation comes to the fore within the OPS.”

Identity Creation

The authors have much to say about how in the process of participating in the forums, individuals create identities that extend beyond the forums and into the OPS at large, or even beyond. “Online poker forums allow players to develop their own online persona,” explain the authors, “through interaction, participation and engagement with the subculture, thus reaffirming their reputation amongst their poker peers.”

They go on to address how “online poker celebrities” sometimes emerge from the poker forums. In fact, they point out how within the OPS it is often the case that “to become a highly successful online poker player and to receive accreditation, monetary results are not solely sufficient,” but some sort of meaningful, “intense interaction” on the forums is needed as well.

A Game-Changer

The authors also come away from their study concluding that their influence upon the way poker is played -- not just online, but live as well -- “has revolutionized the game.”

They go into some detail explaining in what strikes me as a knowledgeable way how forums have affected strategy, introduced new terminology, and sometimes even the behaviors exhibited in live poker (e.g., “the lack [of] social interaction/dialogue during physical game play”).

Two Plus Two’s “Sacred Status”

Having explored all of these areas, the authors are prepared to refer to 2+2 in particular as enjoying a so-called “sacred status amongst this online poker subculture.”

Such talk reminds me a little of some of the fuss that arose couple of years ago when 2+2 Grand Poobah Mason Malmuth once suggested that “2+2 is where the poker community is.” But truthfully the authors are not suggesting 2+2 is “the” poker community. (Neither was Malmuth, in my opinion.) Rather are they pointing out how the site and its forums possess special, extensive influence on the online poker subculture and its functioning.

I’m reminded here that BLUFF just released its “Power 20” last week and once again neither Malmuth nor any representative of Two Plus Two were listed. (I actually was asked to vote this time, and in fact I did include both Malmuth and Kevmath in the lower half of my 20.) I believe the last time any reference to 2+2 was made on the list was 2009.

Anyhow, if you’re at all curious to read a smart, studied analysis of poker forums, go read O’Leary and Carroll’s “The Online Poker Subculture: Dialogues, Interactions, and Networks.” They absolutely prove that the “OPS” exists, in my opinion, and also do a good job explaining the role forums play within that subculture.

The writing is dense, of course, following as it does the dictates of academic discourse (with lots of citation). But the argument is clear and the style still accessible, I think, particularly to readers of this blog who presumably already have an interest in online poker and the way those of us who play it (or used to play it) tend to interact.

(EDIT [added 2/27/13]: Thanks to @PokerScout1 for pointing out to me over Twitter that O’Leary and Carroll’s introduction actually contains a few glaring mistakes regarding online poker’s historical background, most coming in a single paragraph I have to confess to have only skimmed in my haste to get to the study. Also worth noting -- as @PokerScout1 reminded me -- is the fact that in referring to tracking sites the authors failed to mention Poker Scout [!]. I do think the study is insightful and highlights a need for similar kinds of inquiry, although have to acknowledge that as was the case for me with Eolis’s article, I can see how these errors might prevent some from wanting to delve further into what the authors have to say.)

More on Luck vs. Skill in Poker: Two New Academic Studies Reach Opposite Conclusions

Continuing with this “luck-vs.-skill” debate in poker mentioned yesterday in the context of that Brooklyn federal judge’s ruling that Texas hold’em did not constitute illegal gambling as defined by the IGBA, I’m seeing two different academic studies appearing this month attempting to tackle the same issue.

One study comes from a trio of economists working at the Erasmus School of Economics located at Erasmus University in Rotterdam in the Netherlands. Their paper, “Beyond Chance? The Persistence of Performance in Online Poker,” examines a fairly large database of online poker hands to conclude “that players whose earlier profitability was in the top (bottom) deciles perform better (worse) and are substantially more likely to end up in the top (bottom) performance deciles of the following time period.” Such “persistence of performance” thus leads the researchers to conclude “that skill is an important factor in online poker.”

Meanwhile, a second study by another team of researchers working at the Institute of Psychology and Cognition Research at the University of Bremen in Germany arrives at a different conclusion. The four authors’ study, titled “Is Poker a Game of Skill or Chance? A Quasi-Experimental Study,” is due to appear in the September 2012 issue of the Journal of Gambling Studies, although it was made available online last week. Analyzing a much more modest sample size, the Bremen folks determined that “card distribution was the decisive factor for successful poker playing” rather than players’ skill levels, and thus their “findings indicate that poker should be regarded as a game of chance.”

The study by the Dutch group is a 32-page first draft that has been submitted to a journal for possible publication, and is available online for anyone to read at the Social Science Research Network site (click here or click the study’s title above). Meanwhile, the Germans’ full article can only be purchased unless you happen to have an academic affiliation and your school or university subscribes to the journal. Luckily for me I do have such an affiliation and my university does subscribe, so I’ve had a chance to download and read the 16-page study.

I’m neither an economist nor a psychologist, and so have to admit some of the methodology employed in both studies is a bit foreign to me. Still, I was able to follow both arguments reasonably well, and so thought I’d try to provide summaries of each here.

“Beyond Chance? The Persistence of Performance in Online Poker”

The Dutch economists begin by providing some context for their study, including explaining how both the popularity of online poker (and live poker) worldwide and ongoing battles over its legality inspired the study. They show a keen awareness of other research, too, including what behavioral research has found regarding decision-making in poker.

Thus do they begin with an understanding that players bring varying degrees of skill to poker, but also that players behave in ways that affect their performance, too (e.g., overestimating expectations, letting results affect decision-making, etc.). I’m glossing over specifics here, but mean to point out that I’m convinced the researchers appreciate the complexity of poker and how the skill component of the game involves a lot more than understanding probabilities, but also being able to manage one’s emotions and discipline oneself to make sound decisions consistently.

They go on to allude to other luck-vs.-skill studies of poker, including the one co-authored by Steven “Freakonomics” Levitt last year, positioning theirs as being methodologically similar to empirical studies like the one Levitt helped perform.

The researchers then describe how they analyzed a database of 76.7 million hands of real-money online NLHE played over the course of one year (from October 2009 to September 2010). Hands came from three different stakes -- 25NL, 200NL, and 1,000NL -- and involved over half a million different players. One player alone played 760,000 hands (!), while more than half of all players played less than 100 hands. The average number of hands each player played was around 1,000 for the middle and high stakes games and just over 400 for 25NL.

Of interest to many is the finding that 32% of all players in their sample “achieved a positive overall result after the deduction of rake,” a figure much higher than what I would’ve guessed. It should be noted the researchers ended up controlling for rake -- i.e., did not account for it when measuring performance -- since they’re aware of things like rakeback and bonuses and other factors that make it unclear just how much rake ultimately affects players’ profits.

Digging deeper, the researchers split the 12-month period into two 6-month periods, then looked at the first 6-month period, filtering out players who played less than 1,000 hands (the average ended up being about 5,800 hands per player). They worked out the BB/100 for each of those 32,716 players, then looked at those same players’ performances during the second 6-month period.

“In a nutshell,” they explain, “the results indicate that there is substantial and significant persistence in performance: deciles of players that performed relatively well in the first period on average continued to do so in the second period.”

The number-crunching continues from there, including performing similar analyses while breaking down the 12-months into four 3-month periods, as well as doing other work to try to account for varying sample sizes (i.e., to lessen the impact on the findings of those players playing fewer hands). The second half of the study then performs additional “regression analyses” that ultimately reinforce the initial findings that there does exist “persistence in performance” in poker, although to be honest this is where I have to admit my own limitations with comprehension.

The conclusion goes on to talk about brick-and-mortar games, tournament poker, and other variants not covered by the study. Suffice it to say, the study is quite thorough and self-aware, and its conclusion that “the results provide strong evidence against the hypothesis that poker is a game of pure chance” is quite persuasive.

“Is Poker a Game of Skill or Chance? A Quasi-Experimental Study”

I was very curious about this study by four German psychologists given its conclusion that luck rather than skill predominated in poker -- the first academic study I’ve heard of, actually, that does not come down on the side of skill. By the way, a “quasi-experimental study” refers to an empirical study in which the researcher gets to control the conditions. For example, rather than look at hands dealt by others (or by an online card room), the researchers can deal the cards and even minimize the randomness of how they are distributed.

Like the Dutch crowd, the authors of this study also begin with an overview of poker’s popularity, the abundance of online rooms (594 as of 2011, they say), and various legal battles in which the “luck-vs.-skill” issue is often central. They then look at previous studies which they characterize as “heterogenous and fragmentary” given the disparate types of methodologies employed. “Most research findings have suggested that skill is a meaningful element,” they admit, although qualify this statement by suggesting previous research has been “based on a relatively superficial examination.”

Moving on, the group next explains their methodology. They recruited 300 poker players via online advertisements, emailing them questionnaires about their poker skill and behavior, then paying their travel expenses to come participate. Those players were then given a second 27-item questionnaire designed to determine whether to classify each as an “average” or “expert” player.

The questions asked about their frequency of play, their self-perceptions about their own success and skill, and even what you might call their mental toughness (“How often do you mentally deal with poker, even if you do not play?”). A complicated method of grading the questionnaries was established, and ultimately the 300 players were divided evenly into two groups of 150 “average” players and 150 “expert” ones.

The 300 were then sat around 50 six-handed tables, half of which played no-limit hold’em while the other half played fixed-limit HE. The blinds were €0.03/€0.06 for both games, with each player given €10 with which to play. The LHE players got all €10 to start, while the NLHE ones got €5 with a chance to “rebuy” for €5 more.

“To provide an overall incentive to participate in the game and to ensure that the players’ game behavior would reflect actual play to the greatest extent,” there were prizes given to players who did well, although to be honest I’m not really following how those prizes were distributed. The study says “five special prizes of up to EUR 500 were offered to the best players at all of the tables,” whatever that means.

The games were played on computers (with players isolated from one another), following a “duplicate poker” format with all hands -- i.e., all the hole cards, flops, turns, and rivers -- having been predetermined. Players didn’t know this (i.e., they were led to believe the deal was random), nor did they know the game would be stopped after playing just 60 hands.

The deal was manipulated in a way that is also a bit ambiguous to follow, with players being dealt the “winning hand” and “losing hand” a predetermined number of times. I’ll boil it down to all players intentionally being dealt “good cards” and “bad cards” in a way the researchers believed to be evenly distributed, with the idea being to compare how well the “experts” did versus the “average players” in both cases (i.e., when running good and when running bad).

The results indicated what the researchers determine to be statistically insignificant differences between the performance of the “experts” and the “average players” when it came to playing good cards (or being dealt “winning hands”). That is, the “experts” did a little better, it sounds like, but not enough to matter. However, when dealt bad cards (“losing hands”), the “experts... outperformed average players... considerably.”

Again, as with the Dutch economists’ study, there are tables and breakdowns of the results, most of which extend beyond my understanding. But I think I’m following the overall argument, namely, that since “the only significant difference in cash balance outcome between experts and average players was under bad card conditions,” the researchers are concluding “there was no evidence for one group’s overall superiority in this context.”

“Therefore,” they go on to say, “it can be concluded that chance clearly dominates skill; thus, poker should be classified as gambling.”

To their credit, the researchers do admit to various limitations in their methodology, including a recognition that their subjects were self-selected and that their questionnaires had not been adequately tested for reliability. They also acknowledge they’ve gone with a super-small sample size in terms of number hands, although weirdly respond to that objection by speculating that “it could be assumed that with longer play periods, the difference in players’ level of skill would decrease,” a statement that seems to betray an obvious lack of understanding of the game they are studying.

There seem to be some huge problems with the Germans’ methodology that ultimately undermine the study in a significant way -- from the highly-suspect method of classifying of players as “expert” or “average” to the awkward dealing of “winning” and “losing” hands to the teeny-tiny sample size. More than enough to make their conclusion seem prematurely drawn, I’d say.

I remember reading an article in the Journal of Gambling Studies a couple of years ago, one by Kyle Siler called “Social and Psychological Challenges of Poker,” which seemed much more grounded and persuasive (discussed here). But that’s not the case with this one.

No, I’m still looking for a more convincing academic study proving once and for all that luck prevails in poker. After all, there seems to be a “persistence of performance” when it comes to these studies, with most ultimately having shown that skill is significant.

(Groovy “ambigram” graphic up top by Michael Irving.)