Texas Hold-Em, an Evolutionary Approach

By Richard Ten Dyke

This is a story—a real-life story—about three things: the game called “Texas Hold-Em”, object-oriented programming, and evolution. Yes, really.

In the game of Poker, with its many variations, only one thing remains constant: four aces beats a full house. The game has morphed from a legend of the old west to an inter-net and TV phenomenon. From the simple Five Card Stud to Texas Hold-em, the game has acquired a colorful variety with wild cards and cute names. Only the rank of hands remains—sort of. There are some poker games where the “worst hand” wins, and the definition of the worst hand varies from time to time and from place to place.

Texas Hold-Em is but distantly related to the original concept of poker. In this game, a player holds only two cards that one can call one’s own. Five cards additional are shown face up, on the table, for all players to share. A player makes his best hand out of the combination of these seven cards. With two “hole” cards, a player can only have one of 1,326 hands: the number of possible two-card combinations from a 52 card deck. The number of unique hands is even less when you consider that it doesn’t matter if you are holding an Ace-Queen of Spades or an Ace-Queen of Hearts. All that matters in the game is that the two cards are an Ace-Queen and that they are of the same suit. From a practical point of view, there are only 169 unique two-card hands. The value of a hand depends upon how it works with the shared cards on the table.

Object-oriented programming (OOP) is a technique for creating complex and interactive computer programs. The folks at Xerox PARC (Palo Alto Research Center) came up with OOP while developing the Graphic User Interface and the mouse. Object-oriented programming first emerged as a language called SmallTalk, which, along with LISP, was a tool used in Artificial Intelligence. I saw it first in 1983 at an AI conference at the University of Pittsburgh. SmallTalk and LISP were loved by student developers because it provided a means of creating programs with very complex logical structures. And both suffered from the same problem. They were “interpreted” languages rather than “compiled.” Because of this, every user of a program also had the power to change the program itself. Programs written in these interpreted languages could not be widely distributed and at the same time assure that the program would always be running as originally intended. However, one of the best features of object- oriented programming was the ease of re-using code. Functions which were originally designed as sub-routines could be reconstituted as “objects” and more easily reused by other programs. This benefit of SmallTalk was highly valued, and several compiled object-oriented languages including C++ and Cocoa have been created using SmallTalk concepts.

Darwinian evolution will play a role in this story by allowing a computer program to learn to play the game. We will create a poker tournament in which computerized players will play against each other. The players with the better strategies will beget new little poker players to carry on in the family tradition, and the not-so- good ones will die off. What we hope to learn from this is whether the poker strategies which evolve improve over time and also whether they diverge into separate playing styles in the way that a single species can bifurcate into two distinct species over time. Studying how evolution works is the main purpose of the exercise,\; but to be honest, I haven’t figured out how to do it yet, although I have some ideas. Together, we will see if it works. In short, this will be a mystery story, and you and I will get to the solution about the same time.

The Game:
In Texas Hold-Em, each player is dealt two hole cards, face down. A round of betting takes place...Then three new cards are placed face up on the table. This is called “The Flop.”...Another round of betting...A fourth card is placed face up on the table, called “The Turn.” More betting...A fifth card on the table, face up, called the “The River.” A final round of betting...The winner is based on the generally accepted rank of hands: High Card, Pair, Two Pair, Three of a Kind, Straight, Flush, Full House, Four of a Kind, Straight Flush. At each round of betting, players can fold, bet or raise. With so many table cards shared, it is easily possible for two or more hands to be tied as winners. In those instances, the pot is usually divided.
The betting can differ from a normal poker game. Some games are limit games and some are played with pot-limit betting, leading to the expression “All In” when a player wants to bet everything he has on the table. If that player wins, his winnings are limited to the amount he would have won if the betting had stopped at that level. If he loses, he is gone, out of the game. Betting holds the key to playing the game successfully. With numerous opportunities to bluff, and with large stakes, the psychology of the betting game can overshadow the value of the hands. Of course, a computer does not have emotions, like fear or hope. By necessity the computer will play a very mechanical although not a totally predictable game. This may be as easily good or bad. We will have to see.

The Plan
Our first step will be to create a computer program to play a simplified game and play it ourselves for training. This will help when we get to the point where we are trying to develop a computer generated strategy.
After reviewing a few object-oriented programming languages, the one we we will use is called “Real Basic” which is related to the original BASIC programming language in the same sense that the Queen Mary is related to the Staten Island Ferry. Many of the old BASIC data structures and statements are still there, but a lot has been added, making it quite a different language. It employs the basic (small b) concepts of variables, statements and functions, but with the concepts of object-oriented programming added. Real Basic can be compiled on a Mac or a PC, and programs can be created to run on either platform. Another possible choice of language was Cocoa, which is a Mac-only language based on Objective-C which itself was based on SmallTalk. However, compared to Real Basic, Cocoa is more C like, more complex, and requires more attention to be paid to memory management. Another possible choice, C++, is not actually an object-oriented language, but allows the creation of object-oriented data structures called “classes.” Of the three, Real Basic is a more understandable language, and I chose it for that reason.

In the next installment, we will first discuss using Real Basic to create the the first program. Following installments will record our success—or lack of—in getting the computer to play the game successfully.