Texas Hold-Em; An Evolutionary Approach: Part 3
By Richard Ten Dyke

The story continues as we build a program that will use Darwinian techniques to learn to play Texas Hold-Em. We previously discussed using an object-oriented language to create a limited version of the game. In spite of the limitations, we learned a great deal by playing.

In the limited version of the game you play against two opponents. Neither of them uses a betting strategy—we haven’t created one yet. Instead, they stay in the game until it is over. However, you have the opportunity to bet or fold. Playing automatically, you would have a one-in-three chance of winning each hand, but you can improve your own odds by deciding when to bet or fold. With skill at assessing a hand, you can improve your odds of winning to about 50 percent, or so has been my experience. Playing in the simulated poker game is like a tennis player practicing strokes by hitting balls against a brick wall.

Let’s look at some examples:

Suppose your hole cards are two aces. This might seem to be a lock on winning, and it will be a winner in many situations. But it would not be a winner against a straight, a flush, or three-of-a-kind held by an opponent. Therefore, we learn that
assessing the opponent’s possibilities becomes a critically important aspect of the game. So we must study the cards on the table.

Suppose with your pair of aces you see a 7, 9 and 10 on the table. The chances that your opponent has a straight are greatly increased from if there were, say, a 2, 6, and 10. However, suppose that the 2, 6 and 10 are of the same suit. Now there is a chance that your opponent has a flush— and so on. This thinking assumes that your opponent is going to stay in the game. One betting strategy might be to bet a huge amount and hope that your opponents decide to fold. But we haven’t gotten to the betting strategy part of the game quite yet.

As the hand progresses, first with three, then four, and then five cards on the table, assessing the opponents possibilities against your own hand is very important. If there are four cards to a straight or flush, for example, the possibility that an opponent can fill them out is very likely. Similarly, if you are holding a pair of Jacks, and an Ace and Queen are showing on the table, you could easily lose to a pair of aces or a pair of Queens. If you hold those same Jacks, and a pair of threes are showing, you can easily lose to three of a kind.

What is a player to do?


I have found that it is a good idea to fold a bad hand early. The first betting round takes place before the flop, when you are looking at only your two hole cards. If you have a four of spades and a nine of diamonds: Fold! Suppose you have a pair of threes: Fold! Suppose you have an Ace and a Queen: Stay! It’s the middle hands, a Queen—Five, for example—that present the more difficult choices. You might pair up the Queen, but you would still lose to a pair of Kings. But if you have a Queen—Five suited, meaning of the same suit—you could walk into a flush. Or, you have a Queen and a Jack, suited. You now have the chance for a good pair, a flush, or a straight.

The odds will change as you see the flop, the turn and the river cards laid out on the table. Perhaps you might play the Queen-Five-suited for the flop, and hold only if there is improvement.

This sounds complicated, and it is. It makes the game interesting. A good player who has played thousands of hands knows instinctively what to do. The beginning player has little chance against him. But remember, our goal is not to learn to play the game ourselves. Instead we want to teach a computer how to do it. Our problem is how to write a program that can capture years of experience in playing the game.

That leads to the problem that I call characterization. The human will learn from three things from experience: (1) that what shows on the table is important, (2) that certain card combinations are more likely than others to portend success or defeat, and (3) how to categorize those combinations to be able to react when seeing similar although not identical card combinations in the future. But the concept of “similarity” is a difficult one for the computer to grasp. It is necessary to do this because it is not practical to pre-plan for all possible combinations. Grasping the concepts of similarity and analogy are distinctly human capabilities. For the computer to evolve these capabilities over time would require the computer to be able to evolve something that approaches complexity of the human brain which is not likely to happen in this experiment.

At a minimum, it appears necessary for a human to tell the computer what categories to use. I experimented with this idea when playing the game. The computer would identify various card combinations showing on the table, and provide a message or warning about what was showing. The computer would warn about the possibility of a straight or a flush. It would also warn that there were cards on the table of a higher rank than the highest card in my hand. I found I could play the game somewhat successfully based on these warnings alone, without having to look at the actual cards that were showing.

But this is not enough. It is further necessary to place a value on those warnings before the computer. Warning about a three card straight showing shou
ld also indicate whether it is an inside or outside straight. Sometimes more than one possibility has to be considered. One approach, both considered and discarded, was to run a series of simulations with various combinations of hole cards and warnings in order to calculate probabilities of success. Then we would create a table of probabilities to use when playing the game. This looked to be a daunting task. It was clearly time to ask the question: is there a better way?
Yes. There is a better way, and it is another reason why I will not be playing Texas Hold-em for real money on the internet any time soon.

Your opponent in the game has exactly two hole cards, just like you. You ask yourself the question “How many possible card combinations are there that he can hold, and what percentage of them will give him a hand that is better than mine? Starting with the first question, and assuming at this time that all five cards on the table are showing, there are 45 cards left in the deck. All possible two-card combinations is 45 times 23 or 1035, which is not a large number. With my computer at my side, I can feed in the 7 cards that I know about and calculate which of the 1035 card combinations that my opponent may have will give him a better hand. In short, the computer can quickly create a probability of my winning against a single opponent. With a little math, the computer can calculate the probability of winning against several opponents. This is a raw but very useful number. Raw, because it does not take into account that fact that the opponent will make decisions whether to stay or fold so those who are in the game at the last moment will have better than average hands. But it is a useful statistic because it provides a basis on which to make decisions.

We now can discard all the work and anguish of creating scenarios and concentrate on a single number, a “raw” probability of winning against some opponent. What we are doing is using the computer to do what it does best—evaluate a big number of possibilities.

If I were playing Texas Hold-em on the internet, I would have to assume that those whom I might be playing against have this capability, and I would also guess that anyone who plays without it is an idiot. In the next installment, we start to use this insight to develop a computerized betting strategy program.

 


 
 
© Danbury Area Computer Society, Inc. All Rights Reserved.
Web Site Terms & Conditions of Use