SpinGPT

Training a Poker LLM

Narada Maugin

LAMSADE, Paris Dauphine University - PSL, CNRS, Paris, France

Tristan Cazenave

LAMSADE, Paris Dauphine University - PSL, CNRS, Paris, France

What is poker?

• A family of card games
  • Most widespread variant: No-Limit Texas Hold’em
  • 2 to 10 players
  • Dealing and betting rounds (preflop, flop, turn, river)
• Originated in the United States in the 1820s
• ~ 100 million players
• Imperfect-information game
• Enormous game tree

Hello, I’m Narada Maugin, glad to be there to present a joint work on poker and LLM with Tristan Cazenave. We’ve developed SpinGPT, a large language model that plays poker.

Poker is not a game, but a family of card games. The most common variant is No-Limit Texas Hold’em, played from 2 to 10 players, with a part of dealing cards and then after there is 4 betting rounds (preflop, flop, turn and river).

It originated in the United States in the 1820s. Today, it has attract about one hundred million players worldwide. It is an imperfect information game because players don’t know each other’s cards, and the game tree is enormous, making it hard to solve.

Timeline

---
config:
  theme: dark
---
timeline
    title The rise of poker AIs
    2006 : First Computer Poker Competition (ACPC).
    2007 : Counterfactual Regret Minimization (CFR) introduced (Zinkevich et al.)
    2014 : Cepheus solves Heads-Up Limit (Bowling et al.). : Game size 10^13.
    2017 : Libratus solves Heads-Up No-Limit (Brown & Sandholm). : Game size 10^160.
    2019 : Pluribus beats pros in six-player No-Limit Hold’em (Brown & Sandholm). : Game size <br> > 10^220.

To set the context, I’ll briefly review a few milestones in poker AI. Not exhaustive. In 2006, the first Computer Poker Competition (ACPC) launched. It gave researchers a shared benchmark and a place to train agents, compare results, and exchange ideas over many years. In 2007, Counterfactual Regret Minimization (CFR) was introduced (Zinkevich et al.). CFR became the core of many later agents.

In 2014, Cepheus essentially solved Heads-Up Limit (Bowling et al.). Heads-up means 1v1 and limit means that the amount of chips one can bet is limited, which constraint the game size to ten to the thirteenth states.

3 years later, Libratus extanded the way by solving Heads-Up No-Limit (Brown & Sandholm). So this time, the amount of chips one can bet is unlimited, so the game size became enormous, around ten to the power of one hundred sixty.

In 2019, Pluribus beat professionals poker players in six-player No-Limit (Brown & Sandholm), extending strong play to multiplayer settings, with game size exceeding ten to the power of two hundred twenty. Overall, we progressed from solving HU-Limit to beating pros in six-max No-Limit.

What challenges remain for poker AIs?

• Tournaments are the main challenge
  • Varying player counts and stack sizes –> non-i.i.d. dynamics
  • The chipEV objective needs to be reconsidered
• Spin & Go
  • 3-player mini-tournament –> easier to tackle
  • The most played online format

But challenges remain. The main one is tournaments. Tournaments are dynamic: early on there are n players who are divided among tables of 6–9 seats, then step-by-step players bust, so table size and stack distributions keep changing. This breaks stationarity that we had with the previous AIs and makes standard equilibrium methods like CFR harder to apply. Also, maximizing chipEV is not the right objective in this format.

To make progress, we focus on a simpler problem : Spin & Go. It is a three-player mini-tournament where the winner take-all. It’s simpler than tournaments where there can be thousands of players. But still captures two main tournament dynamics : evolution of players, and evolution of stacks. The most played online format by far.

LLMs for poker

• Limitations:
  • LLMs perform poorly at poker with zero-shot or few-shot prompting.
  • Few open, high-quality training resources online.
• Advantages:
  • Prior knowledge (rules, basic strategies, probabilities,…)
  • Strong ability to learn and produce structured text representations
  • –> Fine-tuning is effective (Huang 2024, Zhuang 2025)

Using LLMs for poker can seem odd. Off-the-shelf models play badly, even with few-shot prompts. It’s because there is no large datasets of poker hands available online. So during their pre-training, they don’t see many poker hands, unlike chess for instance, where there is plenty of online games.

Why use an LLM anyway? It’s steal has its advantages. First, it already knows rules and basic strategies (e.g., a pair of aces is the best hand). Crucialy, it can learn hand histories from structured text representations and output actions after training.

Previous studies has shown that fine-tuning is very usefull for a model to play poker.

Data collection

But to fine-tune, we need data. As we said, there is no large, open dataset containing poker hand histories online. So how did we proceed ?

• Hands from my own professional play
  • Dates: 2018-2020
  • Format: Spin & Go (2 or 3 players)
  • Buy-ins: €50, €100, €250
  • Sample size: n = 320,000

We use the fact that I was a former profesionnal poker player to built our own dataset from Spin&Go hands that I played between 2018-2020, on high stakes. We had a total of three hundred and twenty thousand hands that we used.

Data processing

• Original hand:
  Game #20396217527 starts.
  Game #<do not remove this line!> starts.
  ***** Hand History for Game 20396217527 *****
  NL Texas Hold’em €250 EUR Buy-in - Friday, July 10, 18:41:29 CEST 2020
  Table 250€ SIT’N GO JAQKPOT (276572612) Table #1 (Real Money)
  Seat 2 is the button
  Total number of players : 2/3
  Seat 1: Dimitrov98 ( 605 )
  Seat 2: FrenchBaguette ( 895 )
  Trny: 276572612 Level: 3
  Blinds(20/40)
  FrenchBaguette posts small blind [20].
  Dimitrov98 posts big blind [40].
  ** Dealing down cards **
  Dealt to FrenchBaguette [ Kd 7s ]
  FrenchBaguette raises [60]
  Dimitrov98 calls [40]
  ** Dealing Flop ** [ 5h, 8d, 4s ]
  Dimitrov98 checks
  FrenchBaguette checks
  ** Dealing Turn ** [ 4d ]
  Dimitrov98 checks
  FrenchBaguette checks
  ** Dealing River ** [ Td ]
  Dimitrov98 checks
  FrenchBaguette checks
  Dimitrov98 shows [ 9d, Qh ]a pair of Fours.
  FrenchBaguette shows [ Kd, 7s ]a pair of Fours with King kicker.
  FrenchBaguette wins 160 chips from the main pot with a pair of Fours with King kicker.

A poker hand, retrieved from a poker tracker, looks like this. It is very long and contains a lot of unnecessary information (like the date), so we have reduced it to retain only the information relevant to decision-making.

And this is the result. There is the player’s position, their chip count, the opponent’s chip count, our hand, and all the preflop, flop, turn and river actions.

And H colon means that it is the model’s turn to play and here, the expected answer is x, meaning check.

• Processed hand :
  {“instruction”:“pos:H=SB stacks:H=15.1,BB=22.4 hand:Kd7s | pre: H r2,BB c | flop:5h8d4s BB x,H x | turn:4d BB x,H x | river:Td SB x, H:”,“output”:“x”,“input”:““}

A poker hand, retrieved from a poker tracker, looks like this. It is very long and contains a lot of unnecessary information (like the date), so we have reduced it to retain only the information relevant to decision-making.

And this is the result. There is the player’s position, their chip count, the opponent’s chip count, our hand, and all the preflop, flop, turn and river actions. And H colon means that it is the model’s turn to play and here, the expected answer is x, meaning check.

Supervised fine-tuning (SFT)

• Model: Llama 3.1-8B-Instruct

  

• Advantages:

  • Open weights
  • Small enough for single-GPU

• Limitations:

  • Beginner level at poker
  • Lower capacity than larger models

• Training time: 10 hours on one GPU

So with all this three hundred and twenty thousand hands, we were ready for the fine-tuning. We used Llama 3.1 with 8 billions of paramaters. One plus is that the model is open WAIT, so we can use it freely (almost). And it’s small enough to be trained on a single-GPU.

One main limitation is that it is not very good at poker, and it has lower capacity than larger LLMs with their hundreds of billions paramaters. And in this area, quantity matters. We trained it for 10 hours and we called the resulting model SpinGPT-SFT.

SpinGPT-SFT: imitation results

• Test set (n = 32,000)

• Accuracy (exact-match):

  • 80% (exact)
  • 84% (with tolerance)

• Illegal actions: 1.4%

We evaluate SpinGPT-SFT on a 32k test set. Exact accuracy is 80% (and 84% with tolerance). The illegal-action rate is 1.4%.

Exact Accuracy: the model’s action token exactly equals the action played when I was professional. Accuracy with tolerance means taht for bet/raise, we accept a small deviation around the target size. For example, if the target size is 2.6, a prediction of 2.5 counts as correct.

How to read the confusion matrix: rows = true labels, columns = predicted labels. The diagonal gives exact matches.

For instance, if we consider the row check there, …

Before fine-tuning, it was almost a 0% accuracy, and now it’s way better. But at what point ?

Benchmark : Slumbot

• Slumbot
  • ACPC 2018 champion poker AI
  • Common benchmark (easy API evaluation)

We use Slumbot (2018 AI world champion) as a convenient heads-up baseline. It is a common benchmark to play against and to evaluate our AI.

Warning

Depth mismatch: SpinGPT 0–35 BB vs Slumbot 200 BB.
Applying a short-stack policy at deep stacks → over-shoving → losses.

• Patch: replace all-ins with 2/3-pot when facing no bet; otherwise 3x raise

But, there is a depth mismatch: SpinGPT was trained on short stacks (0–35 BB), while Slumbot plays deep stacks (200 BB). A short-stack policy tends to make mistakes at deep stacks, especially tends to all-in way too often, which produces massive losses.

To control this, we had to patch our model when it goes all-in by doing this modifications.

Results vs Slumbot (1)

Agent	Win rate (BB/100)	95% CI (BB/100)	Hands played
SpinGPT‑SFT	13.4	± 12.9	30,000

Interpreting win rate (BB/100)

• < 0: loses
• ≈ 0: break-even
• 0-6: wins
• > 6: wins by a wide margin

This table reports SpinGPT-SFT’s win rate vs Slumbot in BB hundred, confidence interval and the number of hands.

Definition: BB/100 = expected big blinds won per 100 hands. négative: loosing around 0 : equal / break-ven positive : winning if it’s more than 6 : winning large

The winrate indicates a wide-margin, but the confidence interval is large… SpinGPT-SFT is probably better than Slumbot, but we don’t know at what point.

Results vs Slumbot (2)

Agent	Win rate (BB/100)	95% CI (BB/100)	Hands played	Year
BabyTartanian8	3.6	± 1.2	N/A	2016
ReBeL	4.5	± 1.0	N/A	2020
AlphaHoldem	11.2	± 1.6	100,000	2022
PokerGPT	15.8	± 4.9	10,000	2024
SpinGPT-SFT	13.4	± 12.9	30,000	2025

To compare, I reported winrate that others models had against Slumbot. In particulier, we can see the progress made over the years.

Reinforcement Learning (RL)

Objective: retrain SpinGPT-SFT to approach Game Theory Optimal (GTO).

Ok, so SpinGPT-SFT beats Slumbot, but can we do even better? So far, SpinGPT-SFT has been trained on hands that I played. I was a professional, so I played pretty well, but I made mistakes. So SpinGPT-SFT is trying to imitate a strategy that includes mistakes: we can do better than that.

• InstaGTO data
• ORPO (best-vs-worst action per state)
• 320,000 training hands:
  • 270,000 InstaGTO (synthetic)
  • 50,000 pro (to prevent catastrophic forgetting)
• Training time: 10 hours on a single GPU
• Final model: SpinGPT

The idea is to retrain our model with synthetic hands from a solver that approximates Nash equilibrium, using a CFR-like algorithm. We used the commercial solver InstaGTO. This time, we used reinforcement learning with ORPO instead of fine-tuning, which allows us to give the best answer in a state, and also the worst.

We also trained it with real hands played by me to prevent catastrophic forgetting.

So we trained it, and we refered to it as SpinGPT : it’s our latest model.

SpinGPT imitation results

• On the pro test set (n = 32 000)
  • Accuracy: 79% (83% with tolerance)
• On the solver test set (n = 30 000)
  • Accuracy: 72% (78% with tolerance)

On the imitation side, we can see that it achieves similar accuracy to SpinGPT-SFT on the pro dataset, and on the solver dataset, the amount is slightly lower, because, of course, it is more difficult to approximate a solver’s policy than a human’s one.

SpinGPT > SpinGPT-SFT

• Head-to-head: SpinGPT-SFT vs SpinGPT
  • 10,000 hands (duplicate format)
  • Heads-up (1v1), 25BB
  • 13.2 ± 7.24 BB/100 for SpinGPT
• –> RL was effective

We didn’t play SpinGPT against Slumbot because the evaluations require a lot of computations and the game format remains mismatched, but we did play it against the first version, SpinGPT-SFT, in a convenient format (HU, 25 BB). And we can see that RL works, with SpinGPT beating SpinGPT-SFT by a wide margin.

Main limitations

• Absolute strength of SpinGPT unknown
  • No matches vs spin-and-go specialist AIs
  • No matches vs humans
• Numeric understanding issues (e.g., 5 < 5.2 < 5.11?!)

Unfortunately, there is limitations. Main one is that absolute strength of SpinGPT is unknown.

It’s difficult to evaluate our agent, because there is no online Spin & Go bot. Slumbot exists, but it’s not exactly the same format.

Actually, the best evaluation is against humans. And that’s what we’re going to do in the coming weeks.

Also, as an LLM, there is some numeric parsing issues, like considering 5.11 bigger than G.2. But it’s a minor issue.

Improvements and outlook

• Stronger base model
• Hyperparameter tuning
• Hybrid with a solver (preflop, postflop HU, push-or-fold)
• Opponent adaptation
• Explanation of decisions
• …

There is plenty of improvements possibles. I will just focus on one: combine a solver, that can compute Nash equilibrium in certains circuntances, and a LLM, to manage others. That’s crucial in tournaments.

Conclusion

• Two stages pipeline: SFT on expert data, then ORPO on solver data
• SpinGPT-SFT beats Slumbot (13.4 ± 12.9 BB/100)
• SpinGPT outperforms SpinGPT-SFT (+13.2 ± 7.24 BB/100)
• Tolerant accuracy: 83% on human test set, 78% on solver
• Benchmark ≠ target (Spin & Go format): no human evaluation yet (planned)
• Try SpinGPT: spingpt.lamsade.fr

To conclude: We trained SpinGPT in two stages (SFT on expert data and then RL on solver data). Versus Slumbot, SpinGPT-SFT won, but it’s not clear to what extent. Then, SpinGPT clearly outperforms SpinGPT-SFT.

In imitation, SpinGPT matches my professional actions 83% of the time and the solver’s actions 78% of the time.

The benchmark we used, Slumbot, is not the Spin & Go target format, so absolute strength remains uncertain. But evaluations against humans are planned.

You can play against SpinGPT at this website. And the game started.

Thanks for listening ! I will be happy answer any questions you may have.

• Questions?

Thanks for listening ! I’m happy to take your questions.