Skip to content

The Data Scientist

Building AI agents for imperfect information games: what poker research got right

Since Deep Blue defeated Kasparov in 1997, I have been interested in AI research as applied to games. In turn, after chess, Go, StarCraft and Diplomacy fell each few years; poker remained an elusive target for a long period of time. However, the reasons behind poker’s resistance to solution are truly enlightening to anyone developing AI agents today in 2026.

No-limit Texas Hold’em was not solved by scaling up the same methods used to crack chess. Instead, poker researchers needed to develop entirely new methodologies — and these methodologies are beginning to show up in negotiation AI, cybersecurity, autonomous trading and dozens of other areas where you make decisions without having access to complete information.

Why perfect-information methods fail

Let’s start with the obvious. Chess contains approximately 10^47 possible game states. Go has around 10^170 possible game states. However, no-limit Texas Hold’em has 10^164 information sets — not game states — information sets, because each player views different cards. This difference is critical.

In chess, minimax with alpha-beta pruning works well simply because both players view the exact same board. Thus, you may create a game tree, assess position values, and search ahead. AlphaGo extended this with Monte Carlo Tree Search guided by neural networks. While brilliant, it remains fundamentally: examine the state, assess it, and search forward.

However, in poker, you cannot assess a state because you do not know the state. Your opponent’s hole cards are concealed. His strategy is concealed. Whether or not he is currently bluffing is concealed. Therefore, you will attempt to construct a model of what your opponent might possess based upon the pattern of bets — and he knows you are attempting to do this; therefore he will alter his behavior to manipulate your model.

Poker is a structurally different problem from chess — it is not merely a harder version of chess.

Counterfactual regret minimization: the breakthrough idea

The initial breakthrough was an extremely simple concept. Rather than seeking to determine the “optimal” strategy for a given scenario, instead seek to determine how much you regret each decision you made.

Counterfactual Regret Minimization (CFR) was developed at the University of Alberta in 2007. CFR operates as follows: play a hand against yourself, observe the result of the hand, then inquire at every decision point within that hand — “how much better could I have performed by taking an alternate course?” Aggregate those regrets over millions (and eventually billions) of self-play iterations. The resulting strategy will minimize total regret and, due to the minimax theorem, converge to a Nash equilibrium.

Carnegie Mellon’s Libratus used a variation known as Monte Carlo CFR to defeat four top human experts in heads-up no-limit Texas Hold’em in January 2017. Libratus calculated its strategy during the tournament while refining its strategy nightly based upon what its opponents did that evening. Libratus ultimately earned $1.77 million over 120,000 hands. Each of the four experts went into negative territory.

Meta AI’s Pluribus expanded upon the methodology two years later to enable six-player games using a method known as “depth-limited solving with real-time search.” Rather than attempting to solve the entirety of the game tree (which is computationally impossible for six players), Pluribus solved smaller subgames in real time as they occurred, and stitched together those local solutions to form a cohesive strategy.

How neural networks entered

Deep CFR, published in 2019, replaced the tabular regret storage utilized by pure CFR with neural networks capable of generalizing across similar scenarios.

Most recently, Meta’s ReBeL framework combined recursive belief-based learning with neural value functions â€” essentially providing a means for training a network to reason about each player’s current estimate of the other players’ current estimates of their actions — recursively. As difficult as this sounds, it represents exactly the type of thinking involved whenever you consider “he knows that I know that he’s bluffing.”

SpinGPT authors published a study in late 2025 demonstrating that fine-tuning a large language model with solver output from three-player Spin & Go tournaments resulted in a recommendation accuracy matching solver recommendations 78% of the time and demonstrated a positive win rate relative to benchmarks provided by bots. Hybrid architectures combining LLMs trained on domain-specific data represent a potentially effective path forward for many imperfect information application developers.

General lessons for AI developers

As I reflect on nearly ten years observing developments in this area, here are some things that I believe are generally applicable beyond poker:

Self-play should be used more often. 

Most AI development continues to rely upon labeled data from humans or supervised learning from expert demonstrators. Poker AI demonstrated that self-play (where the system produces its own training data through competition against itself) can produce strategies that no human would demonstrate. Libratus demonstrated that its strategy diverged significantly from any strategy recommended by any of the four human professionals it competed against; however, Libratus clearly produced better performance than each of them. If your domain possesses a clear objective function, self-play should likely be your first experiment, rather than your last.

Minimizing regret is better than maximizing reward in adversarial environments. 

Reinforcement learning typically seeks to maximize cumulative reward, which works well for single-agent problems. However, in adversarial environments (when facing an adaptive opponent), maximizing cumulative reward leads to exploitation — your opponent determines your strategy and adjusts accordingly. Regret-minimizing agents converge toward unexploitable equilibria — which provides greater robustness. If you’re developing anything that encounters adversarial input — cybersecurity, auction design, competitive pricing — this is relevant.

Real-time search is preferable to pre-computed solutions for all combinatorial problems. 

Pluribus did not generate a pre-computed strategy prior to deployment and then execute that pre-computed strategy throughout its run. Rather, Pluribus generated partial solutions for small subsets of the game tree online as it played. This allows for handling the combinatorial explosion inherent in pre-computing a complete solution — and allows the solution to adapt specifically to the opponent’s behavior during execution. Real-time inference over learned value functions is generally preferable to looking up in pre-computed policies for production AI systems encountering novel conditions.

Quantifying uncertainty represents the bottleneck for all domains with imperfect information. 

The difficult task associated with playing poker is not computing odds — an undergraduate student can calculate pot equity. The difficulty lies in maintaining accurate probability distributions regarding each player’s possible hands, properly updating those probabilities with each action taken — and propagating those uncertainties through multi-step plans. All imperfect information domains have analogous bottlenecks: what do I not know? How certain am I regarding my assessments? And how does that uncertainty affect my subsequent decisions?

Present developments

Google DeepMind introduced a poker track on its Kaggle Game Arena in early 2026 and pitted emerging models against one another. Early results confirmed what PokerBattle.ai discovered: general-purpose LLMs remain mediocre poker players despite excelling at numerous other reasoning tasks. Architecture is less important than training signal.

Far more significant is the proliferation of poker AI techniques into various other disciplines. Algorithmic trading firms have employed poker AI researchers for several years. Red-teaming tools for cybersecurity employ CFR variations for planning adversarial attacks. Negotiation AI startups are developing belief-space planning techniques first created for multi-player poker.

Poker was never solely about cards. It was always the best testing ground available for decision-making under uncertainty against adaptive adversaries. Given that the research community has largely solved poker itself, techniques are now migrating into areas where they arguably matter greatly more.

This is the real payoff — not the $1.77 million that Libratus earned, but the framework for reasoning under uncertainty that poker research has made possible.