Reinforcement learning · Search · Distillation

Teaching a machine to play Slay the Spire like a top streamer.

Slay the Spire is a deckbuilding roguelike with enormous branching: hundreds of cards and relics, random draws, hidden enemy intents, and a 50+ floor run where one bad fight ends everything. Spire Mind is a from-scratch AI that plays Ascension 20 Ironclad — the hardest difficulty — aiming to win as reliably as a skilled human. This page walks through how it's built: how the game is encoded, how the neural nets are constructed, how a tree search uses them, and how the whole thing trains itself.

brains: combat & out-of-combat

entity token types

1200+

MCTS sims per move

60+

per-encounter specialists

Where it actually stands — honest, not hype

This is a research system in active training, not a finished bot. On 450 matched fights against human replays it currently loses 20.4 HP per fight on average vs a human's 15.6, and matches-or-beats the human on 42% of them. It has one verified A20 win so far. The ~90% figure is the goal — roughly a skilled Twitch streamer — and every training cycle chips away at the gap. Watch it play, wins and deaths alike, on the live runs page.

scroll to explore ↓

Architecture at a glance

Two brains, one run

A run splits cleanly into two decision problems, and the AI uses a dedicated model for each. They talk to each other: the map brain asks the combat brain "how much will this fight hurt?"

⚔️

The Combat Brain

Inside a fight, every turn: which cards to play, in what order, on which target. It sees the literal battle state as a set of typed tokens and runs a Monte-Carlo tree search guided by a transformer that predicts the fight's outcome.

Input tokenized battle state
Model entity-token transformer
Decision MCTS + CVaR risk selection

🗺️

The Out-of-Combat Brain

Between fights: which path to take on the map, which cards to add or remove, what to buy, how to resolve events. It scores whole run states with a win-probability net that reasons over your owned set of cards and relics.

Input deck + relics + run context
Model owned-set attention net
Decision path planner over win-prob

Combat outcome head → danger estimate → Map path planner

Representation

Encoding the battle as a set of tokens

The key design choice: a fight is not a flat vector of numbers. It's a set of typed entity tokens — like words in a sentence — so a transformer can attend over the relationships between them (this card vs that enemy's intent, this relic vs this status).

CLSglobal summary: turn, floor, act, ascension, gold, enemies alive

PLAYERHP, block, energy, character, stance, strength/dex

CARDone per card, tagged by pile: hand / draw / discard / exhaust, upgrade, cost

MONSTERHP, block, and telegraphed intent — attack/defend/buff + damage & target

RELICowned relics, each an embedding

POTIONheld potions

ORBDefect orbs (channel slots)

Each token carries up to 55 scalar features in a shared "union" layout (a slot is filled only when it applies to that token type). Cards separated by pile is what lets the net know the difference between "a Bash in my hand" (playable now) and "a Bash still in my draw pile."

Why tokens, not a vector?

Decks are variable-length and combinatorial. A flat vector forces a fixed slot per possible card and loses relationships. Tokens + attention let the net learn synergies directly: an exhaust engine, a poison stack, a block-infinite loop each "read" differently because the tokens interact — not because someone hand-coded a feature for them.

⚡ The tokenizer is written in C++ (a byte-for-byte match of the Python reference) so the search can encode thousands of states per second.

The model

The combat net: one trunk, many heads

The tokens flow through a transformer trunk, then split into several prediction "heads." One net answers every question the search needs.

CLS

PLAYER

CARD×N

MONSTER×M

RELIC×K

entity tokens

▸

Transformer
self-attention over the token set

shared trunk

▸

Value categorical outcome distribution → CVaR

Policy which action to explore

Outcome end-HP · P(death) · turns · gold

Danger per-encounter fight cost (for the map)

heads

A distribution, not a number

The value head is categorical: instead of predicting a single "win chance," it predicts a distribution over how the fight ends (including the chance of dying). That distribution is what makes risk-awareness possible.

Speed-aware value

The training label grades wins by end-HP and speed — a clean 3-turn win scores above a sloppy 30-turn one — so the net learns to close fights, not just survive them.

The danger head

A dedicated head predicts a fight's HP-loss and death risk from the run state. This is the bridge to the map brain — and it's trained with each specialist so it always reflects how that exact model plays.

Decision making

Search: look ahead, then choose safely

The net is fast but shallow. Wrapping it in a Monte-Carlo Tree Search lets the AI look many moves ahead — playing out lines, backing up the net's value estimates, and concentrating effort on promising branches.

Select — walk down the tree by a confidence bound that balances the policy prior against explored value.

Expand & evaluate — the C++ engine applies the move; the net scores the new state (its value is the leaf — no slow random rollouts).

Back up — propagate the value up the path, survival-weighted so a line that risks death is penalized.

Pick with CVaR — at the root, filter to a death-safe set first, then take the best-value action among the robustly-explored ones.

CVaR: the survival instinct

A greedy "highest average value" pick will occasionally walk into a low-probability death. The root selection uses Conditional Value at Risk — it looks at the worst-case tail of each action's outcome distribution and refuses actions with real death risk when a safe one exists. Survival first, efficiency second.

The map brain

Scoring a whole run: owned-set attention

Between fights the question is "how good is my run right now?" The win-probability net answers it by attending over everything you own — deck and relics in one shared space — so synergies are valued in context.

Corruption

Feel No Pain

Dead Branchrelic

Barricade

Strike ×4

self-attention
every card/relic attends to every other

learned query pools the set →

P(win) = 0.71

Synergy is emergent

Dead Branch is mediocre alone but busted with an exhaust engine. Because the set self-attends, the net rates the combination higher than the sum — no hand-coded "combo" rules.

The path planner

A forward-simulating planner walks candidate map routes, pricing each room's HP cost and death risk with the danger head, and values the horizon with win-probability — then picks the survival-weighted best path.

Crediting the rewards

Elites are priced not just by their cost but by the relic they drop (valued against your actual deck), so the bot stops dodging elites when it's healthy — the way a strong human plays for relics.

Mixture of experts

A specialist for every fight

One shared net can't shift its play toward a hard boss without breaking its micro-decisions elsewhere. So each encounter gets its own fine-tuned specialist, routed by the encounter id — and each carries its own danger head.

Fight starts:
Hexaghost

→ route by id →

combat_state_enc20_player_1

combat_state_enc51…

combat_state.pt (fallback)

Same model plays it, same model prices it

Because each specialist carries the danger head trained on its own play, the map brain prices a Hexaghost fight with the exact model that will play it — the map brain always gets the memo when the combat brain improves. A missing head falls back gracefully to a shared one.

👻

Domain knowledge the net learns: Hexaghost's opening "Divider" deals damage that scales with your current HP (HP÷12 + 1) × 6. The engine simulates this exactly, so the specialist learns the counter-intuitive truth — sometimes you enter a fight low on purpose.

How it learns

The self-improvement flywheel

No human labels. The system generates its own training data by searching, then distills that search back into the net, then searches again with a stronger net — a loop that only ratchets upward.

1 A strong teacher search (thousands of sims) plays each encounter

2 Its decisions + outcomes become training data

3 The specialist net distills the teacher (learns to match it fast)

4 A gate promotes the candidate only if it beats the deployed model on a 1200-sim test suite — never a regression

5 The better net makes a better teacher → repeat

Coverage augmentation

Human data never shows every card in every fight, so specialists went blind to rare cards. Training batteries are augmented so every Ironclad card, colorless card, curse, and relic appears at least three times per encounter — no blind spots.

The ratchet gate

A candidate must pass every tactical case a deployed model ever passed, and fix at least one new one. It's mathematically monotonic — the bot can only get better.

Closing the self-gap

The real headroom isn't "beat humans" — it's the gap between the net and its own search. On the hard bosses that gap is 10–18% win rate, and distillation is steadily closing it.

Putting it together

The live decision stack

A live game streams state to the AI, which routes each decision to the right brain and plays it back — with watchdogs that keep it running and hot-reload new models.

Game (ModTheSpire + mod)

↕ state / commands

Decision router

Combat → specialist + MCTS + CVaR

Out-of-combat → win-prob + path planner

↓ deploy

Poller hot-swaps improved models as the fleet trains them

Every piece — the token encoding, the transformer heads, the CVaR search, the owned-set win-prob net, the per-encounter specialists and their danger heads, and the self-distilling flywheel — exists to answer one question well, thousands of times a run: what's the play that wins the most while dying the least?

Coming soon

Get your runs reviewed

Upload a run and the A20 bot reviews every decision: which card it would have drafted and why, how much HP each fight should have cost, and — for any fight — a step-through replay of the bot playing your exact spot. Join the waitlist and be first in when it opens.

Floor 12 · card reward example report

Uppercut bot's pick · +3.2% win

Anger your pick

Skip