Claude Mythos: The Model That Thinks in Loops

Claude Mythos doesn't think once and answer. It thinks again. And again. Until it decides it's done.

Every AI you've used before works the same way: you ask something, it reads your question once, predicts word by word, outputs an answer. One pass. Done. Claude Mythos is suspected to work completely differently — same weights, run in a loop, each pass refining what the last pass got wrong. Like sleeping on a hard problem, except it takes 16 naps in a single second, all invisible, before writing a single word.

That's the hypothesis. In April 2026, a developer named Kye Gomez reverse-engineered the suspected design from leaked Anthropic documents and posted it to GitHub before Anthropic said a word publicly. This post walks through how it works — and how you can run it yourself today.

What normal AI models actually do

Think of every AI you've used as a straight line.

You type a question. The model passes it through layer 1, then layer 2, then layer 3 — each layer a separate set of trained weights, used exactly once, in sequence. GPT-4 has around 120 of these layers. Every layer adds to the model's file size, memory footprint, and compute cost. To make a model smarter, you add more layers. More layers means more parameters. More parameters means more GPUs to run it.

That's the wall every frontier model is running into. Not "how do we think better?" but "how do we fit more layers?" Claude Mythos appears to be designed around a completely different question: what if one layer could do the work of many?

The core idea: one layer, run over and over

The Mythos hypothesis comes down to one architectural change.

Instead of 6 unique layers with 6 sets of weights, the model has 1 layer with 1 set of weights. That layer runs 5 times (or however many times the input requires). Same weights on every pass. The output of each pass becomes the input for the next one.

The model is less to store. But just as deep in terms of how many processing steps the input goes through.

This class of architecture is called recurrent-depth. The word "recurrent" means the same thing it does in RNNs: a process that feeds its output back as its own input. The difference is that Mythos applies this idea to the full transformer layer, not just the hidden state.

LTI stability is what keeps those repeated passes from exploding. LTI stands for Linear Time-Invariant. It's a control theory concept that describes systems where the same input always produces the same output, regardless of when you apply it. In a looping model, without stability constraints, errors compound across passes and activations blow up. LTI stability prevents that. The layer is designed so that running it repeatedly keeps values in a bounded range instead of drifting to infinity.

Each pass refines the answer

Think of the passes like drafts.

Pass 1 is the first rough thought. The model reads the input and generates an early representation. It's not wrong exactly, it's just shallow.

Pass 2 takes that representation and processes it again. It can catch things the first pass missed. Contradictions get flagged. The answer starts to tighten.

Each subsequent pass runs the same weights on an increasingly refined representation of the problem. By pass N, the model has had N chances to work through the question. The output is the result of the final pass.

This is what the carousel describes as "Same brain. Run again." The weights don't change between passes. What changes is the quality of the representation those weights are working on. Each loop adds depth.

Depth-wise LoRA fits here. LoRA (Low-Rank Adaptation) is a technique for making models trainable with fewer parameters by approximating weight updates as the product of two smaller matrices. In a recurrent-depth model, depth-wise LoRA adds small adjustments at each pass so the same base layer can behave slightly differently on pass 2 than it did on pass 1, even though the core weights are shared. It's how the model avoids doing exactly the same thing every loop.

MLA attention (Multi-head Latent Attention) is the attention mechanism suspected to run inside each pass. Standard multi-head attention is expensive: the key and value matrices grow with the context window, and you need to cache them for every layer. MLA compresses the key-value cache by projecting it into a smaller latent space. In a model running the same layer repeatedly, this matters a lot. Without compression, memory costs multiply with each loop. MLA keeps them manageable.

MoE FFN (Mixture of Experts Feed-Forward Network) handles the feed-forward portion of each pass. In a standard transformer, every token activates every parameter in the feed-forward block. MoE uses a router to activate only a subset of "expert" subnetworks for each token. This means the model can have a high total parameter count in the FFN block while only using a fraction of those parameters per forward pass. In the Mythos context, MoE adds capacity to each loop pass without proportionally increasing the compute cost of running it.

Hard questions get more loops

This is the part that makes recurrent-depth genuinely different from just making a model bigger.

A standard model runs every input through every layer, every time. Whether you type "Hi" or "Explain the thermodynamic basis for the arrow of time," the compute cost is the same. The number of layers is fixed.

In a recurrent-depth model with ACT halting, that's no longer true.

ACT stands for Adaptive Computation Time. It's a mechanism that lets the model decide how many passes to run before producing its output. At each pass, the model outputs a "halting probability" alongside its representation. When the cumulative halting probability crosses a threshold, the loop stops and the current representation becomes the output.

The result is a model that allocates compute to match question difficulty:

Easy tokens get 2 loops. Hard tokens get 10. Nothing is wasted. The model doesn't spend 10 loops on "Hi" just because the architecture allows it.

This is also part of why, within 24 hours of Anthropic's public announcement, the US Treasury Secretary and Federal Reserve Chair convened an emergency closed-door meeting with major bank CEOs about this model's cybersecurity implications. A model that routes its own compute and decides how deeply to think about something is a different class of system than one that always runs the same fixed pass count. The control surface for alignment and capability predictions changes.

Someone rebuilt it open source

Anthropic never confirmed the architecture publicly. But Kye Gomez did the next best thing: he read the available evidence, inferred the design, and reverse-engineered a working implementation.

The project is kyegomez/OpenMythos on GitHub, published in 2026.

The architecture matches every clue that has leaked from Anthropic's research. The model spec leak in early 2026 exposed draft blog posts and internal documents referencing looped weight designs. The source map leak from @anthropic-ai/claude-code version 2.1.88 exposed internal feature flags and architecture notes consistent with recurrent-depth reasoning. None of it was a direct blueprint. Kye Gomez filled in the gaps.

The community shipped it before Anthropic said a word publicly.

How to run your own version

OpenMythos is the fastest path to running a recurrent-depth model locally. Here's the minimum you need.

What you'll need:

• Python 3.10+
• PyTorch 2.1+
• 8GB+ VRAM for small configs, 24GB+ for meaningful scale
• Git

Clone and install:

git clone https://github.com/kyegomez/OpenMythos
cd OpenMythos
pip install -r requirements.txt

The core loop in plain terms. OpenMythos defines a single transformer block, then wraps it in a loop with an ACT halting head. The halting head is a small linear layer that reads the current hidden state and outputs a scalar between 0 and 1. When the cumulative halting score passes a threshold (typically 0.99), the loop exits. Here's the conceptual shape:

# Simplified recurrent-depth forward pass
hidden = embed(input_tokens)

halt_acc = 0.0
n_steps = 0

while halt_acc < 0.99 and n_steps < max_loops:
    hidden = transformer_block(hidden)          # same weights every pass
    halt_prob = halt_head(hidden).sigmoid()     # how confident to stop?
    halt_acc += halt_prob * (1 - halt_acc)
    n_steps += 1

output = lm_head(hidden)

The transformer_block runs the same weights on every pass. The halt_head decides when to stop. The model never takes more passes than max_loops, so you can cap compute.

Depth-wise LoRA in practice. To let the model behave differently at each pass without separate weights, OpenMythos injects pass-index embeddings before the transformer block. Each pass gets a small learned offset that shifts the representation slightly. The base weights stay shared; the offsets give each pass its own character:

# Pass index conditioning
pass_embed = self.pass_embeddings(torch.tensor(n_steps))
hidden = hidden + pass_embed
hidden = self.transformer_block(hidden)

This is a lightweight version of depth-wise LoRA. A full implementation would use low-rank adapters per pass rather than learned offsets, but the principle is the same.

Training with ACT. The training loss adds a regularization term that penalizes unnecessary passes. Without it, the model would learn to always run the maximum number of loops regardless of input difficulty. The ponder cost pushes it to halt early when the representation is already good enough:

loss = cross_entropy_loss + lambda_ponder * n_steps.float().mean()

The lambda_ponder coefficient controls the tradeoff between answer quality and compute efficiency. Higher values produce faster, shallower models. Lower values produce deeper thinkers that use more passes.

Why this architecture matters

Three things change if recurrent-depth becomes the standard approach for frontier models.

Depth without the bulk. A model can reason as deeply as one with 6x the layers while storing only the weights of 1 layer. Smaller model files, lower memory requirements, cheaper to serve. You get the output quality of a large model at the storage cost of a small one.

Variable compute per token. The model spends compute where the problem is actually hard. A prompt that's 90% easy context and 10% hard reasoning doesn't pay the same per-token cost as a prompt that's uniformly hard. This changes what "efficient inference" means.

A new design axis for capability. The standard approach to making models smarter is to train larger models on more data. Recurrent-depth adds a different axis: depth of reasoning per forward pass. A model can be made "smarter" by allowing more loops, not just by adding parameters. Loops, not layers.

If Anthropic got this right, the playbook for scaling intelligence just changed. Not more weights. More passes.

What we don't know

It's worth being clear about what this is and isn't.

Anthropic has not publicly confirmed that Claude Mythos uses recurrent-depth architecture. The evidence is indirect: leaked model spec documents, source map contents, benchmark behavior that's consistent with adaptive computation, and Kye Gomez's reverse-engineering work. The architecture fits. It hasn't been verified.

OpenMythos is a community implementation of a suspected design. It is not an Anthropic product and does not reproduce any of Anthropic's actual training code or weights.

The practical Mythos model, if it exists as described, is Anthropic's most restricted release. It's not available to the public. What you can run today is OpenMythos: the architectural hypothesis made executable.

That distinction matters. Running OpenMythos teaches you how recurrent-depth works. It does not give you Claude Mythos.

Where the pattern applies

The recurrent-depth idea isn't unique to Mythos speculation. The same approach has appeared in academic research under different names: Universal Transformers (Dehghani et al., 2018), Pondering (Banino et al., 2021), and more recently in efficiency-focused work on depth-adaptive inference.

What Mythos represents, if the hypothesis is right, is Anthropic applying this research direction at frontier scale with production-grade engineering. That matters because prior recurrent-depth work mostly stayed in the research lab. Scaling it to a model capable enough that Anthropic restricted its access is a different class of result.

The open-source path exists. Kye Gomez built it. The architecture is understandable. The pieces (ACT halting, MoE FFN, depth-wise LoRA, MLA attention) are each independently documented in the research literature.

One loop at a time, the representation gets better. That's the whole idea.

Posted by @speedy_devv