The black box.

Known recipe, unreadable result.

How to train a model is well understood: architecture, data, an optimization rule. What comes out is not a program a person can read. These systems are grown more than they are built — the growing conditions are set by hand, the grown thing is not. There is no line of code that says if X, then Y — only billions of learned numbers whose joint behavior produces the answers.

Looking inside these systems, what we see are vast matrices of billions of numbers. These are somehow computing important cognitive tasks, but exactly how they do so isn’t obvious. Dario Amodei, The Urgency of Interpretability, 2025

This is the black box problem: full control over the growing conditions, weak insight into the grown thing. Not a bug someone will patch next quarter — a structural property of how these systems come to exist.

The builders say it themselves.

This is not an outsider’s critique. It is the field’s own, repeated admission.

People outside the field are often surprised and alarmed to learn that we do not understand how our own AI creations work. Dario Amodei, CEO of Anthropic, 2025

MIT Technology Review ran the state of the field as a headline: “Nobody knows how AI works” — noting that large language models can even do things they were never trained to do, in ways that current theory does not explain. Researchers there compare the field’s position to physics at the start of the twentieth century: strong results, missing theory.

Reading the inside, slowly.

The field is not shrugging. Mechanistic interpretability tries to open the box directly: Anthropic’s Mapping the Mind of a Large Language Model extracted tens of millions of human-readable concepts from a production model, and follow-up work on circuits traces individual steps of a model’s reasoning. The broader discipline of explainable AI has been surveying approaches for years (Adadi & Berrada, 2018). And there is a sharp counterposition: Cynthia Rudin argues that for high-stakes decisions the honest move is not to explain black boxes at all, but to use models that are interpretable by construction.

Real progress, openly contested methods, and a gap that is still wide. That is the current state.

A posture, not a complaint.

Working daily with systems nobody can fully read calls for a particular posture: observe before theorizing, claim only what the evidence carries, keep the uncertainty visible instead of performing past it. The umbrella page keeps that reminder to a single line. This page holds its sources.

Maintained questions.

• Does interpretability mature before the systems outgrow it? Amodei frames it as a race; the outcome is not decided.
• Where is the line between understanding a system in principle and being able to read it in detail — and which of the two do high-stakes uses actually require?

2026-07 — First version: problem, primary-source admissions, interpretability and its counterposition.