Back in 2023, when ChatGPT was still new and the whole field felt like it was moving weekly, I wrote a short post about a simple idea: chain-of-thought prompting. The trick was almost embarrassingly low-tech. Instead of asking a language model for an answer directly, you fed it intermediate reasoning steps first — like a teacher encouraging a student to show their work. Multi-step reasoning got better, and as a bonus you could actually see how the model arrived at its answer.
It was one of my most-read posts. And now, three years later, I’ve been thinking about how right it was — and how wrong.
What held up
The core intuition turned out to be durable: reasoning is computation spread across tokens. A model that answers in one token has one forward pass worth of thinking. A model that writes out a chain of steps gets a forward pass per step. More tokens, more thinking. That observation from 2023 didn’t age at all — it became, in some sense, the entire direction of the field.
Everything that happened after — longer contexts, “think step by step,” self-consistency sampling, agents that loop — was some version of giving models more tokens to think in.
What surprised me
What I did not predict was that chain of thought would move from the prompt into the weights.
The reasoning models we have now — o1-style systems, open-weight R1 derivatives — aren’t just being prompted to reason. They’re trained to reason. The chain of thought became the training objective: reinforcement learning over reasoning traces, process supervision instead of just outcome supervision, rejection sampling where only correct traces get reinforced, and then distillation so smaller models inherit the behavior.
In other words, the trick I wrote about as a prompting technique in 2023 became, by 2025, the dominant training paradigm. The “show your work” instruction got replaced by “we trained you on millions of examples of showing your work, and rewarded the ones that led to the right answer.” Test-time compute scaling — the idea that you can keep getting smarter just by letting the model think longer — is basically chain of thought as a scaling law.
I had the direction right and the mechanism wrong. Not a bad score for 2023.
What reading “Build Reasoning Models from Scratch” taught me
Lately I’ve been working through Build Reasoning Models from Scratch, implementing the machinery myself rather than just reading papers about it. And building it changed how I see the original idea.
Prompting hides the hard parts. When you write “think step by step,” you don’t see the reward design problem: how do you grade a reasoning trace? Outcome supervision (was the final answer right?) is easy to implement and often good enough. Process supervision (was each step right?) is better in theory and much harder in practice. Rejection sampling, majority voting, the question of how long a trace should be allowed to run before it becomes noise — none of this exists at the prompt level, and all of it matters at the training level.
The humbling realization: the 2023 version of me understood the interface of reasoning models. Building them teaches you the mechanics — and the mechanics are where the interesting engineering lives.
The practical takeaway
If you’re building with LLMs today, here’s my honest, practitioner take:
- Chain of thought still works as a debugging tool. It’s the cheapest diagnostic in the field. If a model can’t reason its way to an answer when prompted, training won’t magically fix it. CoT is where you check the reasoning before you spend money on training.
- The leverage moved. In 2023, prompting was the skill. Now the skill is knowing when reasoning needs to be trained in rather than prompted out — and when the plain old trick is good enough.
- Showing your work matters for users, too. We forget this, but the interpretability benefit of CoT was always half the point. In production systems, a visible reasoning trace is often the difference between a model you can trust and one you can’t.
Three years ago I wrote about a prompting trick. Now I’m building the training loops that made the trick obsolete — and I keep coming back to the same lesson: the simple ideas that survive are the ones that turned out to be load-bearing. Chain of thought was load-bearing.
Next time: I’m going to revisit my Google Next 2018 talks on applied ML for media and publishing, and walk through what that whole pipeline would look like rebuilt in 2026. Some of it aged surprisingly well. Some of it is unrecognizable.
What surprised you most about how fast this moved? I’d genuinely like to know — the field stopped being predictable around the time my 2023 post went live.