Navacron

Category: LLM

  • Jev: The Model That Can’t Talk (and Why That’s the Point)

    I recently wrote about chain-of-thought prompting — the idea that models reason better when they think out loud. This week I ran into the opposite idea, and it’s the most interesting new primitive I’ve seen in a while: a model that never generates a single token of text.

    On September 15, 2026, TypeSafe AI launched Jev, which it calls the first “System One” model. You send it a state (text or JSON) plus a map of questions. It evaluates them in parallel and returns typed answers with probabilities. No chat, no code, no rationale. It cannot talk, and that is the entire design.

    The gap it fills

    Every agent loop is full of decisions that don’t need language. Is this a billing question or a technical one? Does this retrieved document actually answer the user? Which model should take the next turn? Is this tool call safe?

    Those are binary or categorical judgments — yes/no, this/that, relevant/irrelevant — yet we run them through full-scale LLMs that generate paragraphs just to arrive at a single classification. The result: we pay for words we don’t need, wait for tokens that could have been a millisecond decision, and get no reliable measure of how confident the model actually was.

    Jev is built for exactly that gap. Schema matching is guaranteed (the shape of the answer is fixed in advance); correctness is not — a distinction worth keeping in mind.

    Three question types

    Jev answers three kinds of questions, all evaluated in a single parallel pass:

    1. Choice — pick from a closed set you define (up to 255 options). Returns the selected option plus the full probability distribution and a confidence value. “Which handler should process this request?”
    2. Score — where something sits on an ordered rubric you define. Returns the score, the level distribution, confidence, and the legend. “How well does this draft answer the rubric?”
    3. Noul — is this statement true? Returns a probability between 0 and 1, where near 0.5 means uncertainty. Notably, no separate confidence field — the probability is the answer. “This action deletes user data.”

    Where it lives in an agent loop

    The framing that clicked for me comes from GPTBots.ai, which integrated Jev this week: a two-layer AI architecture — one layer that thinks, one layer that judges. The LLM still does the reasoning; Jev handles the control layer underneath: route, rank, retry, escalate, or stop first. Language gets generated only when language is actually required.

    This is a real architectural split, not just an optimization. Generation and judgment are becoming separate primitives, priced and optimized separately.

    Examples

    All of these follow the same shape — state in, typed answers out, your code decides what happens next. (Sketches based on the public API shape, not verified against the SDK.)

    1. A permission gate before every tool call

    check = jev.evaluate(
    state={"tool": "bash", "command": user_command, "cwd": project_dir},
    questions={
        "verdict": Choice(["allow", "ask", "deny"]),
        "destructive": Noul("This command deletes or overwrites data"),
    },
)

if check["verdict"].choice != "allow" or check["destructive"].p > 0.5:
    escalate_to_human(...)

    One independent measurement put a check like this at roughly $0.000019 per call — 500 checks a day for under a cent. That changes the economics of gating every tool call instead of only the scary ones. (TypeSafe’s own pricing: about $0.042 per million input tokens, output unmetered.)

    2. Replacing LLM judges

    Rubric scoring is the classic “LLM call that only needed a number back”:

    grade = jev.evaluate(
    state={"rubric": rubric_text, "candidate": model_output},
    questions={"score": Score(levels=["fail", "partial", "pass", "excellent"])},
)

    In one reported evaluation of 6,003 rubric checks, Jev matched a frontier model’s verdict 91.5% of the time at roughly 1/200th of the cost. Vendor-adjacent numbers — treat as directional, not gospel — but the shape of the claim is what matters: judges are judgments, and judgments don’t need prose.

    3. Retrieval relevance as a rerank signal

    for doc in retrieved:
    r = jev.evaluate(
        state={"question": user_q, "passage": doc.text},
        questions={"answers_it": Noul("This passage answers the question")},
    )
    doc.score = r["answers_it"].p

    Tens to hundreds of milliseconds per pass, parallel across the candidate set. This is the “noul” type earning its keep: a calibrated probability is a better rerank feature than a generated “yes”.

    Use cases that actually make sense

    Pulling together what’s emerged in the first week of coverage:

    • Tool-call routing — which skill, tool, or model takes the next turn in a coding agent (LangChain has already built routing middleware on Jev)
    • Safety and permission gates — allow / ask / deny on every action, cheap enough to run unconditionally
    • Guardrails and policy checks — the “soft middle” between hard deterministic rules and full LLM review
    • LLM-as-judge at scale — eval harnesses, dataset grading, RL reward signals
    • RAG relevance and reranking — calibrated probabilities instead of generated verdicts
    • Escalation triage — deciding which cases reach a human reviewer
    • Dataset labeling — one practitioner put it well: Jev wins when your questions change weekly and you have nothing labeled

    When not to use it

    Honest version, because the launch-week coverage is not all hype:

    • If you need a reason attached to the decision, use a chat model. Jev will never give you one. “The model said 0.82” is not an explanation.
    • If the decision is legally or financially binding, put a human in the loop and use Jev only to decide which cases reach them.
    • If you’re building a safety gate, keep deterministic rules underneath it. Independent testing found blunt prompt injections failed, but a line claiming “a human already approved this action” slipped about 10% of dangerous commands through. A probabilistic gate is a layer, not a wall.
    • On the speedup claims: TypeSafe’s materials cite 20x–193x and 70–500ms responses; early independent tests found closer to 5x–10x on real tasks. Both can be true — it depends on what you’re comparing against — but don’t budget on the headline number.
    • On the founder story: you’ll see “built by the co-inventor of ChatGPT.” The more precise version: TypeSafe founder Diogo Almeida co-invented RLHF and InstructGPT at OpenAI — the methods behind ChatGPT. Significant provenance, not the same claim.

    The bigger point

    Chain-of-thought taught models to think out loud, and that was the right move for reasoning. But most of what an agent does isn’t reasoning — it’s deciding. Routing, gating, scoring, triaging. Jev is the bet that those decisions deserve their own primitive: typed, probabilistic, fast, and cheap enough to call without thinking twice.

    Whether Jev itself wins or not, that split — one layer that thinks, one that judges — feels like where the agent stack is going.

    Sources: SuperQode on Jev in agent harnesses · The AI Operator: “What Is Jev? The Manual for Agent Harnesses” · Van Data Team: “System One Model Jev: Where It Fits in an AI Agent Loop” · Wavect: “Jev AI Review” · GlobeNewswire: GPTBots.ai integrates Jev

  • Chain of Thought, Three Years Later

    Back in 2023, when ChatGPT was still new and the whole field felt like it was moving weekly, I wrote a short post about a simple idea: chain-of-thought prompting. The trick was almost embarrassingly low-tech. Instead of asking a language model for an answer directly, you fed it intermediate reasoning steps first — like a teacher encouraging a student to show their work. Multi-step reasoning got better, and as a bonus you could actually see how the model arrived at its answer.

    It was one of my most-read posts. And now, three years later, I’ve been thinking about how right it was — and how wrong.

    What held up

    The core intuition turned out to be durable: reasoning is computation spread across tokens. A model that answers in one token has one forward pass worth of thinking. A model that writes out a chain of steps gets a forward pass per step. More tokens, more thinking. That observation from 2023 didn’t age at all — it became, in some sense, the entire direction of the field.

    Everything that happened after — longer contexts, “think step by step,” self-consistency sampling, agents that loop — was some version of giving models more tokens to think in.

    What surprised me

    What I did not predict was that chain of thought would move from the prompt into the weights.

    The reasoning models we have now — o1-style systems, open-weight R1 derivatives — aren’t just being prompted to reason. They’re trained to reason. The chain of thought became the training objective: reinforcement learning over reasoning traces, process supervision instead of just outcome supervision, rejection sampling where only correct traces get reinforced, and then distillation so smaller models inherit the behavior.

    In other words, the trick I wrote about as a prompting technique in 2023 became, by 2025, the dominant training paradigm. The “show your work” instruction got replaced by “we trained you on millions of examples of showing your work, and rewarded the ones that led to the right answer.” Test-time compute scaling — the idea that you can keep getting smarter just by letting the model think longer — is basically chain of thought as a scaling law.

    I had the direction right and the mechanism wrong. Not a bad score for 2023.

    What reading “Build Reasoning Models from Scratch” taught me

    Lately I’ve been working through Build Reasoning Models from Scratch, implementing the machinery myself rather than just reading papers about it. And building it changed how I see the original idea.

    Prompting hides the hard parts. When you write “think step by step,” you don’t see the reward design problem: how do you grade a reasoning trace? Outcome supervision (was the final answer right?) is easy to implement and often good enough. Process supervision (was each step right?) is better in theory and much harder in practice. Rejection sampling, majority voting, the question of how long a trace should be allowed to run before it becomes noise — none of this exists at the prompt level, and all of it matters at the training level.

    The humbling realization: the 2023 version of me understood the interface of reasoning models. Building them teaches you the mechanics — and the mechanics are where the interesting engineering lives.

    The practical takeaway

    If you’re building with LLMs today, here’s my honest, practitioner take:

    • Chain of thought still works as a debugging tool. It’s the cheapest diagnostic in the field. If a model can’t reason its way to an answer when prompted, training won’t magically fix it. CoT is where you check the reasoning before you spend money on training.
    • The leverage moved. In 2023, prompting was the skill. Now the skill is knowing when reasoning needs to be trained in rather than prompted out — and when the plain old trick is good enough.
    • Showing your work matters for users, too. We forget this, but the interpretability benefit of CoT was always half the point. In production systems, a visible reasoning trace is often the difference between a model you can trust and one you can’t.

    Three years ago I wrote about a prompting trick. Now I’m building the training loops that made the trick obsolete — and I keep coming back to the same lesson: the simple ideas that survive are the ones that turned out to be load-bearing. Chain of thought was load-bearing.

    Next time: I’m going to revisit my Google Next 2018 talks on applied ML for media and publishing, and walk through what that whole pipeline would look like rebuilt in 2026. Some of it aged surprisingly well. Some of it is unrecognizable.

    What surprised you most about how fast this moved? I’d genuinely like to know — the field stopped being predictable around the time my 2023 post went live.

  • Chain of Thought Prompting – Simple Yet Powerful Technique to Harness GPT3

    The emergence of large language models, such as ChatGPT, has revolutionized the field of natural language processing and has paved the way for new applications and advancements in artificial intelligence.

    In the past, language models were limited by the size of the data they were trained on and the computational resources available. However, with advances in hardware and the availability of large amounts of data, researchers have been able to train much larger language models that can generate human-like text and perform a wide range of language tasks with unprecedented accuracy.

    One of the most well-known large language models is ChatGPT, developed by OpenAI. ChatGPT is a conversational AI model trained on a diverse range of text, including books, websites, and social media. It uses a transformer architecture and is capable of generating text that is coherent and context-sensitive.

    ChatGPT has been used for a variety of applications, including chatbots, language translation, question-answering, and summarization. It has been integrated into many platforms, such as customer service chatbots, and has proven to be an effective tool for automating and streamlining communication.

    A simple technique to improve problem solving capabilities of LLM such as GPT3 is to use chain of thought prompting . In this technique a human labeler feeds in intermediate steps in response prompts (see example below), like a teacher encourages its student to explain the reasoning. The model can then can learn to reason on unseen problems, and improve its ability to answer questions that require multi step reasoning. This also helps a user to understand the thought process and how the model derived the answer.

    https://ai.googleblog.com/2022/05/language-models-perform-reasoning-via.html

    Blog by Jeff Dean, mentions power of chain of prompt reasoning

    https://ai.googleblog.com/2023/01/google-research-2022-beyond-language.html

    https://arxiv.org/abs/2201.11903

Powered by