Not every AI answer needs a paragraph: Hugging Face makes decision models a first-class citizen 🎯

A lot of what we ask LLMs to do in real software isn't writing at all. Which team should handle this ticket? How urgent is it? Is the customer asking for a refund? Usually we prompt a chatbot to answer in JSON and hope it sticks to the schema. On 10 November, Aritra Roy Gosthipaty, Lysandre, Pedro Cuenca and Tom Aarsen published a post on the Hugging Face blog that takes a cleaner route: decision models, now with proper support across the Hugging Face ecosystem.

What a decision model does

The interface is simple: state plus questions plus allowed answers go in, decisions plus probabilities come out. No text is generated. You hand the model some context (a string or a JSON object), name your questions, and it scores the answers your application already understands. There are three primitives:

  • Choice picks one option and returns the full probability distribution.
  • Score rates something on an ordered scale and reports the expected value, so an urgency of 1.7 on a 0 to 2 scale means "probably high, maybe medium".
  • Noul (from Bernoulli) returns the probability that a yes/no statement is true.

Choice and Score also return a normalized confidence value, which is 0 for a uniform guess and 1 when all belief sits on one option. In the blog's example request, the response reports 148 input tokens and 0 output tokens.

Why I like the approach

The authors are honest that classification with probabilities, or labels you define at request time, is nothing new; zero-shot classifiers have done that for years. What's new is a shared contract: several typed questions in one request, non-generative scoring, and room for architectures trained specifically for calibrated decisions. Hugging Face's stated goal is a common interface without forcing a common implementation, and the pieces are already landing:

  • Transformers has a decision pipeline, AutoModelForDecision, DecisionConfig, and a /v1/systemone endpoint in transformers serve.
  • llama.cpp serves the same /v1/systemone API, for example with the ggml-org/Kev-4B-GGUF model.
  • The Hub has a decision-model tag, covering encoder models like ModernBERT as well as larger generative backbones.
  • Training works with the regular Trainer and a plain cross-entropy loss over the options, TRL support is in an open pull request, and Unsloth has a LoRA recipe to turn a generative model into a decision model.

My favourite detail: the same API also works with compatible ordinary autoregressive LLMs. Transformers builds the decision prompt, reads the logits of the answer labels in one batched forward pass and applies softmax. So you can try the interface today with a model you already run, without training anything new.

Why this matters for agents

Agents are full of small decisions: route this, escalate that, call a tool or don't. Getting real probabilities instead of a confident-sounding sentence means your code can set thresholds, ask a human when confidence is low, and log why it did what it did. For a learning platform, I can picture questions like "which topic is this student stuck on?" answered in milliseconds on modest hardware. I covered the Swift side of this idea in October; seeing it arrive in Transformers and llama.cpp feels like the moment it becomes a normal building block.

Give the decision pipeline a try on one of your own routing problems. It might be the most boring, and most useful, upgrade your agent gets this month. 🙂

Sources

Reacties

Een reactie posten

Populaire posts van deze blog

Meta just shared the full recipe for a 30B agent model you can run at home 🧠

When Meta released Muse Glimmer in August, the pitch was simple: an open-weight, Apache 2.0 model built for AI agents that run on your own Mac or PC. What was missing was the "how". On 19 October, Meta Superintelligence Labs published the full technical report, and for anyone who cares about distillation and local agents, it is one of the most useful reads in a while. What the model is Muse Glimmer is a dense model of about 29.6 billion parameters, including a 1.8B perception encoder, so it reads text and images and writes text. It has a 128K+ context window and was trained on data from more than a hundred languages. The design goal was strict: everything has to fit in 32 GB of GPU memory, and a smaller 4-bit variant targets 24 GB cards. The parts I found most interesting Dense beat mixture-of-experts here. That surprised me. The team trained scaling ladders for a dense model and two MoE designs with the same memory footprint. Under a fixed memory ceiling and their compu...
Meer lezen

SpaceX just asked to fly Starship hundreds of times a year 🚀

Last week the U.S. Federal Aviation Administration took a quiet but important step: it opened a formal review of SpaceX's request to launch Starship from Starbase in South Texas up to 456 times per year . Today the site is cleared for only 25 flights a year. That jump is the clearest signal yet that Starship is meant to become routine hardware, not a rare test rocket. What the FAA is reviewing On 14 October 2026 the FAA published a Notice of Intent to prepare a Programmatic Environmental Impact Statement (PEIS) for SpaceX's Starbase plans. A PEIS looks at a whole program of operations rather than a single launch, which is exactly what you need if you want a reusable rocket to fly the way airlines fly planes. SpaceX's proposal comes in two phases, according to the Federal Register notice: Phase 1: up to 100 launches and 200 landings per year (100 Ship, 100 Super Heavy), any day or time of week. Phase 2: up to 456 launches and 912 landings per year at Starbase. La...
Meer lezen

AI that can actually "see" a crystal 🔬

Most of the solid stuff around us, from battery materials to chips, is made of crystals: atoms stacked in neat, repeating 3D patterns. Large language models are great at reading papers about those materials, but they have never really looked at the structures themselves. A team at MIT just showed a clever way around that, and the trick is one I keep coming back to: good synthetic data. What they built The model is called CLaC, short for Contrastive Language-Crystal model, and it is described in a paper in Nature Communications by Yang Jeong Park, Mayank Kumaran, Chia-Wei Hsu, Elsa Olivetti and Ju Li. MIT News covered it on 18 October 2026. If you know CLIP, the model that taught image generators to connect pictures and words, you already get the idea. CLaC has two branches: a graph neural network that reads the 3D crystal structure, and a text encoder that reads a description. Contrastive training pulls matching structure-text pairs together in one shared space and pushes mismatche...
Meer lezen