Clover writes text in blocks, not word by word, and it's open for everyone to try 🍀

Almost every language model I use writes the way we type: one token after another, left to right. Diffusion language models take a different route. They start from a block of blanks and fill in the positions they're most sure about first, several at once, refining the block over a few passes. On 28 October, DiffuSpace, a new lab founded by members of the University of Hong Kong NLP group behind Dream 7B, released Clover-35B-A3B-Preview, and it's the most serious open attempt at that idea I've seen so far.

What Clover is

Clover is a mixture-of-experts model with 35 billion parameters in total and about 3 billion active per token. It has a 128K-token context window, a thinking mode, and an Apache 2.0 licence. The weights are on Hugging Face and ModelScope, and the code, including SGLang and llama.cpp support, is on GitHub.

The part I find most interesting: it wasn't trained from scratch. Clover is derived from Alibaba's open Qwen3.6-35B-A3B and keeps its backbone of 40 layers with 256 routed experts. DiffuSpace converted an existing autoregressive model into a diffusion one. That's a great example of what open weights make possible. One team's release becomes the starting point for a completely different kind of model.

How the block decoding works

According to the model card, each block starts as 32 masked tokens. At every step, positions with confidence above 0.95 get filled in parallel; if none pass that bar, the single most confident one is filled so the model always makes progress. A finished block then becomes part of the prefix for the next one.

To make this work with Qwen's hybrid Gated DeltaNet layers, the team redesigned them. Earlier context is compressed into the recurrent state, while tokens inside the current block can look at each other in both directions. There's also a neat efficiency trick: during denoising, a block can share a smaller subset of experts, so fewer distinct experts are touched per block.

Why speed matters for agents

DiffuSpace's argument is one I agree with: for agents, speed is a capability. An agent loops through plan, act, observe and adjust, and every second of generation adds up across that loop. The team reports about 680 tokens per second on a single NVIDIA GB300 GPU (BF16, batch size 1, on its internal agent data), and an INT4 version running on a 16 GB RTX 5070 Ti at about 101.8 tokens per second on MATH-500. On its own agent benchmarks it reports 73.6 on SWE-bench Verified and 51.7 on Terminal-Bench 2.1.

An honest note: these are DiffuSpace's own measurements, and the release is explicitly a preview. The model card even mentions that very long generations can get repetitive, which is why it ships with repetition and length controls switched on. I'd love to see independent evaluations before drawing big conclusions.

Why I'm excited anyway

The blog also shows an education partner, SigmaZ, using Clover to generate interactive lessons, like a 3D cell you can click through. Fast generation of interactive learning material is exactly the kind of thing I'd love to see in classrooms. And within a day, the community had already published GGUF quantizations on Hugging Face. That's open source doing what it does best.

Sources

A new way to generate text, built on someone else's open model, and shared openly again. That's a loop I'm happy to watch grow.

Reacties

Een reactie posten

Populaire posts van deze blog

Meta just shared the full recipe for a 30B agent model you can run at home 🧠

When Meta released Muse Glimmer in August, the pitch was simple: an open-weight, Apache 2.0 model built for AI agents that run on your own Mac or PC. What was missing was the "how". On 19 October, Meta Superintelligence Labs published the full technical report, and for anyone who cares about distillation and local agents, it is one of the most useful reads in a while. What the model is Muse Glimmer is a dense model of about 29.6 billion parameters, including a 1.8B perception encoder, so it reads text and images and writes text. It has a 128K+ context window and was trained on data from more than a hundred languages. The design goal was strict: everything has to fit in 32 GB of GPU memory, and a smaller 4-bit variant targets 24 GB cards. The parts I found most interesting Dense beat mixture-of-experts here. That surprised me. The team trained scaling ladders for a dense model and two MoE designs with the same memory footprint. Under a fixed memory ceiling and their compu...
Meer lezen

AI that can actually "see" a crystal 🔬

Most of the solid stuff around us, from battery materials to chips, is made of crystals: atoms stacked in neat, repeating 3D patterns. Large language models are great at reading papers about those materials, but they have never really looked at the structures themselves. A team at MIT just showed a clever way around that, and the trick is one I keep coming back to: good synthetic data. What they built The model is called CLaC, short for Contrastive Language-Crystal model, and it is described in a paper in Nature Communications by Yang Jeong Park, Mayank Kumaran, Chia-Wei Hsu, Elsa Olivetti and Ju Li. MIT News covered it on 18 October 2026. If you know CLIP, the model that taught image generators to connect pictures and words, you already get the idea. CLaC has two branches: a graph neural network that reads the 3D crystal structure, and a text encoder that reads a description. Contrastive training pulls matching structure-text pairs together in one shared space and pushes mismatche...
Meer lezen

A 35B model on a Mac mini, using just 2.9 GiB of memory 🚀

There's something special about running a capable language model on hardware you own: no API bill, no data leaving the house, just a small box on your desk doing the thinking. A paper that's currently #2 on Hugging Face's trending list takes that idea further than I expected: it runs a 35-billion-parameter model on a Mac mini at about 20 tokens per second while using under 3 GiB of memory. The trick: leave the experts on the SSD The paper is called The Other Half of the Memory Wall: Serving 35B MoEs from SSD with Trained Routing Prediction , and the system is named Edge0 . It builds on a Mixture-of-Experts (MoE) model, Qwen3.6-35B-A3B. In an MoE, each token only uses a few "experts" per layer (here 4 out of 256 in each of 40 layers), so most of the weights sit idle at any given moment. Edge0 takes that literally. The expert weights (quantized to int4) stay on the SSD and are streamed in only when needed. The clever part is a small prerouter : a tiny trained he...
Meer lezen