Google wants LLMs to sleep on it, and that might fix their memory problem 😴

Every large language model I use has the same quiet limitation: it stops learning the day it ships. It can read a long document in its context window and reason about it brilliantly, but once the session ends, that knowledge is gone. Fine-tune it on new data instead and you risk catastrophic forgetting, where learning something new damages what the model already knew. On 27 October, Google Research published a blog post about an idea I really like for breaking that trade-off: let the model sleep.

Wake, then sleep

The work comes from Ali Behrouz, Farnoosh Hashemi (Cornell), Adel Javanmard and Vahab Mirrokni, and is described in the paper "Language Models Need Sleep: Learning to Self-Modify and Consolidate Memories". The core idea is that a continual learner has no real split between training and inference. Instead, it alternates between two states:

  • Active (wake): the model works normally and keeps new information in fast, short-term memory, such as its context.
  • Sleep: the model stops taking new input and spends its compute on consolidating those memories and improving itself.

The authors are clear that "sleep" and "dreaming" are borrowed words. Under the hood this is an automated post-training cycle inside a fixed architecture.

Stage one: memory consolidation

During sleep, the model unlocks pre-allocated, low-rank mixture-of-experts (MoE) memory blocks. Short-term knowledge is then distilled into those new, more stable parameters, a step the team calls Knowledge Seeding. It combines on-policy distillation with a reinforcement-learning imitation signal that rewards the student when its completions match the teacher's text in meaning and in Levenshtein distance. Because the knowledge goes into new parameters, the older ones stay intact. Afterwards the temporary short-term parameters are reset, a bit like synaptic pruning.

As someone who follows distillation closely, I find this the most interesting part: it's distillation used not to make a model smaller, but to move knowledge from fragile memory into lasting weights.

Stage two: dreaming

The second stage is self-improvement. The MoE router deliberately activates unusual experts while the model generates synthetic practice examples, its "dreams", so different areas of knowledge get mixed. Those samples are filtered with gradient-based data selection, and the model fine-tunes on the best ones, checking with RL whether they actually help on downstream tasks.

What the results show

According to Google's post, the sleep framework kept a near-zero performance drop when learning translation for unseen languages such as Manchu and Kalamang step by step, while in-context learning, Cartridges and supervised fine-tuning forgot badly. On the BABILong long-context benchmark it scaled nearly perfectly up to 10M tokens by compressing context into parametric memory. For factual knowledge on SQuAD it reached 48.9% (single task) and 46.2% (continued pre-training), ahead of SFT and SEAL. And a sleep-enhanced base model scored 79.2% on AIME 24, 69.0% on AIME 25 and 46.1% on HMMT 25, beating SFT, GRPO and on-policy self-distillation.

Why this is exciting, and what to keep in mind

For AI agents, this is the missing piece: an assistant that actually remembers what it learned last week, without a full retrain. It also suggests a practical rhythm for homelab and on-device setups, where a model could consolidate overnight when the hardware is idle anyway.

To be fair, this is research, not a product. The paper has been on arXiv since June 2026 (an earlier version appeared on OpenReview in September 2025), the numbers are the authors' own, and sleep costs extra compute. I'd love to see independent replications and open code.

Sources

Maybe the best thing we can teach our models is something we all need more of: a good night's sleep.

Reacties

Een reactie posten

Populaire posts van deze blog

Meta just shared the full recipe for a 30B agent model you can run at home 🧠

When Meta released Muse Glimmer in August, the pitch was simple: an open-weight, Apache 2.0 model built for AI agents that run on your own Mac or PC. What was missing was the "how". On 19 October, Meta Superintelligence Labs published the full technical report, and for anyone who cares about distillation and local agents, it is one of the most useful reads in a while. What the model is Muse Glimmer is a dense model of about 29.6 billion parameters, including a 1.8B perception encoder, so it reads text and images and writes text. It has a 128K+ context window and was trained on data from more than a hundred languages. The design goal was strict: everything has to fit in 32 GB of GPU memory, and a smaller 4-bit variant targets 24 GB cards. The parts I found most interesting Dense beat mixture-of-experts here. That surprised me. The team trained scaling ladders for a dense model and two MoE designs with the same memory footprint. Under a fixed memory ceiling and their compu...
Meer lezen

AI that can actually "see" a crystal 🔬

Most of the solid stuff around us, from battery materials to chips, is made of crystals: atoms stacked in neat, repeating 3D patterns. Large language models are great at reading papers about those materials, but they have never really looked at the structures themselves. A team at MIT just showed a clever way around that, and the trick is one I keep coming back to: good synthetic data. What they built The model is called CLaC, short for Contrastive Language-Crystal model, and it is described in a paper in Nature Communications by Yang Jeong Park, Mayank Kumaran, Chia-Wei Hsu, Elsa Olivetti and Ju Li. MIT News covered it on 18 October 2026. If you know CLIP, the model that taught image generators to connect pictures and words, you already get the idea. CLaC has two branches: a graph neural network that reads the 3D crystal structure, and a text encoder that reads a description. Contrastive training pulls matching structure-text pairs together in one shared space and pushes mismatche...
Meer lezen

A 35B model on a Mac mini, using just 2.9 GiB of memory 🚀

There's something special about running a capable language model on hardware you own: no API bill, no data leaving the house, just a small box on your desk doing the thinking. A paper that's currently #2 on Hugging Face's trending list takes that idea further than I expected: it runs a 35-billion-parameter model on a Mac mini at about 20 tokens per second while using under 3 GiB of memory. The trick: leave the experts on the SSD The paper is called The Other Half of the Memory Wall: Serving 35B MoEs from SSD with Trained Routing Prediction , and the system is named Edge0 . It builds on a Mixture-of-Experts (MoE) model, Qwen3.6-35B-A3B. In an MoE, each token only uses a few "experts" per layer (here 4 out of 256 in each of 40 layers), so most of the weights sit idle at any given moment. Edge0 takes that literally. The expert weights (quantized to int4) stay on the SSD and are streamed in only when needed. The clever part is a small prerouter : a tiny trained he...
Meer lezen