Robots that feel with their fingertips: Panasonic and partners show T-Rex at CoRL 🦖

Most robot hands I read about still "see" more than they feel. A camera decides where to go, and contact is treated as a side effect. That works for picking a box. It falls apart when you turn a key, slide one card out of a sleeve, or screw in a light bulb without crushing anything. On 5 November, Panasonic Holdings and partners from UC Berkeley, NVIDIA, Stanford, Sapienza University of Rome and ItalAI announced T-Rex (Tactile-Reactive Dexterous Manipulation): a Vision-Language-Action model that keeps correcting its motion from fingertip touch while it works. The results land at CoRL in Austin from 9 to 12 November.

What T-Rex actually does

T-Rex splits control into two speeds. A slower stream uses vision and language to plan the overall motion at about 5 Hz. A faster tactile stream reads contact at about 20 Hz and fine-tunes the action when something slips, presses too hard, or deforms. You do not have to re-run a heavy image pass every time the fingertip changes. That frequency gap is exactly where many contact-rich tasks fail today.

The training recipe has three stages. First, broad visuomotor priors from 22,889 hours of human egocentric video. Then mid-training on a new 100-hour bimanual robot dataset with synchronized tactile signals: more than 200 everyday objects, 22 basic actions, over 7,700 episodes, and force plus deformation from 10 fingertips. Finally, roughly 100 demonstrations per downstream task. That mid-training dataset is meant to be open-sourced under MIT, which matters a lot more to builders than another closed demo reel.

Why the numbers are interesting

On 12 real contact-rich tasks (page turning, transferring an egg, applying toothpaste, unlocking a lock, extracting a card, screwing in a bulb, and more), T-Rex reached an average success rate of 65%. The strongest baseline in their comparison sat around 35%. Without tactile input, the same setup dropped to about 42%. So touch is not a nice-to-have add-on here; it is carrying a large share of the gain.

I also like that the paper is honest about failure modes: collisions from rushed alignment, small objects slipping, too much toothpaste squeezed out, multi-finger friction opening the wrong lid. That is the useful part of a robotics paper for me. Progress is real, and the remaining hard bits are named.

Why this feels like a step forward

For education tools, homelab experiments and future assistive robots, contact is the boring bottleneck. Vision-only agents look clever until they meet a soft object or a tight slot. A policy that plans slowly with eyes and language, then reacts quickly with touch, is a cleaner engineering story than "just scale the VLA". Panasonic frames the near-term use cases as factory insertion and handling soft or fragile goods in logistics. Fair. The research stack behind it (dataset + MoT architecture + real benchmark) is what I will keep an eye on.

If you care about dexterous hands, read the paper and the project page before the CoRL talks. Feeling the world at 20 Hz sounds small. On a key and a light bulb, it is the difference between almost and actually.

Sources

Would you rather give a robot better eyes, or better fingertips first?

Reacties

Een reactie posten

Populaire posts van deze blog

Meta just shared the full recipe for a 30B agent model you can run at home 🧠

When Meta released Muse Glimmer in August, the pitch was simple: an open-weight, Apache 2.0 model built for AI agents that run on your own Mac or PC. What was missing was the "how". On 19 October, Meta Superintelligence Labs published the full technical report, and for anyone who cares about distillation and local agents, it is one of the most useful reads in a while. What the model is Muse Glimmer is a dense model of about 29.6 billion parameters, including a 1.8B perception encoder, so it reads text and images and writes text. It has a 128K+ context window and was trained on data from more than a hundred languages. The design goal was strict: everything has to fit in 32 GB of GPU memory, and a smaller 4-bit variant targets 24 GB cards. The parts I found most interesting Dense beat mixture-of-experts here. That surprised me. The team trained scaling ladders for a dense model and two MoE designs with the same memory footprint. Under a fixed memory ceiling and their compu...
Meer lezen

AI that can actually "see" a crystal 🔬

Most of the solid stuff around us, from battery materials to chips, is made of crystals: atoms stacked in neat, repeating 3D patterns. Large language models are great at reading papers about those materials, but they have never really looked at the structures themselves. A team at MIT just showed a clever way around that, and the trick is one I keep coming back to: good synthetic data. What they built The model is called CLaC, short for Contrastive Language-Crystal model, and it is described in a paper in Nature Communications by Yang Jeong Park, Mayank Kumaran, Chia-Wei Hsu, Elsa Olivetti and Ju Li. MIT News covered it on 18 October 2026. If you know CLIP, the model that taught image generators to connect pictures and words, you already get the idea. CLaC has two branches: a graph neural network that reads the 3D crystal structure, and a text encoder that reads a description. Contrastive training pulls matching structure-text pairs together in one shared space and pushes mismatche...
Meer lezen

A 35B model on a Mac mini, using just 2.9 GiB of memory 🚀

There's something special about running a capable language model on hardware you own: no API bill, no data leaving the house, just a small box on your desk doing the thinking. A paper that's currently #2 on Hugging Face's trending list takes that idea further than I expected: it runs a 35-billion-parameter model on a Mac mini at about 20 tokens per second while using under 3 GiB of memory. The trick: leave the experts on the SSD The paper is called The Other Half of the Memory Wall: Serving 35B MoEs from SSD with Trained Routing Prediction , and the system is named Edge0 . It builds on a Mixture-of-Experts (MoE) model, Qwen3.6-35B-A3B. In an MoE, each token only uses a few "experts" per layer (here 4 out of 256 in each of 40 layers), so most of the weights sit idle at any given moment. Edge0 takes that literally. The expert weights (quantized to int4) stay on the SSD and are streamed in only when needed. The clever part is a small prerouter : a tiny trained he...
Meer lezen