The Little Beginning
How small can a language model get before it stops making sense?
LittleBit
In early 2025, we tried to find out. We made a model much smaller. It still worked. We made it smaller again. Still working.
This was getting interesting.
That was the beginning of LittleBit.
The rough recipe: approximate a large weight matrix with a combination of smaller matrices (SVD), represent those matrices with very few bits (low-bit quantization), and let the original model teach the compressed one how to behave (QAKD). Three technical names for a simple question: how much space does intelligence actually need?
We pushed down to an effective 0.1 bits per original weight. Compared with 16-bit weights, that is a 160x reduction in the bits used to represent the compressed weights. That ratio applies to the compressed weight representation, rather than the entire model’s memory footprint. Even so, 0.1 bits is a strange place to find a working language model.
We published LittleBit in 2025 and presented it at NeurIPS that December. People kept asking us the same question we were asking ourselves:
“Why does this work?”
Fair question. We wanted a better answer, too.
In July 2026, we presented LittleBit-2 at ICML. It offered a piece of the answer: how you arrange the information matters, as well as how much you keep. Aligning that structure more carefully gave us better performance. A little more understanding. A better model. Still a lot left to figure out.
On-device models
Then came the next question. If this works for LLMs, what else does it work for?
We started with speech recognition. We built a remarkably small model that could still recognize speech, and presented the work at Interspeech 2026. But one of the biggest surprises came from speculative decoding.
The idea is simple. A small model drafts the next few tokens. A larger model checks them together. The small model is a fast first-draft writer. It needs to be quick, and its guesses need to be useful.
We made that writer roughly 100x smaller. Its acceptance length, the number of draft tokens accepted in a round, still held at about 60–80% of the original draft model’s level. A hundredth of the size, and still writing useful drafts. Think 1 GB becoming roughly 10 MB.
The technology reached commercial deployment as a key on-device acceleration model for next-generation Galaxy smartphones. In our internal comparisons, our draft was more than 2x faster than Google’s Gemma 4 E2B MTP draft. Against Liquid’s DSpark draft, it was at most a quarter of the size and about twice as fast.
Tiny models were doing useful work on real devices. We call it our little bit of magic.
Into robotics
Then we decided to give it a body.
A robot is a demanding place to put intelligence. Cameras and sensors keep sending data. The body needs frequent updates to stay balanced and move. Memory is limited. Chips get hot. Batteries run down. And the physical world does not pause while the model finishes thinking.
Intelligence has to fit inside all of that.
This was the question LittleBit had been preparing us to ask. Maybe robotics was where this technology belonged all along.
We want robots that see, think, and act using the computers inside their own bodies. Robots that do useful work around us, within a practical budget for hardware, power, and memory.
So we put a very small model into a real robot.
And we saw our first bit of magic. It picked up a piece of fruit and put it in a basket.
Here is how that went.
References
- Banseok Lee, Dongkyu Kim, Youngcheon You, and Youngmin Kim. LittleBit: Ultra Low-Bit Quantization via Latent Factorization. NeurIPS, 2025.
- Banseok Lee and Youngmin Kim. LittleBit-2: Maximizing the Spectral Energy Gain in Sub-1-Bit LLMs via Latent Geometry Alignment. ICML, 2026.
- Jinsu Yeo, Banseok Lee, and Youngmin Kim. Pushing the Limits of Compression: Sub-1-Bit Conformer via Variable-Rank Binary Decomposition. Interspeech, 2026, pp. 1739–1743.
- Google. Speed-up Gemma 4 with Multi-Token Prediction. Google AI for Developers.
- Liquid AI. LFM2.5-DSpark: Up to 3.2x Faster Inference from H100 to MacBook. 2026.
Citation
Download .bib@article{lee2026littlebeginning,
author = {Banseok Lee},
title = {The Little Beginning},
journal = {Superfast Robotics Writings},
year = {2026},
note = {superfastrobotics.com/writings/essay-001},
}