colibri streams a model's experts off the disk instead of loading them, so a 2.8-trillion-parameter model answers on a machine that could never hold it.
coli chat attached to a running
server, then the same model in brio mode. The numbers are the ones it produced.Measured today on machines people actually have
Pull a converted container where one is published, or point the engine at the original checkpoint.
The model colibri was built around. 78 layers, 256 routed experts, streamed from disk on a machine that could never hold them.
hf download mastouri/GLM-5.2-colibri-int4-g64-with-int8-mtp --local-dir ~/Models/glm52_i4Natively multimodal. Routed experts convert to int4-gs64, the dense set stays BF16 and its precision is a load-time choice.
hf download Justvugg/GLM-5.3-colibri-int4-g64 --local-dir ~/Models/glm53_i4Runs off the official checkpoint untouched: experts are already fp4, the dense set fp8-e4m3. Routed experts cost 4.5 GB per token against GLM-5.2's 12.7.
hf download deepseek-ai/DeepSeek-V4.1-Flash --local-dir ~/Models/DeepSeek-V4.1-FlashNative fp4 experts, fp8-e4m3 dense. The REAP-pruned 150B cut, 132 of 256 experts and 85 GB, loads with the same engine and no conversion.
hf download deepseek-ai/DeepSeek-V4-Flash-0731 --local-dir ~/Models/DeepSeek-V4-FlashThe largest model colibri runs. 93 layers, 69 Kimi Delta Attention plus 24 gated MLA, and 16 of 896 experts per token. QAT-trained MXFP4 streams straight from the original shards.
hf download moonshotai/Kimi-K3 --local-dir ~/Models/Kimi-K3Sliding-window attention with short convolutions. Audio input is supported when the audio tensors are present; the dense set fits a 25 GB host at 15.3 GB.
hf download nbeerbower/Inkling-colibri-int4 --local-dir ~/Models/inkling_i448 layers, 512 experts, 10 routed per token. 125B of backbone plus a 51B n-gram embedding that stays pageable and a 4B MTP head; the experts stay native block-FP8. CPU only for now.
hf download Qwen/Qwen3.8-Flash-Next-FP8 --local-dir ~/Models/Qwen3.8-Flash-Next-FP8The one to start with: 20 GB on disk, gated attention interleaved with gated DeltaNet. With CUDA=1 the VRAM expert tier measured 1.44 to 10.05 tok/s on two 8 GB cards, output bit-identical to the CPU path.
hf download Kreuzzelg/qwen36-35b-a3b-colibri-i4-gs64 --local-dir ~/Models/qwen36_i4_gs64Fully open weights and data, 16 layers and 64 experts. The one to learn the tooling on: the whole container is 7 GB.
python3 c/tools/convert_olmoe_merged.py --repo allenai/OLMoE-1B-7B-0125-Instruct --out ~/Models/olmoe_i8Nothing matches that.
A mixture-of-experts token touches a small fraction of the weights. colibri keeps that fraction resident and reads the rest as the router asks for it, so the model does not have to fit.
GLM-5.2 int4, decode speed reported by the people who ran it. The hardware changes where the experts live, not what the model answers.
See the benchmarks →Nine families run today, one hand-written C file each. Not a wrapper around someone else's runtime.
Total parameters. Only the active ones are computed per token, and only the routed experts move. The engines, by the names the docs use: GLM-5.3-Flash, GLM-5.2/5.3, Inkling, Kimi K3, OLMoE, Qwen3.6-35B-A3B, Qwen3.8-Flash-Next, DeepSeek V4 Flash, DeepSeek V4.1 Flash.
Browse all models →Most of what people ask a model for is a choice, not an essay. Hand it the options it may pick and it reports the probability of each one. Nothing is generated, so there is nothing to parse and no answer outside the list.
Measured on qwen36 (22 GB), same engine, separate processes. Time from the prompt to the answer.
Anything that ends in a bucket: a ticket to a queue, a pull request to a verdict, a message to a policy.
Braces, quotes and field names are data. The engine fills one slot at a time, so the JSON is valid by construction.
Read the contract once, then ask it twenty things. The first question pays for the reading, the rest cost their own words.
{
"model": "qwen36",
"state": "340 lines, 8 files, no tests.",
"question": "What should the reviewer do?",
"options": ["merge", "request changes", "close"]
}{
"answer": "request changes",
"entropy": 0.121,
"choices": [
{"option": "request changes", "p": 0.974},
{"option": "merge", "p": 0.023},
{"option": "close", "p": 0.004}
],
"usage": {"completion_tokens": 0}
}Same request, same model, three fields of one
JSON schema. On area it is sure and it is right. On the other two the entropy is
near the maximum: it does not know, and it says so. The chat arm wrote
"risk": "high"
with exactly the same face.
That number is what you put a threshold on to send a case to a human. Brio does not write prose, so anything open-ended is still the chat's job, and the speed multipliers above were measured on an engine reading experts from disk.
Read the brio guide →The engine is one C file per family with no runtime and no dependencies. It runs on a laptop with the network off, and the model files never leave the machine. Apache 2.0, and every container keeps the licence its authors gave it.
| what you keep | |
|---|---|
| The weights | on your disk, in a format you can read |
| The prompts | never sent anywhere |
| The engine | one file you can read in an afternoon |
| The numbers | token-exact against the reference implementation |