NVIDIA GeForce RTX 24 GB
3090
2.2×Ampere sm_86 faster
Animation: Radeon RX 7900 XTX, Qwen3.8-27B UD-Q4_K_XL. The same 500-token answer written nonstop at the measured pace: the typical app at 35.9 tokens per second, our engine at 114.8 tokens per second. Our engine finishes it in 4.4 s, the typical app in 13.9 s. The same measurement script on both sides.
Same hardware. Same model. Just faster.
Radeon RX 7900 XTX · Qwen3.8-27B UD-Q4_K_XL · the same measurement script on both sides (ABBA) · a 500-token answer, example text, measured pace
Three steps. Each one measured.
01Scan
First, we read your card.
Locally, without admin rights, without sending anything. This is what we pick the model from.
Scan of the card in our lab, Sapphire PULSE Radeon RX 7900 XTX: Navi 31 chip (gfx1100), ROCm 10.0 (HIP) driver, 24 GB of GDDR6 on a 384-bit bus, 960 GB/s bandwidth, default power limit 303 W. Recommended model: Qwen3.8-27B UD-Q4_K_XL.
02Pick
A model that really fits.
We count memory with context and buffers, not just the file size. A model that is too big is out straight away.
fits our engine uses 22.8 GB / 24 GB
does not fit the same model in Q8_0 · 29.1 GB > 24 GB
Qwen3.8-27B UD-Q4_K_XL 17.6 GB · model file
Radeon RX 7900 XTX: 24 GB VRAM. Qwen3.8-27B UD-Q4_K_XL: model file 17.6 GB, our engine uses 22.8 GB — fits. the same model in Q8_0: 29.1 GB — does not fit.
03Accelerate
Three setups. Our engine is fastest.
Same model, same card, same measuring script. Every bar is a separate measurement.
- Typical app llama.cpp with default settings : 42.9 tokens per second
- Best flags + MTP upstream llama.cpp, MTP speculation : 75.3 tokens per second, 1.8× vs the typical app
- Our engine lighter draft and our own kernels : 93.3 tokens per second, 2.2× vs the typical app
From 35.9 to 114.8 t/s. 3.2× faster, same card.
Qwen3.8-27B · Radeon RX 7900 XTX · chat · tokens per second · every step measured with the same script
- Typical app upstream llama.cpp, default settings : 35.9 t/s
- Best flags + MTP upstream llama.cpp with its own speculation : 60.0 t/s
- Our engine, Oct 12 first version
- Our own kernels for RDNA3
- Our engine, Oct 15 production release
- Fewer launches and fusions
- A faster sampler and host
- Smarter MTP speculation
- Our engine today Oct 19
- Stable PM4 command replay
- Verification on the GPU
Same model,same quality.
Our engine changes the model’s answers as much as llama.cpp’s own noise. Changing the model format makes a difference about 310× bigger.
Underdog Saluki 27B: our engine vs llama.cpp 0.00065, llama.cpp vs itself 0.00061; Ternary Bonsai 2 27B: our engine vs llama.cpp 0.00032, llama.cpp vs itself 0.00021. a model format change (QAD-Q4_0): 0.203.
How we measure it
Linear scale. Metric: KLD, how much the next-word probabilities differ (0 = identical). llama.cpp noise: the same llama.cpp with a different batch size (ubatch 256 vs 512). Saluki and Bonsai measured on the GeForce RTX 3060 12 GB; the format change is another model (LFM2.5 on the GeForce RTX 5060 Ti 16 GB), shown for scale.
Speed-ups don’t change the text
On or off, they give the same greedy text, bit for bit (md5 checks). We differ from llama.cpp only in the order of some multiplications, at noise level; hence KLD.
- speculation
- GPU verification
- PM4 replay
Measured, card by card.
Every card has its own page: the models that fit, the results and the measurement conditions.
NVIDIA GeForce RTX 16 GB
5060 Ti
1.8×Blackwell sm_120 faster
NVIDIA GeForce RTX 12 GB
3060
1.6×Ampere sm_86 faster
NVIDIA GeForce RTX 16 GB
4060 Ti
1.9×Ada sm_89 faster
AMD Radeon RX 24 GB
7900 XTX
3.2×RDNA3 gfx1100 faster
On the tiles: the card’s recommended model or, when it has no same-script comparison, the first model that does. Our engine vs the typical app; conditions are on the card pages.
Open the cards explorerThe scan runs on your computer. Your chats never leave it.
The scan reads only your hardware spec and sends nothing by itself. The result is plain text: you can read every field.
See what the scan readsYour card can do more.
See how much we can get out of it. Without changing the model.
GeForce RTX 3090, Qwen3.8-27B UD-Q4_K_XL: the same answer streams at 42.9 tokens per second without Enigma LABZ and at 93.3 tokens per second with Enigma LABZ.
Before you ask.
Is this a different model from the one I know?
No. We use the same model files (GGUF) as other apps and speed up the engine that runs them. Our output is not bit-identical to upstream, so we check quality with KLD; it stays within upstream’s own noise.
Does it work on NVIDIA cards?
Yes. We have measurements on NVIDIA and AMD cards, each with its conditions and source. The cards with results are in the cards explorer, and more are waiting in the measurement queue.
How much faster will it be on my computer?
It depends on the card and the model. On the GeForce RTX 3090 with Qwen3.8-27B we measured 2.2×; on weaker cards with small models the gain can be small. For cards we have not measured we show an estimate from our formula, always labelled as an estimate.
What do you send to your servers?
Nothing by default. The scan and your chats run locally. Sharing a result with the profile database is a separate opt-in, off by default, with a data preview.
Why tokens and not words?
Engines count tokens. In English a word is about 1.17 tokens, in Polish about 2.69, so next to answer times we also give words.
Does it work on Windows?
We design for Windows, Linux and macOS, but today we measure on Linux. Results on Windows may differ; we will show them once we have measured them.