Measured on this card

AMD Radeon RX 7900 XTX

24 GB · RDNA3 gfx1100 · 960 GB/s

Qwen3.8-27B UD-Q4_K_XL: token generation

Typical app Our engine t/s faster 3.2×

Model: Qwen3.8-27B UD-Q4_K_XL
File: 17.6 GB
Memory used: 22.8 GB / 24 GB
A 500-word answer: 16.3 s → 5.1 s

Memory: 24GB
Bandwidth: 960GB/s
Power limit: 303W
Architecture: RDNA3 gfx1100

What fits

24 GB of memory. Each block is a model loaded by our engine, with its context and KV cache.

Qwen3.8-27B UD-Q4_K_XL

22.8 GB

file 17.6 GB used 22.8 GB

LFM2.5-24B Q5_K_M

17.0 GB

file 17.0 GB

Models for this card

Every measured model that fits in the card’s memory. The best one first.

Qwen3.8-27B

UD-Q4_K_XL 17.6 GB Licence: Apache 2.0: commercial use allowed

Our pick for this card
Writing speed

3.2× faster

Same card, same model, same script. Only the software changes.

Honestly: the details

Code edits (the same script, T=0, 4 files): typical app 35.5 t/s, best flags 89.0 t/s, our engine 419.7 t/s. Without speculation (llama-bench tg128): 35.3 → 43.4 t/s.

Typical app 35.9 t/s

Best flags 60.0 t/s

Our engine 114.8 t/s

Typical app: llama.cpp with default settings, like a typical app (on the RTX 3090 the same as Ollama) · Best flags: upstream llama.cpp with the best flags we know (e.g. MTP speculation) · Our engine: our engine with the same flags plus our patches · spec_sweep through the server, the same script on both sides: chat, 6 prompts × 3 seeds, T=1, fresh server, median; ABBA order (the second pass within ±0.2 t/s)

A 500-word answer
derived from t/s × tokens per word × 2.69 tok/słowo (raport 06 §2.1.2) × 1.17 tok/słowo (raport 06 §2.1.2)

in Polish

in English

Replay at real speed

The same answer, written out at the measured speed.

Typical app: 35.9 t/s, Our engine: 114.8 t/s. spec_sweep through the server, the same script on both sides: chat, 6 prompts × 3 seeds, T=1, fresh server, median; ABBA order (the second pass within ±0.2 t/s)

Typical app UD-Q4_K_XL

35.9 t/s

First I collect the sensor data and check which readings are reliable. Then I compare them with yesterday’s run, flag the outliers and suggest three fixes. I describe each one briefly, with its risk and cost. Finally I write a summary for the team and a list of questions worth answering before the rollout. If something does not add up, I go back to the raw data, check the calibration and compare the results with last week. I note what I changed and why, so anyone can repeat the steps. Only then do I send the report, with a chart and a short recommendation for what to do tomorrow morning. Each figure links to its source, and every assumption is written down next to it, so nothing has to be taken on trust.

156 / 156 tokens done ·4.3 s
Our engine UD-Q4_K_XL

114.8 t/s

First I collect the sensor data and check which readings are reliable. Then I compare them with yesterday’s run, flag the outliers and suggest three fixes. I describe each one briefly, with its risk and cost. Finally I write a summary for the team and a list of questions worth answering before the rollout. If something does not add up, I go back to the raw data, check the calibration and compare the results with last week. I note what I changed and why, so anyone can repeat the steps. Only then do I send the report, with a chart and a short recommendation for what to do tomorrow morning. Each figure links to its source, and every assumption is written down next to it, so nothing has to be taken on trust.

156 / 156 tokens done ·1.4 s 3.2×
Conditions

spec_sweep through the server, the same script on both sides: chat, 6 prompts × 3 seeds, T=1, fresh server, median; ABBA order (the second pass within ±0.2 t/s)
- ROCm
- the same server and script on both sides
- MTP speculation (best flags and our engine)
- 200k context (n_ctx 204800)
spec_sweep through the server, the same script on both sides: chat, 6 prompts × 3 seeds, T=1, fresh server, median; ABBA order (the second pass within ±0.2 t/s)
About the model

A dense 27B model with hybrid attention (Gated DeltaNet + full attention) and a built-in MTP head that guesses several tokens ahead.

Our engine was built on this card: Qwen with MTP speculation writes 114.8 t/s, 3.2× faster than the typical app.
- chat
- code
- quality
app/landing/src/lib/data/models.jsonapp/landing/src/lib/data/models.json (raport 06 §2.4.2)app/landing/src/lib/data/models.json (raport 04 §2.3.1)
All measurements
Instruments

Typical app → Our engine
Step time: with MTP a step takes longer but accepts several tokens at once · Prefill 4096: reading the prompt, tokens per second

Step time

27.9 27.7 ms

≈ no change
Prefill 4096

1,056 1,126 t/s

↗ better
Same model, same quality: measured

Honestly

Saluki 27B

KLD ours vs upstream 0.00065

upstream noise 0.00061
Bonsai 2 27B

KLD ours vs upstream 0.00032

upstream noise 0.00021

In plain mode our engine does not produce bit-identical text to upstream (patches 0001/0002, plus 0013 for Qwen). So we measure quality with KLD: on the RTX 3060 the difference stays within upstream’s own noise.
LFM2.5-24B-A2B

Q5_K_M 17.0 GB Licence: LFM Open License v1.0: commercial use only below USD 10M annual revenue
Writing speed

1.7× faster

Same card, same model, same script. Only the software changes.

Typical app 169.3 t/s

Our engine 289.7 t/s

Typical app: llama.cpp with default settings, like a typical app (on the RTX 3090 the same as Ollama) · Our engine: our engine with the same flags plus our patches · llama-bench tg128 (128 generated tokens), the same script on both sides. Through the server our engine does 293.7 t/s.

A 500-word answer
derived from t/s × tokens per word × 2.69 tok/słowo (raport 06 §2.1.2) × 1.17 tok/słowo (raport 06 §2.1.2)

in Polish

in English

Replay at real speed

The same answer, written out at the measured speed.

Typical app: 169.3 t/s, Our engine: 289.7 t/s. llama-bench tg128 (128 generated tokens), the same script on both sides. Through the server our engine does 293.7 t/s.

Typical app Q5_K_M

169.3 t/s

First I collect the sensor data and check which readings are reliable. Then I compare them with yesterday’s run, flag the outliers and suggest three fixes. I describe each one briefly, with its risk and cost. Finally I write a summary for the team and a list of questions worth answering before the rollout. If something does not add up, I go back to the raw data, check the calibration and compare the results with last week. I note what I changed and why, so anyone can repeat the steps. Only then do I send the report, with a chart and a short recommendation for what to do tomorrow morning. Each figure links to its source, and every assumption is written down next to it, so nothing has to be taken on trust.

156 / 156 tokens done ·0.9 s
Our engine Q5_K_M

289.7 t/s

First I collect the sensor data and check which readings are reliable. Then I compare them with yesterday’s run, flag the outliers and suggest three fixes. I describe each one briefly, with its risk and cost. Finally I write a summary for the team and a list of questions worth answering before the rollout. If something does not add up, I go back to the raw data, check the calibration and compare the results with last week. I note what I changed and why, so anyone can repeat the steps. Only then do I send the report, with a chart and a short recommendation for what to do tomorrow morning. Each figure links to its source, and every assumption is written down next to it, so nothing has to be taken on trust.

156 / 156 tokens done ·0.5 s 1.7×
Conditions

llama-bench tg128 (128 generated tokens), the same script on both sides. Through the server our engine does 293.7 t/s.
- ROCm
- no speculation
- KV q4_0 (upstream) and q8_0 (ours, 2.6% slower)
llama-bench tg128 (128 generated tokens), the same script on both sides. Through the server our engine does 293.7 t/s. ;
About the model

A mixture of experts (MoE): 24B parameters, about 2B active per token. That is why it writes several times faster than a dense model.
- speed
- chat
app/landing/src/lib/data/models.jsonapp/landing/src/lib/data/models.json (raport 06 §2.4.2)app/landing/src/lib/data/models.json (raport 04 §2.3.1)
All measurements

Same model, same quality: measured

Honestly

Saluki 27B

KLD ours vs upstream 0.00065

upstream noise 0.00061
Bonsai 2 27B

KLD ours vs upstream 0.00032

upstream noise 0.00021

In plain mode our engine does not produce bit-identical text to upstream (patches 0001/0002, plus 0013 for Qwen). So we measure quality with KLD: on the RTX 3060 the difference stays within upstream’s own noise.

What next

Compare with another card Check your hardware

AMD Radeon RX 7900 XTX

Writing speed

A 500-word answer

Replay at real speed

Conditions

About the model

Instruments

Honestly