Recipe by sudoingX @sudoingX · based on Qwen, Unsloth, ggml-org (llama.cpp), 40 community contributors

Original recipe · GitHub · X

License: Apache-2.0 · Qwen: Apache-2.0 · ggml-org (llama.cpp): MIT.

TESTED BY sudoingX on RTX 3090 24 GB and RTX 5090 laptop 24 GB, Q4_K_M, n-max 2 · their results (benchmark table)

Qwen3.8-27B faster on a 24 GB card: one llama.cpp flag

ADVANCED: llama.cpp. 2 commands.

FORNVIDIA card, 24 GB. The author tested an RTX 3090 and an RTX 5090 laptop.

NEEDllama.cpp, newest build. github.com/ggml-org/llama.cpp/releases

NEEDPython, for the download tool.

WHYAbout 33% faster replies on his 3090. Same file, one flag.

GET

pip install -U "huggingface_hub[cli]"
hf download unsloth/Qwen3.8-27B-GGUF Qwen3.8-27B-UD-Q4_K_M.gguf --local-dir .

RUN

llama-server -m Qwen3.8-27B-UD-Q4_K_M.gguf -c 131072 -ngl 999 -fa 1 --cache-type-k q4_0 --cache-type-v q4_0 --spec-type draft-mtp --spec-draft-n-max 2 --parallel 1

OPENhttp://127.0.0.1:8080 in your browser. Chat there.

TUNETry --spec-draft-n-max 3 or 4. Keep the fastest.

HIS SETTINGS, LINE BY LINE

QUANTQ4_K_M (Unsloth UD-Q4_K_M file)

FILEthe llama-server command above

-c 131072              # 128K context
-ngl 999               # all on GPU
-fa 1                  # flash attention
--cache-type-k q4_0    # fits long context
--cache-type-v q4_0    # fits long context
--spec-type draft-mtp  # built-in draft
--spec-draft-n-max 2   # 24GB sweet spot
--parallel 1           # one user

This is the README full launch command, split up so you can see why each flag is there. The README names the file Qwen3.8-27B-Q4_K_M.gguf; on Unsloth it is Qwen3.8-27B-UD-Q4_K_M.gguf, so we use that name. From README 1, README 2.

SAFETYIt answers only on this PC (127.0.0.1). Do not add --host 0.0.0.0 unless the network is yours.

OR · LET A CODING AGENT DO IT

PASTEInto Claude Code, Codex or Cursor:

Set up the Qwen3.8-27B MTP recipe by sudoingX on this computer: https://github.com/sudoingX/qwen38-mtp
Follow its README.
1. Check first and tell me: NVIDIA card with 24 GB of VRAM, and the newest llama.cpp build (llama-server --version). Stop if the card is smaller.
2. Download unsloth/Qwen3.8-27B-GGUF, file Qwen3.8-27B-UD-Q4_K_M.gguf, into this folder.
3. Start it with the README full launch command, unchanged.
4. Run the README probe.py once without the three spec flags and once with them, both at --parallel 1. Show me both numbers.
5. Do not add --host 0.0.0.0. Ask me before deleting anything.

If it fails

Context fails to start past about 90K tokens.
Keep both --cache-type-k q4_0 and --cache-type-v q4_0 in the command.
Slower than without the flag.
Update llama.cpp, then try --spec-draft-n-max 3 or 4. The best value depends on the card.
Speed halves for no clear reason.
Close the browser and other apps that use graphics memory. A busy desktop can push part of the model off the card.
Many chats at once are not faster.
Expected. The gain is for one user (--parallel 1).

From the original README, shortened. Read it for full details.

What they tested

RTX 3090 24 GB: 31.0 tok/s without the flag, 41.3 with it (author row) · evidence
RTX 5090 laptop 24 GB: 36.7 tok/s without, 50.9 with (author row) · evidence
probe.py: the script behind every number, same settings both runs · evidence

Not shown in their results: Speed only. The table does not measure answer quality. Most rows are other people’s runs. We list only the author’s as TESTED BY.

RunCards does not test recipes. We only list recipes whose authors publish their own test results. Checked 2026-10-19.

Who made what

Qwen: the Qwen3.8-27B model and its built-in MTP draft layers
Unsloth: the GGUF files the command loads
ggml-org (llama.cpp): the engine; draft-mtp support came in PR #22673 (also)
40 community contributors: the benchmark rows and the seven tuning rules, each credited by name in the README

The README names Qwen, Unsloth and the llama.cpp PR, and credits every table row and rule to its contributor. A fork exists at github.com/jaisusx/qwen38-mtp; sudoingX is the original.