DeepSeek-V4.1-Flash-1017
Minor update to DeepSeek-V4.1-Flash (same architecture, 763B total params incl. Engram tables, 1M context, MIT license) with better reasoning, coding, agentic and vision scores and shorter outputs. Official weights are FP8 + MXFP4 experts (~763 GB repo). Day-one local options are early: a llama.cpp PR branch, MLX oQ quants and an NVFP4 checkpoint for vLLM on Blackwell.
Recommended quants
- Official FP8 / MXFP4 weights (~763 GB repo)
deepseek-ai/DeepSeek-V4.1-Flash-1017Server GPUs. The bundled inference/ folder is a readable reference implementation (TP8 via torchrun), not a production server. - NVFP4 (399.5 GiB)
LibertAIDAI/DeepSeek-V4.1-Flash-1017-NVFP4Same layout as nvidia/DeepSeek-V4.1-Flash-NVFP4; routed experts transcoded bit-exactly, Engram tables lossy MXFP4 (cosine 0.9934). Blackwell, vLLM TP4. - mlx-community oQ4e-mtp (432 GB / 402.4 GiB)
mlx-community/DeepSeek-V4.1-Flash-1017-oQ4e-mtpoMLX mixed precision, keeps native FP4/FP8 projections and quantizes only the Engram tables. Smaller oQ3e, oQ2.5e and oQ2e variants are also up. - mrfakename GGUF Q2_K (246 GiB) / Q3_K_M (323 GiB) / Q4_K_M (414 GiB) / Q8_0 (473 GiB)
mrfakename/DeepSeek-V4.1-Flash-1017-GGUFPlain quants, no imatrix, text only. Needs the llama.cpp branch from ggml-org/llama.cpp#28696; mainline doesn't support V4.1 yet. - unsloth GGUF
unsloth/DeepSeek-V4.1-Flash-1017-GGUFRepo is a placeholder as of Oct 17 (WIP, no files yet).
Chat template Caveats
Like V4.1-Flash, no Jinja chat template ships. DeepSeek provides a Python reference encoder (encoding/encoding.py with tests for thinking, numeric reasoning effort, tools and images) and the deepseek-recipe Rust/Python libraries.
Fix: Use a runtime that implements the DeepSeek V4.1 encoding (vLLM uses --tokenizer-mode deepseek_v41 in NVIDIA's command). Generic GGUF chat templates may be wrong.
Sampling settings
| Official | temp 1.0, top_p 1.0, max_tokens 256K or more, 1M context window |
|---|
Runtimes
| vLLM | Works | NVIDIA's documented NVFP4 command (TP4, --tokenizer-mode deepseek_v41) applies to the LibertAIDAI NVFP4 repo. Tested hardware is GB300. |
|---|---|---|
| SGLang | Caveats | The MXFP4 Engram tables in the NVFP4 repo aren't supported in SGLang yet, per the uploader. |
| llama.cpp | Caveats | Not in mainline. Use the ggml-org/llama.cpp#28696 branch; keep Engram tables on CPU with -ot "engram_embd.weight=CPU". Text only. |
| Ollama | Unverified | |
| LM Studio | Unverified | |
| MLX | Works | mlx-community oQ quants made with oMLX v0.7.1, including the MTP head. |
Tool calling in agent tools
| Claude Code (local endpoint) | Unverified | DeepSeek used the Claude Code harness for its Terminal-Bench 4.0 score. |
|---|---|---|
| OpenCode | Unverified | |
| Cline | Unverified | |
| Aider | Unverified |
Known issues
Sources checked
- https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017
- https://huggingface.co/LibertAIDAI/DeepSeek-V4.1-Flash-1017-NVFP4
- https://huggingface.co/mlx-community/DeepSeek-V4.1-Flash-1017-oQ4e-mtp
- https://huggingface.co/mrfakename/DeepSeek-V4.1-Flash-1017-GGUF
- https://huggingface.co/unsloth/DeepSeek-V4.1-Flash-1017-GGUF
Spotted something wrong or a new fix? Corrections are reviewed before publishing. (Submission form coming soon.)