Ling-3.1-flash (560B-A25B)
Hybrid-linear MoE (63 KDA + 9 gated MLA layers), 560B total / 25B active, 1M context, MIT license. Native hybrid reasoning aimed at agents, coding and office work. The official weights are FP8 (~574 GB).
Recommended quants
- Official FP8 (~574 GB)
inclusionAI/Ling-3.1-flashOfficial recipe: 8x H200 (TP8 + EP8) or 4x GB300 on SGLang. - AtomicChat AD-IQ1_S (141.8 GB) / AD-IQ2_XXS (174.1 GB) / AD-IQ2_M (218.1 GB)
AtomicChat/Ling-3.1-flash-GGUFImatrix GGUFs with published KL divergence (IQ2_M: top-1 agreement 92.7% vs BF16). Needs llama.cpp b11571 or newer per the uploader. - AtomicChat AD-IQ4_XS (337.8 GB) / AD-Q4_K_M (354.4 GB)
AtomicChat/Ling-3.1-flash-GGUFQ4_K_M top-1 agreement 96.9% vs BF16 per uploader. - bartowski GGUFs (IQ1_S 131 GB to Q8_0 608 GB)
bartowski/Ling-3.1-flash-GGUF
Chat template Works
Thinking controlled by chat_template_kwargs enable_thinking (model card recommends true). Reasoning in <think> blocks. Custom architecture (bailing_hybrid), so transformers and vLLM need trust_remote_code.
Fix: On vLLM use --tool-call-parser ling3 --reasoning-parser ling3 and --mamba-cache-mode align (from the official command), or tool calls and thinking won't parse.
Sampling settings
| Official | temp 1.0, top_p 0.95, top_k 20, enable_thinking true |
|---|
Runtimes
| vLLM | Caveats | Official command uses a vllm branch built from source (git clone -b vllm), not a release wheel. TP8, expert parallel, MTP speculative decoding. |
|---|---|---|
| SGLang | Works | Recommended. Day-0 image lmsysorg/sglang:dev-ling-3.1-flash with MTP (NEXTN) speculative decoding. |
| llama.cpp | Works | Community GGUFs; AtomicChat says b11571 or newer. |
| Ollama | Unverified | |
| LM Studio | Unverified | |
| MLX | Unverified |
Tool calling in agent tools
| Claude Code (local endpoint) | Unverified | Model card says it works well in Claude Code, Kilo Code, Qwen Code and Hermes Agent; not tested here. |
|---|---|---|
| OpenCode | Unverified | |
| Cline | Unverified | |
| Aider | Unverified |
Known issues
Sources checked
- https://huggingface.co/inclusionAI/Ling-3.1-flash
- https://huggingface.co/AtomicChat/Ling-3.1-flash-GGUF
- https://huggingface.co/bartowski/Ling-3.1-flash-GGUF
Spotted something wrong or a new fix? Corrections are reviewed before publishing. (Submission form coming soon.)