Mistral Small 4 (119B, 2603)
119B MoE (128 experts, 4 active), 256K context, text+image input, Apache-2.0. reasoning_effort none/high.
Recommended quants
- Unsloth UD-Q4_K_XL (74.2 GB) / MXFP4_MOE (71.8 GB)
unsloth/Mistral-Small-4-119B-2603-GGUFGood fit for 96-128 GB unified memory. - UD-Q3_K_XL (54.8 GB) / UD-Q2_K_XL (40.2 GB)
unsloth/Mistral-Small-4-119B-2603-GGUF64 GB Macs / 2x24 GB. - UD-Q6_K_XL (108.3 GB)
unsloth/Mistral-Small-4-119B-2603-GGUF
Chat template Works
reasoning_effort="high" for complex tasks, "none" for direct answers.
Fix: Match temperature to effort (below).
Sampling settings
| reasoning_effort high (official) | temp 0.7 |
|---|---|
| reasoning_effort none (official) | temp 0.0-0.7 depending on task |
Runtimes
| vLLM | Works | Official docker image route. |
|---|---|---|
| llama.cpp | Works | Mistral points to Unsloth GGUFs. |
| Ollama | Unverified | No library page found. |
| LM Studio | Unverified | |
| MLX | Unverified |
Tool calling in agent tools
| Claude Code (local endpoint) | Unverified | |
|---|---|---|
| OpenCode | Unverified | |
| Cline | Unverified | |
| Aider | Unverified |
Known issues
Sources checked
- https://huggingface.co/mistralai/Mistral-Small-4-119B-2603
- https://huggingface.co/unsloth/Mistral-Small-4-119B-2603-GGUF
Spotted something wrong or a new fix? Corrections are reviewed before publishing. (Submission form coming soon.)