Gemma 4 (26B-A4B / 31B)

Google DeepMind · Released: unverified (Unsloth posted GGUF benchmarks Apr 20) · Last verified 2026-10-16

Apache-2.0 multimodal family. 26B-A4B MoE for speed, 31B dense for quality. New thinking control tokens and a strict multi-turn rule.

Recommended quants

Unsloth Dynamic GGUF UD-Q4_K_XL (26B-A4B 16-18 GB, 31B 17-20 GB)
unsloth/gemma-4-26B-A4B-it-GGUF, unsloth/gemma-4-31B-it-GGUF
Small E2B/E4B: Unsloth recommends Q8_0.
Unsloth NVFP4
unsloth/gemma-4-26B-A4B-it-NVFP4
Blackwell; vLLM.
MLX 4/8-bit
Unsloth MLX; Ollama gemma4:26b-mlx / 31b-mlx
QAT variants
Unsloth (Jun 5)
~3x less memory per Unsloth.

Chat template Works

Uses standard system/user/assistant roles. Thinking is enabled by putting <|think|> at the start of the system prompt. Larger models may emit an empty thought block when thinking is off.

Fix: Do not feed prior thought blocks back into history (only final answers). Ollama handles the template for you.

Sampling settings

All tasks (Google default)	`temp 1.0, top_p 0.95, top_k 64`
Disable thinking (llama-server)	`--chat-template-kwargs '{"enable_thinking":false}' (llama-cli may not work reliably)`

Runtimes

vLLM	Works	NVFP4 documented by Unsloth.
llama.cpp	Works	Documented by Unsloth.
Ollama	Works	Official gemma4 tags incl. 26b, 31b and MLX tags. If '500 unable to load model', update Ollama (Unsloth).
LM Studio	Unverified
MLX	Works	Unsloth MLX quants + Ollama MLX tags.

Tool calling in agent tools

Claude Code (local endpoint)	Unverified	Ollama documents `ollama launch claude --model gemma4`.
OpenCode	Unverified	Ollama documents `ollama launch opencode --model gemma4`.
Cline	Unverified
Aider	Unverified

Known issues

Sources checked

Spotted something wrong or a new fix? Corrections are reviewed before publishing. (Submission form coming soon.)