Quality fixes for Gemma 4 E4B
Quality fixes for Gemma 4 E4B
FYI
If you use Gemma 4 E4B with llama.cpp, do yourself a favour and grab Google’s latest official chat template:
https://huggingface.co/google/gemma-4-E4B-it
https://huggingface.co/google/gemma-4-E4B-it/blob/main/chat_template.jinja
Use it with --jinja --chat-template-file chat_template.jinja.
The improvement in tool use is remarkable but it also seems to improve conversation continuity:
“Here is XYZ. Now add ABC. Now DEF. Actually, what were Y, B and C?”
That makes a big difference is multi step reasoning IME
Sadly, still no fixes for the other llama.cpp quirks.
- llama.cpp still reserves separate compute buffers for the target and MTP contexts, despite never using them concurrently.
That wastes about 1 GB of VRAM / causes OOM
PR #27489 fixes it, but remains not merged.
https://github.com/ggml-org/llama.cpp/pull/27489
- Audio also still fails through the llama.cpp WebUI.
The model itself can "hear". just fine - a direct curl request confirms that.
(“In this sound clip, the man says XYZ.”).
See #PR21825
https://github.com/ggml-org/llama.cpp/issues/21825
Dunno how these are resolved in other back ends / front ends, so just an FYI