Grug (caveman compression) of Qwen3.6-35 and 27B is good!
Grug (caveman compression) of Qwen3.6-35 and 27B is good!
Hi all, if you've not heard, there's been a Caveman compression version of Qwen 3.6-27b and Qwen 3.6-35b-3ab.
https://huggingface.co/ProCreations/grug-27b-gguf
https://huggingface.co/ProCreations/grug-35b-v2
https://huggingface.co/ProCreations/grug-35b-v2-gguf
I've been playing around with 35B and I am able to run it on my ancient Quadro P1000 4gb at ~10tok/s.
But beyond that, the quality of the reasoning and the token discipline / output is actually higher than a stock, in my opinion.
You can read the benchmarks above; I am also uploading a HTML file here for your consideration.
(Sorry for the Limewire link; I dunno where else to share throw-away files. It's HTML, though the side by side probably works better if you actually download it)
Anyway...I'm doing more testing right now...but so far, this is a good cook.
Or -
Grug good. Me like.