Yet another Hacker News

Gemma-4-26B-A4B with 39-69% less thinking at same accuracy

58 points armcat 4 hours ago 6 comments

PaulKeeble 47 minutes ago | parent

I have tried the Thinking cap and the Swift model for QWEN-3.8 27B and my experience was its not as smart and capable, the results on the same task were less impressive. For the amount of tokens saved it might very well be worth it but I wanted to run the xhigh precisely because I wanted the peak result. I haven't tested this but I suspect it ends up much the same trade off, in theory it does well in the benchmarks but in practice on your own tasks its not as capable.

dofm 33 minutes ago | parent

Yeah. I didn't find these any good either.

However, Gemma 4 26B is a reasonably good general chat inquiry model (it's also not too bad at commonplace code questions, if not as good for agentic code).

So this could be a good, fast model for that, in some situations; I may test it.

I still broadly just prefer working with Muse Glimmer, to be honest; we get along OK. I am trying out the Nemotron Bolt 74B/A9B MoE and I really do think it could be quite good, but it's still slow on Metal.

solarkraft 11 minutes ago | parent

> I still broadly just prefer working with Muse Glimmer

Welcome to our very exclusive club of now two people! Ever since it came out I’ve been trying to find a better model, but just can’t.

dofm 5 minutes ago | parent

It is hard to explain why it is better. Benchmarks etc., context length, it should be worse than Qwen 3.8 27B and I am sure for long context work it is.

But on my old M1 Max, with "Reasoning strength: low" in the system prompt, it's just kind of nice to use. It's not really slow (despite being a dense model it is faster to actual solutions than the Qwen 3.6 35B MoE), it writes well, etc.

So I use it as a developer-friend and researcher at the moment. It's the first local LLM I've considered building hardware for.

Foobar8568 12 minutes ago | parent

But Qwen 3.8 27b is unbearable slow, you get 10minutes of "let me think" over any trivial stuff.

simonw 8 minutes ago | parent

Qwen 3.8 27B defaults to "xhigh" (its maximum level) for thinking, which makes it diabolically slow. Setting it to "medium" or "low" speeds it up a lot.