21 points neo2006 7 hours ago 6 comments
platinumrad 7 hours ago | parent
This will be very useful to me as someone who still uses LLMs interactively rather than trusting the harness to one-shot the whole thing.
rvz 6 hours ago | parent
Please continue. [0] Let's see what happens when it goes even faster than 2,400 TPS and why not host an abliterated version of this model as well.
What could possibly go wrong?
xlayn 2 hours ago | parent
you know who was state of the art abliterated models who already have hack a lot of goverment services in a lot of places worldwide? openai, anthropic, google, facebook...
Thanks god you are not a wacko suggesting Qwen4B-abliterated will hacK someone
rvz 45 minutes ago | parent
Yes. If we’re not prepared for frontier models breaching companies at 300 TPS, we are certainly not prepared to defend against frontier abliterated cyber models running at 2,400+ TPS.
It takes the attackers to successfully breach their target once vs the defenders having to stop all attempts.
jahala 2 hours ago | parent
I'll just go get my 8 x B300 (TP8) from the shed and start this up real quick
xlayn 2 hours ago | parent
I have been doing some poking at llama.cpp for amd gpus and at some point I ask claude if it was already 10x faster than GB200 and claude told me to not be funny, one of those things were more expensive than my house and nvidia sold them in a 72pack unit that needs it's own hvac....