10 points frag 1 hour ago 10 comments
smashspectacle 1 hour ago | parent
Ed Zitron is going to become the Simpsons of ai critiques...
jdietrich 1 hour ago | parent
Siri, what's the spot price for an A100?
orwin 57 minutes ago | parent
I don't want to argue here, but once models are trained, using them is quite cheap. Also in the last few months most improvements I've seen are in the smaller sized models, which might make using models even cheaper.
nisegami 55 minutes ago | parent
There's this weird tension where small models are getting a lot better, but the gap between the frontier and "what I can run on my gaming GPU" at home isn't necessarily getting smaller. That said, Qwen-3.8-27B-Q3 runs on my 9700 XT and performs well enough for tool calling in agents, which is a critical floor in local model performance imo.
TonyStr 19 minutes ago | parent
Do you use qwen for any coding? Do you make a frontier model orchestrate it? Are you running with llama.cpp? Sorry for the barrage of questions, but I've been interesting in local AI for a while, yet I've struggled to set up anything usable.
esseph 14 minutes ago | parent
Just go download and install lemonade server. Open web ui for front end. There are mobile apps to connect to it.
Driver4732 38 minutes ago | parent
If the models are cheap to run, while good for customers, isn't that a problem for hyperscalers and all the investment poured into the AI data center build out?
deadlast2 35 minutes ago | parent
If you use opencode for your AI work or becomes affordable. To the point where you would reasonably build applications with intelligence the LLM version. I think in the near future the intelligence will be free and there is no business in free. I know it is a big statement but if you look around there is already a large amount of free models of gradually increasing strength. For most applications you don't need a gpt6
mono442 22 minutes ago | parent
Another thing to pay attention to is the margins of hardware companies like nvidia or micron. Hyperscalers are spending huge amount of money but the money mostly goes to the profits of hardware companies.