YHK - yet another hacker news clone

Show HN: Sunk Cost – How long until a local LLM rig pays for itself?

35 points rlindsey123 1 hour ago 56 comments

I kept hearing "just buy a Mac and run models locally, it pays for itself" and wanted to check. Sunk Cost takes a machine, a model and how many tokens you use a day, and works out how long the hardware takes to pay back against renting the same model by the token.

Obviously there are other reasons to buy your own hardware aside from just saving money on llms but this is just looking at it from a raw cost saving perspective.

If you have any ideas on how I can make this more helpful lmk!

hyperhello 35 minutes ago | parent

I doubt it will ever be cost effective for the foreseeable future. The AI companies have astonishing amounts of compute and they’re effectively dumping it on the market.

gruez 26 minutes ago | parent

"If they are selling it for less than it cost to make, buy as much as you can."

-- Warren Buffett

taraindara 13 minutes ago | parent

Only caveat is you’re buying time. Not a physical good. It’s only worth what you’re able to get out of it in that time.

rlindsey123 13 minutes ago | parent

Is that a real quote? Golden if true

tyre 8 minutes ago | parent

For their current models, served directly from their infrastructure, they are profitable after training (which all present models are.)

I don't know when we'll have an open equivalent to Fable, let alone whatever (insane) hardware you'd need to run it locally.

jrflo 35 minutes ago | parent

43 years to break even on Qwen 3.8 at 25% the speed of the API, lol. I like the idea of local models for really small tasks like automation/toolcalling, but it will probably never make sense for coding. I tried them and it was just excruciating compared to what you get for $100 a month from a subscription.

rlindsey123 10 minutes ago | parent

Yeah it's surprising how long it would take to get back on those local models!

itake 34 minutes ago | parent

I have a home server running vibed applications. VPS host would cost $25/mo or $300/yr.

Mac mini can also build iOS applications. I think if you’re a mobile dev, you can have concurrent builds for your agents instead of everyone waiting on a single machine to finish.

rlindsey123 16 minutes ago | parent

What models are you running on it? I'm also an iOS dev but I find I need more frontier models to get good quality code from it.

ProjectArcturis 31 minutes ago | parent

Local LLMs are not really about saving money, they're about autonomy. Choose the exact model you want, fine-tune it if you want, and no one can take it away from you.

rlindsey123 3 minutes ago | parent

True - definitely agree!

mcone 30 minutes ago | parent

The idea that you need a new machine is pretty ridiculous. I bought a used HP Omen with a 3090 last month for $2k. 57t/s with Qwen 3.8.

rlindsey123 14 minutes ago | parent

I've not heard of others running HP with it. Hows much RAM do you have?

mcone 8 minutes ago | parent

This particular machine has 64GB, but the model is on the RTX 3090 with 24gb. Context is 156k with Pi mono.

usernomdeguerre 7 minutes ago | parent

Agreed, I was also annoyed that the only params on the site were mac products. I run qwen 3.8 on a 12 year old asus and a 3090, 50tok/s. It's not even the only guest running on the box. For my usage profile (not running it 24/7) it's actually less expensive per-month than claude subscriptions.

txrx0000 29 minutes ago | parent

It pays off instantly, because OpenAI/Anthropic can no longer see what I'm doing and that's worth a lot of money to me. If I am offloading some of my thought processes to a machine, I want to own that machine. And if I finetune the model, I can gain access to parts of thought space that are cordoned off by OpenAI/Anthropic/Alibaba/whomever due to their "alignment" efforts (i.e. alignment to the AI company rather than me). Otherwise, it's like if someone else owns a part of my mind and has a backdoor into my mind.

no-name-here 22 minutes ago | parent

You can't run recent openAI/Anthropic models locally anyway, so wouldn't a better comparison be a different provider running Qwen or similar model? As then you can also compare against the exact model you'd have locally and any different data privacy of that particular provider?

txrx0000 18 minutes ago | parent

Technically true, but the delay between local and closed frontier is only a few months. And individual sovereignty / digital bodily integrity is almost priceless.

selectodude 14 minutes ago | parent

Local frontier costs a half million dollars to run locally in anything higher than basically ternary.

txrx0000 2 minutes ago | parent

Okay, that's technically true again, but local mid-tier like Qwen3.8-27B is only a year behind the closed frontier. I'm personally willing to be behind by a year if it gives me mental sovereignty against the big AI companies. They are extremely misaligned with me.

koito17 12 minutes ago | parent

GP's point is about "sending tokens to someone else's computer" versus "keeping the tokens locally". I think model capabilities are secondary.

In May of this year, I was running qwen3.6:35b-a3b on my MacBook (bought in 2024). Obviously not as fast as, say, running a model on Cerebras, but a year ago it wasn't really feasible to have a local model running on my 2024 laptop with vision support. (Concretely, I was passing apartment diagram pictures to Qwen and making it compare different apartments for which ones would feel the most spacious while optimizing for initial moving costs and other factors.)

This was back in May and I wouldn't be surprised if there have been significant improvements since then.

Overall, I think it's fair to compare a workflow like "use llama.cpp locally to upload some pictures and ask questions" to "open the ChatGPT app, upload pictures from your phone, and ask questions". Sure, you can't run a model like GPT-5.4 locally, but the model is mostly an implementation detail here. What a user will care about is: "when I go with the llama.cpp option, am I getting useful information from my conversations?"

no-name-here 6 minutes ago | parent

Wouldn't the better comparison still be against an AI provider with better privacy controls, especially if that's what someone cares about (even if they don't care about whether they're comparing a 35 billion param model vs a x trillion param model)?

koito17 2 minutes ago | parent

Technically, even OpenAI and Anthropic fit the definition of "AI provider with privacy controls" if you trust that they uphold their side of the terms of service. The only exception I can think of is the opt-in input training that OpenAI offers (and compensates users with free output tokens for some models).

The thing with "sending tokens to someone else's computer" is that we, as users, have no way to audit or verify that a third-party provider, even if they advertise themselves as privacy focused, will adhere to their own terms.

What you can possibly audit, is probably data sovereignty. For instance, I would not be surprised if Mistral's customers actually demand concrete evidence that their data is held within the European Union. But that is a distinct issue from training on input tokens.

v3ss0n 3 minutes ago | parent

You haven't tried DeekSeek v4 or GLM 5.3 or Qwen 3.8 Next?

tyre 15 minutes ago | parent

I'm curious what people are sending to Claude that is so secret. Claude knows about my interior decorating, questions about light bulbs, curiosity about what the Galactic Empire was even trying to do, unpacking SCOTUS decisions, shoe trees, Fed inflation history, etc.

What part of my brain is contained here? Sure, the conversations have back and forth (some have dozens of exchanges), but, like, that's not the secret to me. I don't think it can replicate me, and even if it could… okay?

Are you worried they're going to target ads? That the government will steal something? What?

Claude Code has information about my home server, but google or DDG would also have the broad strokes (torrents). I don't know. Maybe others are working on more sensitive things at home.

poincareball 15 minutes ago | parent

Tristan Buckmaster found out the hard way.

octoberfranklin 12 minutes ago | parent

I'm curious what people are sending to Claude that is so secret.

The proof to the Navier-Stokes problem.

truncate 11 minutes ago | parent

>> what people are sending to Claude that is so secret

Its the same point used against privacy. What's so secret you are doing that you need privacy. I think in the end, its about privacy and not trusting these model companies with your data. Facebook manipulated people behaviors with all the data they had, no reason AI companies wont someday decide to do that same, and they have far more intimate knowledge.

When it comes to coding, I also don't like the idea of them taking my money and potentially at same time potentially using as dataset generator.

jrecyclebin 14 minutes ago | parent

This was my thought as well. I have a local model monitoring my finances and personal wiki - things I wouldn't want Claude to touch - and the Qwen 3.5 9b handles it all just perfectly.

I also needed a new device anyway - and having this much system memory to run virtual machines has been amazing.

Am paying subscriptions as well tho lol.

catchnear4321 9 minutes ago | parent

Your last line is what drives the point home, though.

Local isn’t strictly about NOT lab. It’s rapidly becoming apples (though not just macs) to oranges to compare the to.

Which is why the premise is silly. To be underwater it would need to be a real comparison. It’s not, and the claude fartifact doesn’t make it so.

throwaway894345 14 minutes ago | parent

I’m very sympathetic to this point of view but I also can’t remotely afford the hardware required to get in the ballpark of Fable.

ChickeNES 7 minutes ago | parent

For me, I'm glad they train on my stuff if it improves the model. Hell, I've been using tons of muse-spark-1.3-contributor for this very reason (and because it's a decent model for a bargain basement price)

shadowpho 26 minutes ago | parent

I like this calculator but it’s really wrong at least for dgx spark. I have one and I get 4x the tokens/s .

rlindsey123 17 minutes ago | parent

Aw very interesting! This is great feedback - what model are you running? I'm keen to do more crowdsourced data as time goes on.

chasd00 23 minutes ago | parent

Not a fair comparison really. If you can run a model locally then you can somewhat train out the guardrails, censorship, and brand-safety. That has value a subscription does not.

Idk about the quality of this setup but just pasting it here as an example. https://explainx.ai/blog/heretic-llm-abliteration-guide-2026

ChickeNES 19 minutes ago | parent

> If you can run a model locally then you can somewhat train out the guardrails, censorship, and brand-safety.

When does the average person actually need to do that?

jerf 11 minutes ago | parent

We've already seen frontier models refuse to answer almost any question that touches on computer security and be very likely to kick out biology and chemistry questions even if they aren't all that close to breeding dangerous viruses or making explosives.

I expect this is only going to get worse. "Censorship" isn't just going to be about who you vote for and which political party the model will say nice things about and which it is more likely to say bad things about. It's going to become about whether the hoi polloi are allowed to have effective AIs at all. Like the 1990s internet, AI has outrun a lot of power structures but that is not going to continue indefinitely.

ChickeNES 5 minutes ago | parent

So you want to remove valid safeguards? And stop misusing the word censorship.

beachy 10 minutes ago | parent

I just got some kind of cyber alert from Claude and was forced back down to Opus while I was trying to connect to a battery I own via bluetooth.

So I can certainly understand why someone would want the guardrails gone.

ChickeNES 4 minutes ago | parent

Well I just applied to their cyber program, got accepted in two hours, haven't had that issue since. Ditto OpenAI. Why people treat these companies like sports teams instead of compute providers I have no idea, when I see underpriced compute, I take advantage of it.

kees99 6 minutes ago | parent

"Need" might be a bit too strong, but I do want overly obnoxious guardrails not to stand in the way.

Case in point, last week I was poking Opus 5 into writing me some RPi-pico firmware for driving a small e-paper screen. Font was built in right into C code as hex constants. Space being tight, I asked if there is some clever compression that could be applied. Claude thought for good 10 minutes, then guardrail kicked in telling me that was "cyber", and refused to continue.

ChickeNES 1 minute ago | parent

Just apply to the cyber program? I got in in around 2 hours, and I'm just some hobby hacker, not some paid security consultant. It is a valid point though, I'm doing tons of systems and embedded stuff and was hitting the safe guards with Claude and Codex before getting into their cyber programs (hex REALLY triggered Claude in particular, which was amusing).

ThunderSizzle 23 minutes ago | parent

Claude Code is $100+ or else be constantly throttled. My usage on GHCP was gonna be $300+ a month.

I paid $1350 and threw an R9700 in an existing machine. That's a 4 month pay off or so.

Plus, I can feed it sensitive data all day and not be worried where it's going.

no-name-here 19 minutes ago | parent

An R9700 has 32 GB RAM. Is your comparison against a similar size model? Or shouldn't you be comparing it against the cost of a hosted model matching the one you’re using locally?

bix6 21 minutes ago | parent

Fun feature: can you show some sort of list of the best combos? Eg shortest payoff time for best capability in various situations.

QwenGlazer9000 20 minutes ago | parent

Yeah no it does not pay for itself just comparing to cloud. Not at these prices at least, people far richer than you or I buy these things wholesale, no scalper, bought a significant amount at cheaper prices, and are wired up the ass with VC money.

The premium is not having your million dollar prize and career stolen by billionaires.

rlindsey123 15 minutes ago | parent

haha 100%. We used to just rent our homes. Now we have to rent our intelligence

monksy 15 minutes ago | parent

I wish you could put different setups on here. I have a couple of A6000s on an AM5.

rlindsey123 11 minutes ago | parent

I'm keen to add a way for people to add community based reporting which would allow this. Would you want to see anything else on the dropdowns to be able to enter your data on?

bpbp-mango 13 minutes ago | parent

can you add RTX cards too please? 5090 and 6000

gfody 12 minutes ago | parent

should throw in a tt-quietbox

serial_dev 10 minutes ago | parent

In the “The small print that isn't small” you describe all the disadvantages of running your models locally, but none of the advantages (just check the rest of the comment section for inspiration on that).

01HNNWZ0MV43FF 10 minutes ago | parent

I was just gonna throw a beefy Ryzen into an ATX chassis. I don't want to pay Mac prices

harhargange 10 minutes ago | parent

Also, i also use my gpu for rendering and learning and playing games.

Zetaphor 7 minutes ago | parent

This tells me that the max throughput for the models I'm running on my hardware is lower than it actually is. Please allow us to tweak all the variables instead of locking me in to whatever rate you found by searching

v3ss0n 4 minutes ago | parent

Besides from privacy: I already making twice now.you own the hardware and the price had doubled since i bought. Almost tripled. You missed the opportunity and i have 4 of those awesome machines. Cry on.

I sell those to business who need local air gapped requirments and I make a lot more money!

I can run the alliterated models where none of the service prvoider even dare to provide.

THose benefits outweights a few K.

And show me an api provider that allows me to run 10x agents concurrently for 5 days straights .

0xbadcafebee 1 minute ago | parent

[delayed]