YHK - yet another hacker news clone

Meta's GB{2,3}00 Cluster

38 points stokedmartin 1 day ago 16 comments

markhahn 1 hour ago | parent

shell glob rather than RE? what's the world coming to!

alberth 1 hour ago | parent

Actual title: ”Inside Prometheus: Meta’s Journey to a Gigawatt-Scale Supercluster”

sgvwthry 45 minutes ago | parent

Posting this two days before earnings seems to indicate that there is going to be some bad capex news.

zer00eyz 26 minutes ago | parent

Ya dont say.

Negative Free Cash Flow are the words every one on HN needs to learn.

Every thing before now, you will look back upon as the happy time. When every year storage, ram and compute grew and the prices went down. Your cloud provider made so much money they paid cash for hardware.

Now they borrow at 8% and net margins on ram/storage were around $0.10/GB ... today they sit at over a dollar. (SEE micron's s10's if you want to check the math on that).

Forget circular financing, its a circular firing squad because your going to have to figure out where the hell your going to run any of your gear.

4x to 10x for ram +8% for financing +30percent AWS margins. Good luck everyone its gonna get rough out there.

jacquesm 14 minutes ago | parent

I am very lucky to have stocked up on RAM before this happened (256G) but I wished I had doubled or quadrupled that, an upgrade simply isn't going to happen now.

le-mark 6 minutes ago | parent

Or hyperscalers revenue will continue to disappoint, there is no $60 trillion market for ai and there will be a glut of capacity for years to come!

xeubie 42 minutes ago | parent

It seems that Meta didn't mention one aspect of their "next-generation" AI training system: their incompetent web crawler hammers websites while completely ignoring robots.txt. Meta's 57.141.0.0/24 made up about 90% of all requests to my server this past week, and it didn't stop after I blocked it. They continue to make a request every 2 seconds despite me responding with a 403 every time. Truly an engineering marvel that Meta has bestowed upon us.

ctoth 32 minutes ago | parent

nft add table inet filter

nft add set inet filter meta_crawl '{ type ipv4_addr; flags interval; }'

nft add element inet filter meta_crawl '{ 57.141.0.0/24 }'

nft add chain inet filter input '{ type filter hook input priority 0; }'

nft add rule inet filter input ip saddr @meta_crawl tcp dport {80,443} reject with tcp reset

xeubie 25 minutes ago | parent

I did finally block them at the firewall level last night. They are constantly trying to connect anyway.

figmert 30 minutes ago | parent

> incompetent

I think that's not incompetence, I think that's by design.

solyin 13 minutes ago | parent

How? What's the incentive structure for that design? There are no extra ads to be displayed here, so how does Meta get value from that design? Seems way more likely to be incompetence in my world view. Explain why that design is superior!

jacquesm 16 minutes ago | parent

I've blocked Meta, OpenAI and Anthropic and yet they seem to find ways to get back in. It's pretty annoying.

ma2kx 36 minutes ago | parent

Imagine your job is just replacing millions of Dollars worth of hardware every day. I mean there must be a reason the hold 8 GPUs per rack spare.

solyin 10 minutes ago | parent

Yes the reason is they're pushing the GPUs hard so that they fail, and having hot spares saves the job until helping hands can go replace the bad component.

lmc 36 minutes ago | parent

Is this one of the pilot models they claim R&D tax credits for?

https://news.ycombinator.com/item?id=49921118