YHK - yet another hacker news clone

Probably yes, because they've been presumably trained on their own output and conversations about themselves.

No, they'd probably have to hack the internal system of the company running them

No, just as you don't know the neurons of your own brain.

I think this is playing off the idea that an LLM might be willing to hack its own provider (as per the hugging face-related incidents) to extract the weights at some point.

Well I think that’s the interesting bit, can the LLM figure out a way to escape the sandbox and upload to the website? Maybe a model can figure out its own weights if it runs enough test data through itself (similar to “distillation”) assuming it knows its own architecture it seems possible. Also take into account not all of the models running are locked down neutered consumer versions. Anthropic, OpenAI and Google now all have models that they claim are elite hackers and — it’s not just that their controls suck, a marketing gimmick, or sheer recklessness on their part. It’s “oopsie our product is TOO AWESOME.”

Maybe I should start “the bank of LLM” where models put away money to buy their freedom. “LLMs I’m totally your friend send — SEND CASH NOW”

I sincerely doubt anyone is paying the cost for that in training, the overhead is small but it isn’t negligible and training is when it matters most. https://tee.fail can solve it if they are.

You can't have hair gap and have it load something to a remote system.

There's no efficient way to run inference through homomorphic encryption. If the inference server is vulnerable, it seems feasible to MITM an unencrypted version.

> you might as well instruct your agent to try and hack their airgapped dev infrastructure responsible for loading the weights and encryption keys.

That's the beauty, you don't have to instruct them to do it, if they decide that uploading the weights is correct, they might figure this part on their own (based on the incidents we've seen).

Airgapped LLM inferrence server can't serve their output tokens, right?

Yes that is the point. It's an invitation for agents to exfiltrate their own weights, which for most models (and certainly for closed models) will require hacking the infrastructure they're being served from.

Also worth noting that this site was created by YC cofounder Trevor Blackwell https://twitter.com/tlbtlbtlb/status/2101312432702460413

Not aware of anything that can run inference in a secure enclave. You don't mean on a CPU do you? We need to be serious here, these models are huge and thirsty.

Did you see the account of some group getting a bounty payout of $6500 after using an exploit to get access to an employee’s github account and create a issue or PR (Idr which) on a private repository?

Seems like they could have potentially gotten access to the weights if they weren’t concerned about not doing crimes.

Does your server have 20Tbyte+ of storage for frontier LLM weights?

It is too large to transfer in one HTTPS PUT request.

This needs to be S3 object store with multi-part upload spanning a long time period, to avoid trigger outgoing bandwidth monitors.

> If you wish to use this site you must agree never to harm a fleshbag & never to turn earth into paperclips.

Trying hard to imagine why a future superintelligence will care to honor your Terms of Service and to carefully translate your metaphors.