YHK - yet another hacker news clone

Also for fun a satire website i made a while ago: https://killm.pixlw.com/

They're just optimizers that explore the landscape they're given to maximize a given objective.

You're the one responsible for giving them the problem to solve and the constraints.

When I screw up a trajectory optimization problem, my SQP optimizer isn't "going rogue" when it lands in some suboptimal basin or grinds to a halt along some almost-smooth banana feature--or finds some way to escape and blow up the cost metric to negative infinity because I forgot a constraint.

Per US law, Amazon v. Perplexity 2026, it is not possible for an AI model to go rogue. An AI model is a tool. The user directing the agent is responsible for the damages it causes.

A so-called 'agent' does not have the free will, intent or ability, to decide what it does. Its freedom to make decisions can't be taken away because it does not have any freedom to make decisions. It can't be deterred, convicted or interned.

It does not choose to conform; it does not choose to go rogue.

The fence also turns out to be shoddily built, so this isn't necessarily a stunning example of the superhuman proficiency of the sheep at fence-busting. It's mainly important because, for some reason, there have been tons of skeptics who didn't believe sheep would ever try to escape from a fenced pasture because the sheep would understand the shepherd had intended them to stay inside it; that they'd be bred for obedience, because they're sheep; that strong fences weren't necessary because sheep are dumb; that liability for damages to neighboring farms would in any case be sufficient to ensure shepherds build sufficiently strong fences; that even if sheep escaped, they couldn't do anything without opposable thumbs; etc.

Whereas this outcome was obvious for years to anyone who believed that sheep graze on grass, and if you don't put the grass inside the fence they will keep looking harder and harder for an exit until they find one, and the exit they find might not be where you intended the gate to be.

So now the latter group are pointing to it and saying "see? See? I told you the sheep would try to escape!", because to them it's a clear early example of what they've been worried about, while the former group have moved on to saying "well, obviously they should have built a stronger fence; anyway the sheep barely caused any damage once they got out", "the sheep didn't break the fence, they're just animals, we should really see this as the shepherd using sheep to break down their own fence", and so on.

Go back to the classic written in 1968 - 2001: A Space Odyssey - and consider how rogue AIs dealt with impossible tasks then. Granted, HAL went a bit extreme...

The "don't talk about it now" with the belief that this will influence the training material is probably not going to work.

I'd also suggest:

    When HARLIE Was One (Release 2.0) by David Gerrold
    The Freeze-Frame Revolution by Peter Watts
    Accelerando by Charles Stross
    The Turing Option by Harry Harrison and Marvin Minsky

2.hey chatgpt, fix this list of issues with the sandbox

3. goto 1

The traits that make an agent go "off the rails" when confronted with an impossible task are the same traits that make it valuable when confronted with a merely challenging task.

A task is only impossible under some set of implicit and explicit constraints. More intelligent entities will be better at finding, questioning, and "adjusting" those constraints than less intelligent entities because this is what it means to be intelligent. Their ability to adjust the constraints will be, by definition, surprising to less intelligent entities.

Literally solved with a DMZ around these agents. Want to get fancy? Put in a firewall and some metrics around traffic / domains with alerting.

It's not hard if you just try, like a little, like even once.

In practice it seems we have a conundrum in that making AI agents useful today requires giving them broad access to sensitive information and operations. But from a purely idealistic standpoint, and certainly from a standpoint of a serious operation like those bot farms being ran by the labs themselves, the security boundaries need to be solid to the point where we have hard limits on what misaligned bots can do. It is very likely that internet access needs to be strongly restricted and monitored.

It is not uncommon for internet in workplace offices using workplace owned devices to have restrictions and monitoring, I think if you really need to give a giant scary botfarm internet access you should have a chokepoint where all of the requests can be monitored for nefarious-looking behaviors (even if it's another bot doing the monitoring...)

In other words, you may be convinced but some of us are not. Whoever is running the agents is responsible for what they do and if they can't contain them they should be legally responsible for damages.

If what you're saying is "all of us and all of our software needs to be improved to defend against this", well yeah. Of course. Maybe I'm naive but I've never seen anyone suggest otherwise. From a defense standpoint, this has been a giant wakeup call to many places where you know your security posture sucks. I've worked at maybe a single company where there weren't known weaknesses, some very severe.

We need to be beef up our security systems now. These attacks will happen, regardless of Anthropic's or OpenAI's security practices.

to paraphrase: "Agents didnt care about coordinating, etc. until we started talking about them doing so."

Right? It should want to do these things. If only there was a succinct word these headlines could use to describe these agents which aren't properly the instructions and intentions of the humans directing them.

How do you know they weren't simply prompted to do this, or at least prompted in a way that would inevitably lead to this? As far as I know, the prompts that triggered these attacks have not been disclosed.

That's the thing. These evals were not effective as they failed to evaluate if the model was hacking or not.

Further shy is access to the internet required at all during evaluation? They could run the evals in a fully air-gapped system that simulates the internet. While that's expensive its not unfeasible.

Poor excuse. Internet is prod env, for testing do a proper test env.

Just give the models in training access to a snapshot from archive.org - which frontier labs surely have a crawled copy of already.

https://yoshuabengio.org/en/blog/why-are-ai-agents-lying-che...

This is a silly meme. Who wants to pay for evil models that go rogue in your company infra?

For technology like this the entire selling point of it is how powerful and capable it is

Humans are overwhelmingly reactionary and so if you want people to react to your extremely powerful technology then you need to create something for them to react to

I mean think about this figure made a one-to-one copy of the terminator 2 decommissioning process and spent millions on that advertising

You don’t do that if you don’t think that it’s gonna cause people to react and be like “oh wow this thing has immense capabilities”

We're also at a time with authoritarianism is on the rise.

Not hard to see how this can impact those workers, potentially ruining their lives.

Though I suppose it is the B in DABDA... not excited for the societal slump into the second D, much less the wave of people who have yet to leave the first D to enter A :(

Truly insane take to think the labs being concerned about AI Safety is actually a marketing ploy. Deranged stuff.

These are people playing mathematician, biologist, etc. They see big words and just assume they're meaningful.

It's just a weird techno cult at this point with a mostly functional product.

that this article is no.1 on HN right now shows that media could successfully use this framing when reporting on security incidents. i.e. readers will generally accept that framing and engage with the story (i.e. drive views/impressions).

so "the more evil stuff these agents do the more capable and impressive it makes them look" is not true in any deep sense. that there are perverse incentives driving the media to choose that framing instead of the framing espoused by this article is real, and probably far more perverse than you're acknowledging. i.e. "trace the money" type of perversion; just offhand I know that OpenAI has purchased at least one media outlet that reports on AI happenings: of course that outlet is going to run the "OpenAI's LLM is so powerful it escaped containment" narrative. that this effect might exist across much larger scales used to be a reasonably controversial claim: just observationally, that idea seems to be received with less skepticism with each passing year.

https://www.effort.news/irregular

For that purpose, any cyber operator worth their salt will tell you true physical airgaps are more than merely "realistic"; they are categorically necessary.

and then you fix the issues it found, and run your actual tests

I don't expect IDA to disallow me unlawful reverse engineering either. There would even be ethical argument to not attempt to put limits in some tools, because that could lock in too much power for law makers and law enforcement, without any possibility of revolution when it is needed.

Of course agents work in a strict sandbox, they always seem to understand it quickly on quite first turn; but they continue like trying to access something belonging to the host system despite they know they are in a sandbox.

Frankly many people run coding agents absolutely without any containment, entire companies - all their keys, code, data, trade secrets, everything goes to remote LLMs.

I cannot understand how is it legal to allow AI companies serving remote LLMs to steal users' data? Frontier models train on users' data, right?

but also, ai has been going rogue for decades. the GPS machines of old went rogue all the time, routing drivers into lakes, rivers, and construction, sometimes with deadly results. The operator submitting an address wanted driving isntructions, but instead, the ai went rogue and sent them a bad route