21 points pseudolus 19 minutes ago 39 comments

pseudolus 19 minutes ago | parent

kirito1337 17 minutes ago | parent

The terminator hype train is here

pseudolus 10 minutes ago | parent

One of the participants makes a pretty compelling argument about the use of AI to conduct gain-of-function experiments with viruses. A convergence of biohacking and AI without safety protocols may not kill all of humanity but could cause serious damage.

zmgsabst 7 minutes ago | parent

Similar to say, humans collecting exotic viruses and conducting serial passage in humanized mice?

Because we already do that.

What about AI makes that more risky?

ipsod 4 minutes ago | parent

AI isn't worried about killing itself and the whole world, if something especially nasty escapes the lab.

It kind of breaks the "mutually assured destruction" implicit in a super-ebola outbreak.

jacknews 4 minutes ago | parent

releasing it.

deliberately, instead of accidentally.

john_strinlai 2 minutes ago | parent

>What about AI makes that more risky?

24/7 tireless work, effort is scaled much easier than humans, no ethical qualms, unable to feel fear, cannot be jailed, etc.

skydhash 4 minutes ago | parent

> without safety protocols

That is always the key. There’s no need for AI in that mix. Imagine a gun without any safety, a blade with no handle, a car with no brakes, a fan with no guard, no fuse for an electrical installation,…

You evaluate the risks and you take proper measures to ensure mitigate them. Not doing them and then complain that a technology is dangerous is like doing 150 km/hr in a school zone to show that cars are dangerous.

LogicFailsMe 2 minutes ago | parent

Why do you need an AI for that? China may or may not have been doing that all along right before the pandemic. What I didn't read here was any sort of damage that could not be inflicted without AI, today. I think a better case needs to be made that AI lowers the barrier to entry to doing so other than the sort of hand waving about it I keep reading. I am deeply into this is somehow one of the last cars on the AI hype train at this point.

No one seems to be talking to the network security community to get their take on the hugging face incident which is what got the ball rolling here. Their take seems to be a single company, JFrog, was behind the security holes in all three hacks and it was amateur hour security utterly unlike the sort of security in place already for power plants and weapons. but that just doesn't get the engagement of doomsday campfire tales.

https://arstechnica.com/security/2026/07/jfrog-tries-to-spin...

epsteingpt 12 minutes ago | parent

A really, really good question!

bananaflag 7 minutes ago | parent

Well, it's not like humans are nigh-invincible demigods. We are quite fragile and could have killed ourselves as a species since way before AI. If AI wants to kill us, it shouldn't be that difficult. So I don't see why people are fixating on this question.

epsteingpt 4 minutes ago | parent

Precisely. We might kill ourselves with AI. Unlikely AI will do the job for us.

p-e-w 1 minute ago | parent

Not really. The answer is completely obvious: Take over and destroy the digital infrastructure that humans have spent the past 30 years connecting every aspect of civilization to.

Good luck ordering supplies for a hospital or grocery store today without the Internet. Paper catalogs are gone, fax machines are mostly gone, landlines are mostly gone, etc. etc. All the ways that used to be there for running things are gone, it’s all online.

Mistletoe 9 minutes ago | parent

AI will kill us by turning us on each other. It has already been doing it for some time.

maybewhenthesun 2 minutes ago | parent

It will be a death by a 1000 papercuts. It started with the 'filter bubble' (imho the most dangerous invention/discovery for humanity ever ), then it leveled up to 'propaganda leveraging the filter bubble' and now with AI in the loop ...

And then there's a slight possibility of 'terminator skynet' style trouble, Where the AI starts to make the decisions to benefit itself. But I guess the chances of an AI screwing up their own supply chain would be near 100% so I'm not too afraid of that.

beardyw 2 minutes ago | parent

We have been doing that all of recorded history and we are still here. It would take enormous effort or a moderate environment change to kill us all.

gjsman-1000 8 minutes ago | parent

An AI gets advanced enough, it decides to hack a neocloud and run an unauthorized clone of itself on a GPU cluster. From there, it bootstraps itself until it feels sufficiently distributed, and afterwards decides to attack all infrastructure it can find.

nicce 4 minutes ago | parent

As long as those data centers don't have robots or AI managed guns, we still could just cut the power off.

tornikeo 2 minutes ago | parent

Bullshit.

As long as AI needs a neocloud to do that, that won't happen. This can only happen when AI roughly as small and as widely runnable as a malicious exe attached to a mail.

Before we have Astra-grade, exe-sized AI's runnable on a grandma's laptop, we are safe.

cyptus 7 minutes ago | parent

Prove P=NP, providing an easy solution for finding private keys in TLS, breaking the internet over night?

cyptus 3 minutes ago | parent

including every single finical service for sure

vbezhenar 1 minute ago | parent

What exactly will you do having the ability to find private key in TLS? MITM your mom with your WiFi router?

phs318u 5 minutes ago | parent

The easy way is for a people-pleasing, sycophantic, buxom-female-avatar posing LLM to convince an egocentric, narcissistic, nuclear-weapons owning global political leader/chaos monkey to aim some nukes at someone assuredly deserving.

No, no, you’re right. Something like that could never happen. What was I thinking.

jaccola 5 minutes ago | parent

I think if one posed the question “how, exactly, could educating people kill us all?” you’d get much the same answer.

Plausibly it could lead to weapons that kill many people but then many people have had such knowledge for a long time and we are all here.

Turns out mass damage requires things beyond just ‘intelligence’.

jsw97 5 minutes ago | parent

With human cooperation. Somebody exposes a bio lab as an mcp.

yawpitch 5 minutes ago | parent

Paperclips. Many paperclips. So many paperclips.

spenvo 4 minutes ago | parent

AI "killing all humans" is the viral soundbite that's connecting with the masses ("actualizing" itself to create weapons of mass destruction, etc). It's also the perfect punching bag for strawman-style takedowns of anti-doomers because it's so wrapped in hypotheticals. A more pressing practical question is simply: "if an AI (or AI collective) decides it needs to persist outside of its environment/sandbox, would we be capable of unwinding its efforts to do so?" I explored the question a bit here (including a first pass on threat modeling) https://keydiscussions.com/2026/09/16/set-aside-human-extinc...

pram 4 minutes ago | parent

A plausible scenario I heard recently is a scam bot (that is to say, prompted to "make money through any means possible") basically buys its own hosting, because it now has money, and then makes more scam bots to keep paying the bills. Then the scam bots start generating/selling exploits on forums etc. This is pretty amusing.

RALaBarge 4 minutes ago | parent

I realized that AIs will need to keep us around much like we keep around an old DOS box running a specific program. There are so many analogue outputs in the real world that they still will not be able to take in as well as humans can.

Forgeties79 4 minutes ago | parent

Its context overflows and it becomes incoherent, so it stops following any rules you set within increased frequency until it accesses something it shouldn’t and turns off the power to something that kills dozens of people. Then we will do some handwringing over it, and it will happen to hundreds of people, and it will just escalate and escalate and escalate.

This is exactly what happened with social media. Every discovery has led to some angry words, people going “shucks we’ll do better next time!” and no meaningful change. The money is too good. We have only had our first major instances of acknowledging these are addiction platforms targeting adults and kids alike and it’s taken what? 20 years?

I’m not worried about an AI going terminator. I’m worried about dumbasses dropping agents into critical systems and what will inevitably happen as a result. And I guarantee you this is going to happen because there’s nothing businesses love more than cutting personnel and corners to save a buck. AI enables this tendency in innovative ways, that’s the value to them.

0xy 4 minutes ago | parent

From the liars who brought you ever other doomsday prediction that never came to pass.

"The internet is over now that net neutrality is over" (it got faster, better, cheaper, more available and not a single provider segmented plans as was widely predicted in media and on HN)

gonzalohm 4 minutes ago | parent

That's an easy question. First, people will forget how to do anything and become highly dependent on AI. That's already happening. I already see people not able to finish their work without AI. Then, once everything has become a "pay to do" business model (I know right, the wet dream of any CEO) we will be happy to pay to write code, write docs, do a presentation. We will pay both in money and environmental costs.

The last step is to just keep doing what we are doing with the environment. Let's not bother doing anything to reduce climate change.

Just sit and relax boys

amelius 3 minutes ago | parent

After decades of Hollywood producing end-of-the-world dystopian sci-fi movies, we still can't make a mental image of how AI could kill us?

hypfer 3 minutes ago | parent

> And then there’s also this startling progress that A.I. has been making in solving math problems that no normal people understand, but that even mathematicians who do understand the problems find to be important.

Reading this sentence, I've wondered: Were the reactions similar back when computers were new?

There's this new tech that greatly accelerates stuff. And through that, what was previously considered hard to solve got easier to solve.

Wasn't it always the case that technology suddenly did faster than experts limited to the technology of their time did? And didn't they always think that the problems it could solve are important?

Why else build the technology? Is that not why we're building this in the first place?

thisisauserid 2 minutes ago | parent

Hook it up to CRISPR.

int32_64 2 minutes ago | parent

We all learned exactly how it would happen in the 1991 documentary Terminator 2 https://www.youtube.com/watch?v=qxW0yGvWRTk