YHK - yet another hacker news clone

Pentagon: Palantir AI Overreliance Led to Strike Killing 123 Iranian Children

99 points devonnull 41 minutes ago 49 comments

allears 39 minutes ago | parent

Given that everything emanating from this administration has a political agenda, does this mean that Thiel is on the outs?

ambicapter 29 minutes ago | parent

> unreleased Pentagon probe

From TFA.

esseph 17 minutes ago | parent

He's rich and connected, of course not.

vfclists 37 minutes ago | parent

The one that pulls the trigger bears the responsibility.

verdverm 22 minutes ago | parent

I would argue `s/pulls the trigger/gives the go ahead/` is intuitively more appropriate. The sailor on the boat that fired off the tomahawk (iirc) was likely not in a position to make an in-the-moment determination

esseph 18 minutes ago | parent

The one that pulls the trigger is often not the one doing any targeting, especially on a strategic strike weapon like a tomahawk. They are receiving a target data package with preprogrammed instructions. Their role is far more... mechanical. They're not seeing the targets at all.

derektank 18 minutes ago | parent

Completely disagree. When we’re talking about something like air, missile, or artillery strikes, the operator of the weapon system may literally never even see the target, on a screen or otherwise. They may simply be entering targeting information provided to them by an operations center. These decisions are social and group decisions, with distributed responsibility, which is why it’s so important to have institutions like the Civilian Protection Center of Excellence which historically has identified and helped implement the policies which prevent these kinds of disasters, but which unfortunately the current administration decided to shut down without a replacement.

Sometimes, the difference between a successful strike on a lawful combatant and disaster is one analyst pointing out that the building the enemy is hiding in is next to a square that’s used for a market on Tuesdays and wouldn’t you know it, it’s Tuesday. That’s not something the person “pulling the trigger,” can always be expected to know, which is why every ops center needs to be designed to elevate those voices, when the time comes

mito88 12 minutes ago | parent

the operator was following orders...

jacquesm 18 minutes ago | parent

No. That's too easy on the higher ups, they never pull triggers.

mhh__ 33 minutes ago | parent

Blowing up a school is proven route to diplomatic triumph and regime change!

colingauvin 27 minutes ago | parent

While I think that it's important to do root cause analysis, we need to be very careful not to absolve blame from the people that decided to shift a significant weight of decision-making authority to an AI. An AI can be tried in a court. There has to be a responsible human for everyone one of its actions, especially when it's killing (-people-) children.

I cannot believe this has come to pass.

jacquesm 20 minutes ago | parent

can -> can't

feverzsj 26 minutes ago | parent

Despite what AI told them, they knew it was a school full of kids. They still chose to kill them all with bomb. Only maniac would do such thing.

throwaway198846 16 minutes ago | parent

> they knew it was a school full of kids.

I would like to know how you know this

bix6 25 minutes ago | parent

DOGE strikes again. Edit actually looks like it was Hegseth himself, king Testosterone.

> Staffing on civilian harm mitigation teams across the U.S. Department of Defense had also fallen by roughly 90% over recent years, shrinking to fewer than 20 people overall. Centcom’s group shrank from 10 people to one.

https://www.propublica.org/article/hegseth-civilian-harm-mit...

krapp 22 minutes ago | parent

"maximum lethality, not tepid legality."

verdverm 20 minutes ago | parent

"stupid and energetic, the worst kind of general" - Indy Neidell

derektank 15 minutes ago | parent

Paraphrasing the German General Kurt von Hammerstein-Equord

verdverm 14 minutes ago | parent

in reference to a WW1 Austrian General (for others who pass by and are curious, since you seem to be in the know)

https://en.wikipedia.org/wiki/Franz_Conrad_von_H%C3%B6tzendo...

macintux 23 minutes ago | parent

Related: the U.S. nearly boarded a Chinese boat that AI incorrectly flagged as carrying nuclear weapons materiel.

https://gizmodo.com/almost-started-a-war-us-military-nearly-...

chews 23 minutes ago | parent

nice, Karp really reeled in a nice big fish, and ofcourse he is going to blame OpenAI/Anthropic for giving the targets, this explains his CNBC rants about the frontiers, his tool was the trigger in a killchain of hallucination.

verdverm 21 minutes ago | parent

time to make Palantir the next Flock

France gave them the boot, others can too

vikas123456789 21 minutes ago | parent

Whole world is turning into an unlivable place and people who are doing it are not taking responsibility.

jacquesm 19 minutes ago | parent

They never have. Their money insulates them from the consequences of their actions.

andsoitis 11 minutes ago | parent

There’s a threshold where the money won’t insulate anymore.

sys32768 21 minutes ago | parent

Tragically this won't be the last time AI kills a lot of innocent people.

angoragoats 10 minutes ago | parent

AI didn’t kill innocent people here, the Department of Defense and its decision-makers did.

elAhmo 10 minutes ago | parent

This is not AI killing poeole, this is US government killing poeole.

igleria 20 minutes ago | parent

so who is paying for this? "humans must always remain accountable for artificial intelligence (AI) decisions"

1239971 20 minutes ago | parent

An Maven AI used Claude, which was denied here multiple times. Maybe call the next model after Fable Minab!

legitster 20 minutes ago | parent

> It found the U.S. “failed in its obligation to do everything feasible to verify” that the school was a military objective and that the failure “went beyond mere negligence.” The report said the United States “directed the strikes at the building of the school while being aware of a substantial risk of striking a civilian object and acting recklessly as regards the possibility that this would happen.”

Reading the details, "AI" doesn't really seem like the culprit -it's a scapegoat.

The intelligence that it was no longer a military target never entered the target database, the team that was responsible for vetting the target list was gutted, and said team was never even consulted.

The White House wanted 1000 targets and pulled from their database without any due diligence. Whether it was an AI call or an SQL query - this was from pure human maliciousness and incompetence.

jmathai 18 minutes ago | parent

> The Minab site, which was cataloged as an Islamic Revolutionary Guard Corps facility due to outdated data, was fed into Maven with other candidates and came out as a recommended day-one target. Target-list work that once took hours was condensed into minutes.

I feel like we're optimizing the wrong metric here.

jacquesm 16 minutes ago | parent

I'm going to bet that absolutely nobody higher up will end up going to jail for this.

sublinear 15 minutes ago | parent

Is that really the case? Stories like this always have all sides muddying waters.

The terrorists are well known to use children and the rest of the public as hostage meatshields, and the US military is always looking for something/someone else to blame for the dirty work they have to do.

The broader public is so polarized by performative politics and botspam that they eat these stories up, but nothing I'm saying is secret. It's been this way since forever. Slow news today?

I'm not even defending AI at all and think these are crappy choices, but compared to humans gathering intelligence who can go rogue, what alternative does anyone really have?

23j423j423hj 13 minutes ago | parent

It was a school. No one on either side is denying this.

imranq 15 minutes ago | parent

> Officials said some users expected Maven to flag stale records or contradictions in the intelligence assembled for potential targets, though it is not clear why they thought the system would do that

This is what happens when people who don't understand AI and its limitations embrace it wholeheartedly, thinking it's going to behave like The ultimate analyst. Except it's not. Its got blind spots in more places than anyone knows. And not to mention the system itself has multiple areas of failure.

banubis 12 minutes ago | parent

The pentagon says it was Palantir software’s fault, Palantir says it was the fault of bad data input to their software. Where’s the accountability by both organizations? How did we get to a world where military decisions leading to innocent deaths are responded to as if it was just another miscommunication implementing a B2B SaaS platform.

angoragoats 12 minutes ago | parent

In a just world, Pete Hegseth and the entire board of Palantir would be in prison for this.

nozzlegear 9 minutes ago | parent

[delayed]

sedan_baklazhan 7 minutes ago | parent

That is absolutely irrelevant. USA will never take responsibility. Nobody will be prosecuted. No compensations will be paid.

RIMR 7 minutes ago | parent

I don't want to see the Pentagon try to pass the blame here, but I am all for giving half the blame to Palantir.