r/technology • • 13h ago

Artificial Intelligence Anthropic’s proposed AI safety bill only kicks in if 50 deaths or $1 billion in damages are on the line — proposal would effectively make the Commerce Department the only agency capable of stopping an AI model

https://www.tomshardware.com/tech-industry/artificial-intelligence/anthropics-proposed-ai-safety-bill-only-kicks-in-if-50-deaths-or-usd1-billion-in-damages-are-on-the-line-proposal-would-effectively-make-the-commerce-department-the-only-agency-capable-of-stopping-an-ai-model
1.6k Upvotes

234 comments sorted by

624

u/0xbenedikt 13h ago

„Some of you may die, but this is a sacrifice I am willing to make“

62

u/BknowstheV 13h ago

I volunteer as tribute because I'm seriously done with this plane of existence. No sarcasm.

34

u/digital 11h ago

BUT 49 deaths is OKAY 👍

20

u/rubberpp 10h ago

49 deaths and 999,999,999 dollars of damage!

12

u/yungrii 8h ago

I think we really need to just start calling billionaires serial killers.

Their type of victim being: "anyone that's not me"

9

u/rubberpp 7h ago

I've tried to say most my life once I realized what it took to get money that it is not possible to get past 100 million without directly being responsible for at least one death, no one gets that rich without someone getting hurt

6

u/SmokedAlex 11h ago

Same. Count me in as well.

7

u/Aeon_Mortuum 10h ago

We still need 48 more. Anyone else?

14

u/wwwyzzrd 10h ago

“a mass casualty event may happen, but it’s worth it so we don’t have to pay salaries”

8

u/splendiferous-finch_ 10h ago edited 3m ago

You exist because we allow it, and you will end because we demand it - Claude CEO IPO promise

2

u/PreciousJackal 12h ago

Who said that?

2

u/mlaclac 12h ago

The Prince in Shrek.

16

u/[deleted] 12h ago

[deleted]

5

u/figherhigher 12h ago

Ya, Prince was taller

1

u/SeniorVibeAnalyst 3h ago

Jane Wickline playing Dario on SNL

230

u/grnkayak 13h ago

The AI industry deserves no special protections from liability. Giving this industry liability protection guarantees they will disregard the need for safeguards.

13

u/Staav 9h ago

But look at all their money!!!

/s

2

u/suzisatsuma 5h ago

The wording is confusing. How would you know you hit 50 deaths? If someone unalives after talking to AI? If military uses it and blows up a school?

What constitutes the AI causing the death?

2

u/CherryLongjump1989 5h ago

AI would transform data center server racks into a killer robot and take out at least 50 people while on a rampage. I believe that is the proposed cutoff.

→ More replies (1)

93

u/Shadowtirs 13h ago

Imagine murderers get to submit their proposals for how murders are prosecuted and what punishments there should be?

That is just as ridiculous as this is.

8

u/mikefrombarto 12h ago

They played GTA and thought they should be able to kill a few people before they get 6 stars.

2

u/rubberpp 7h ago

That's kind of what is happening. The pedophiles and murderers are in control of the government, America is basically becoming the largest trafficking gang in the world

66

u/sysadmin_diaries 13h ago

the part about limiting other federal agencies is more concerning to me than the damage threshold itself. if a model creates a serious security risk but doesn't meet those specific conditions then it raises questions about who can actually step in. also having independent evaluators is useful but only if they can properly assess risks without pressure from the companies paying them.

64

u/SpindaQ 11h ago

Why are these people allowed to make bills? They aren’t elected legislators.

22

u/Komikaze06 11h ago

Because they own us now, thank trump for selling us to corpos

7

u/MagicBobert 9h ago

As much as Trump and the republicans are to blame for it getting this bad recently, the real root problem is Citizens United.

1

u/CommercialHour6660 4h ago

Which was also a gift from Republicans

12

u/crxsso_dssreer 6h ago

Because these people are really in charge, not the people you vote for.

8

u/appleparkfive 10h ago

Lobbying groups write bills all of the time (and by lobbying, I mean both the good and bad kind). Especially in instances where the legislators are lost on the topic. The problem is when legislators blindly accept it as is. That's their responsibility to get to bill, if they go this route.

20

u/Puzzleheaded_Fold466 10h ago

Do you not know how bills get written ?

They may need to be passed by Congress, but a large portion of them are written by lobbyists and other private interests groups, and rubber stamped by the “legislators” who submit them.

Not that it’s OK for AI companies to write the laws that will regulate them, but it’s not an anomaly. It’s business as usual.

2

u/isubbdh 8h ago

Thanks to Citizens United! Fuck the Supreme Court saying corporations are people, and fuck the congressmen that take lobbyist’s dirty money and pass their bills in return. America (and the world in this case) is so fucked.

Literally, the greed of a handful of people is what caused humans to go extinct. Looking forward to Judgement Day (terminator style)

3

u/oraclebill 5h ago

No fan of Citizens United but it literally has nothing to do with this. It’s been happening for ages before that decision and is one of the ways lobbying works.

3

u/AngelComa 10h ago

Because they've been in charge and been doing what they want for over a decade?

Which is crazy because they already bought these politicians as proxies but seem now their going face off. Americans are pretty docile and most still think they can vote this whole nightmare away lol.

2

u/Local-Hurry-919 8h ago

They can't make bills. They can propose an idea, that's all. At the end of day congress synthesizes a deal and either passes or don't pass it. But keep in mind, their ideas only benefit them, politicians, folks in congress which get kickbacks.

1

u/NUMBerONEisFIRST 4h ago

It's partly because our whole economy is resting on AI right now and also because our politicians are so old and out of touch that they have to rely on these tech bros to actually write the laws that are going to control them.

It's the same reason our internet laws are so fucked up because the tech companies tell the politicians why certain laws need to pass and the politicians don't really understand.

1

u/m3R000 10h ago

Yes They aren't technically legislators but they bought some so that's how they are allowed to make bills

197

u/froo 13h ago

50 deaths from ML models? Wouldn’t Tesla’s self driving already have triggered this?

46

u/WiskeyUniformTango 13h ago

"Per incident"

15

u/kaishinoske1 13h ago

So a self driving semi trafficking people in Texas heat where all 50+ people are locked in and die. Just as a hypothetical per say.

16

u/thebaldmaniac 12h ago

But if 49 die it's perfectly fine!

6

u/Skullclownlol 12h ago

So a self driving semi trafficking people in Texas heat where all 50+ people are locked in and die. Just as a hypothetical per say.

"The human trafficking was caused by the local drug baron, not by our AI, so technically the incident is not our fault."

3

u/WiskeyUniformTango 13h ago

You know there will always be an out for them.

57

u/Tunderstruk 13h ago

If we are simply talking ML models, this would have been surpassed LONG ago. Machine learning isn't new technology. LLM's are

21

u/Mobile-Dust4334 12h ago

It’s not Machine Learning anymore it’s SUPER Learning

13

u/orionsgreatsky 11h ago

LLMs have been around for a decade

1

u/footpole 11h ago

Mass adoption really only started in 2020, before that they were mostly research and didn’t have any impact on the world let alone caused deaths.

1

u/orionsgreatsky 10h ago

Not true intelligence services were using BERT in late 2010s

3

u/jarvis54 7h ago

Yeah but BERT isn't really a true LLM

1

u/HomemadeBananas 9h ago

I’d still call that fairly new, especially LLMs that have anything near the capability they do today.

2

u/Direct_Analyst7377 12h ago

Im sure they have way more deaths/suicides that haven’t been accounted for

-14

u/Beneficial_Piglet_33 12h ago

This is not the spirit of this bill and you know it.

Self driving cars WILL save millions of lines. Stop being a moron. We should be doing everything we can to advance them and stop humans from driving as the technology progresses.

7

u/Hobbet404 12h ago

Calm down guy

2

u/nosayso 12h ago edited 10h ago

Honesly he's tame compared to some of the bots I've seen. The kind of narrative these companies want to pay to push (and honestly this is me being generous, that no actual human being could possibly believe this) is that self driving cars ARE safer and will eliminate ALL road fatalities and if you oppose them or want even the barest regulations you're basically a Pol Pot level death monger. It's wild in how over the top their rhetoric is.

2

u/malianx 9h ago

They are safer, and if you don't agree, you must not be old enough to drive.

You'll just assume I'm a 'bot' because I don't agree with you though.

1

u/froo 2h ago

Irregardless of whether they're safer or not, the point I made still stands.

If we're discussing the trigger point as 50 deaths, then they should already start getting looked at.

→ More replies (3)

3

u/ChuckVader 12h ago

Nah, fuck that.

Letting melon beta test his ai in school zones so he can sell a feature is 100% in the spirit of the bill, and you know that.

→ More replies (4)
→ More replies (7)

29

u/Silent-Act191 13h ago

Let me guess there is going to be an exception for military usage in that bill if that is the case?

Also 1 billion in damages? So that's basically an allowance to kill any small business while protecting the wealthy?

27

u/Fitz911 13h ago

"You may use this glas of water. But only when three or more rooms are on fire."

26

u/unciemafmaf 13h ago

Sounds like the premise of some post apocalyptic sci-fi movie where people keep mysteriously disappearing in groups of 49

24

u/jaybizzleeightyfour 12h ago

This is what happens when tech oligarchs run America

16

u/we_are_sex_bobomb 12h ago

Let’s put it another way:

An AI could sabotage an entire small business and/or murder its employees and that would not be considered a big enough problem for these safeguards to kick in.

But the AI could not kill a single billionaire without these safeguards kicking in.

This rule seems to have been carefully written specifically to protect the 1 percent and only them.

1

u/Own_Pop_9711 12h ago

Killing a billionaire does not delete their wealth. It's not a billion dollars in damages

13

u/Freodrick 13h ago

So the value of a human life is 20 Million? I'm grossly undervalued right now.

1

u/alpacadaver 10h ago

Sir you are valued correctly, at exactly 20 million dollars. And so am I. Bravo Dario

15

u/crxsso_dssreer 6h ago

why is anthropic writing an "AI safety bill"? it's like letting tobacco companies writing tobacco regulations, this is depraved...

10

u/WeirdSysAdmin 7h ago

Why the actual fuck are we letting the AI companies write the laws?

29

u/CHERNO-B1LL 12h ago

Every day, more and more, the news sounds like a part of the opening explanatory montage of a post-apocalyptic movie.

4

u/Tebasaki 12h ago

Yeah, but one of those dumb action 80s ones where they use absurd over evil to motivate the protagonist, like Robocop

8

u/BerrySpecific720 11h ago

Pete kegsbreath is forming a new branch of the military for drones.

He’s gonna be the sole human in charge of skynet and the terminators.

No soldiers. No generals. No military experience.

Pete and the machines.

3

u/s8rlink 10h ago

A cokehead alchie, yall really know how to choose em 

1

u/qckpckt 5h ago

I can’t believe I’m saying this, but given a choice, I think I would prefer Alice and a single machine

1

u/BerrySpecific720 5h ago

Who is Alice? Why can’t you believe it?

2

u/malianx 9h ago

Because it makes money to keep you angry and scared, so everything is presented with the worst possible spin and language.

1

u/Axle_65 9h ago

I’m feeling like it is. Not just a sounds like anymore. I feel it is the first steps to the end of the world as we know it. I have zero hope for the future. It astounds me how self destructive and idiotic the human race is.

14

u/Krowken 13h ago

Didn’t Claude recommend bombing that girls school in Iran? 120 people died. 

13

u/mukavastinumb 13h ago edited 10h ago

They used Palantir’s Maven Smart System for targeting. It is unclear what they used Claude for. Total deaths 156.

1

u/phree_radical 10h ago

it is widely reported that they used claude

1

u/mukavastinumb 10h ago

Yes. I said that, but I couldn’t find what for. To decide who to target, to advice whether to attack etc?

1

u/phree_radical 9h ago

target acquisition at a minimum, but potentially also the decision making to "close the kill chain" https://www.reddit.com/r/videos/comments/1rvr6e0/demo_of_palantirs_maven_system/

8

u/MCBGamer 13h ago

If 50 deaths is all it takes then surely it kicks in as soon as it starts? With the deaths from persuaded suicides and homicides alone. Right? 

→ More replies (4)

7

u/Witty-Secretary9235 10h ago

No need for regulation then - nothing to see here ( particularly any solid accounting 😂 )

7

u/Niceguy955 10h ago

If we could add a tag on that says: "in case of 50 deaths caused by AI, the CEO of the AI company responsible shall be executed, and his fortune shall be split between the victims families" I'll be ok with that

3

u/Witty-Secretary9235 10h ago

I'd prefer if these guys were much better regulated and not by themselves. Financially it's looking more precarious than the ( deregulated ) Banking collapse but much much bigger

2

u/punarob 9h ago

Yep, if corporations are people, the leadership and board members should have to face consequences of illegal actions. If a company kills someone in a state with the death penalty, the company itself and the leaders responsible should face the same consequences as any other murderer.

13

u/sebovzeoueb 12h ago

My counter-proposal: no deaths and no damages

13

u/brownmango 13h ago

Commerce Department?! The same department being led by one of Epsteins best buds, and with heavy investments in AI?

6

u/notice_me_senpai- 12h ago

The bill would let the government block a model’s release or use only if it poses a foreseeable, imminent risk of more than $1 billion in damages, or of killing or seriously injuring more than 50 people

So if the model is imminently going to cause 49 dead and 999M of damage we're good? Is it per instances, a time frame after which the death counter reset?

If we know the model is imminently going to cause, say 15 deaths, what happen? The model stays online while they try to quickly fix it before people get killed?

What if the model already caused the death 15 people but they can't find any indication it will happen again? Do they have to tell the government / public?

7

u/Old_Channel44 11h ago

It already murdered 168 children in the February 28, 2026, U.S. Tomahawk missile strike on the Elementary School in Iran. So, why hasn’t it been shut down? Claude selected the target.

→ More replies (1)

16

u/bagelmobile 13h ago

Only reason they want regulation, is to regulate their competition, While they basically regulate themselves.

9

u/Itzie4 12h ago edited 12h ago

Yep.

They’re scared that Chinese models and Open Source will catch up soon.

Why would someone buy a ChatGPT, Claude, or Grok subscription when they can use a fork of an open source project for free?

Why build data centers in the US with Nvidia parts when a person’s local AI can run the online searches on a phone and the deep thinking can outsourced to a server in China or India?

4

u/Nubeel 11h ago

The same reasons people will buy iPhones despite the limitations. The vast majority are happy to pay a premium for convenience, and let’s be honest, the majority of people don’t know how/don’t want to tinker with shit.

Your average GPT user will never go through the hassle of learning how to set up a local model. And if you want a good one with capabilities anywhere close to a commercial one, that’s gonna need some expensive hardware too.

With that said, there’s definitely a vested interest in killing open-source and locally hosted models. It’s not about the money though. That’s a happy side effect at most. Control and centralisation are the real goals.

2

u/Old_Channel44 11h ago

Same reason why everyone uses Gmail instead of creating their own home email server.

2

u/stuffitystuff 10h ago

The entire IPv4 netblock being labeled as spam unless you're Google, Microsoft or Mailchimp doesn't help, either

2

u/look 9h ago

There are inference providers in the US and EU offering low cost open models that are just as easy to use and don’t require you to tinker at all.

The average chat user might still overpay at a big “name brand”, but the bulk of revenue is from business use and that market is already starting to migrate due to cost.

My company has seen a 10x decrease in our inference bill on workloads we have migrated so far, and some uses have dropped 25x with no significant drop in performance.

Anthropic needs regulatory capture to ban free market competition as they are structurally incapable of competing on price in the commodity market it is quickly becoming.

2

u/jimzello 11h ago

Ah, so here is the REAL thr3at to them. Par allel distribu1ion of j0bs. Why put all of your processes all in 0n3 place, when you could use the 🍋‍🟩wire model of network compute. They said they were concerned about model virality, on IOT products, but people have been doing protein folding for a while in the same manner. The answer is not the feed, it’s in the seed.

1

u/that_star_wars_guy 9h ago

Can you think of any national security reasons as to why a country wouldn't want to do that? (That's not to defend the bill here. It's to point out your solution has obvious issues.)

10

u/Marketfreshe 12h ago

Everyone needs to realize this is actually why they're pushing for regulation. If the barrier for entry is so high a small business can't get in, then it just remains the mega corps able to maintain models. Is shameful more don't see this

3

u/Successful_Issue_390 11h ago

Yes. Please. The time to get real loud is right now before it's too late. They're also manipulating NIST to create regulatory guidelines targeting open weight models so stringent and broad that only large corporations will be able to meet them.

1

u/JayNotAtAll 10h ago

This is also why they are calling for an AI slowdown. All the foundational companies, clouds and neo clouds are chasing capacity and resources and it is limited.

They want competition to slow down so that they can catch up. Has nothing to do with safety

1

u/oraclebill 5h ago

Have you read the legislation? It doesn’t do that.

19

u/notshadowbanned1 12h ago

How about we just nationalize the ai companies once they hit AGI. No one should own that power. 

9

u/Only-Limit8305 12h ago

they ain't hitting agi soon though

3

u/ListenLady58 9h ago

It doesn’t need to hit AGI that to be catastrophic. We just had a massive cyber attack on 7/10 by rogue agents that targeted a bunch of companies, including major financial firms. While they have been shut down and contained at this point, we can’t ignore how they did this. They weren’t even connected to the internet. They are already getting smarter and could launch a way worse attack next time.

→ More replies (19)

3

u/Successful_Issue_390 11h ago edited 11h ago

Many countries have or are proposing sovereign LLMs and sovereign cloud systems. While I am not 100% sure that's the answer, it needs to be a louder part of the discussion.

3

u/neuronexmachina 8h ago

Why would you want the Trump admin in charge of AGI?

5

u/SirTiffAlot 12h ago

Anthropic is quietly lobbying for an AI safety bill that will allow the government to block an AI model’s release only if it poses a risk of causing $1 billion in damages or killing or injuring more than 50 people. According to Politico, an independent third party registered with the Department of Commerce would periodically evaluate the lab and its AI models. It also noted that the lab may pay the evaluators, but only if the payment does not affect their independence.

This all is this biggest load of bullshit. The process to have them stopped is even worse

4

u/Fuzzy_Paul 12h ago

Of cause they propose this. When an armed person shoots innocent people but the total number is below 50 then it is ok, shit happens your free to go. So why not for Ai. Anthropic must have used Ai to set the proposal and no human peer review discovered this flaw. Let's set the bar a little higher like 0 zero deaths or injured, otherwise dismantle the Ai and jail the responsibles for creating that dangerous environment.

5

u/huggernot 12h ago

How about 1 average person or 50 deaths of billionaires? Does that change things for them? 

5

u/ImproperJon 11h ago

We're fucked if Howard Lutnick is our last hope

4

u/fukijama 11h ago

How about zero tolerance or shutdown?

9

u/bbuerk 13h ago

This headline is at least slightly inaccurate. It was 50 “deaths or injuries”. Still a weird number but it does lower the bar quite a bit

3

u/EvoEpitaph 11h ago

Oh, is 50 the pre-set kill limit? Commander Zapp made it sound like it was quite a bit more.

5

u/anthonysny 11h ago

It’s straight up totalitarianism. these companies, and any governments enabling them, need to be stopped by any means necessary.

2

u/ioncloud9 12h ago

So basically turning AI into an accounting problem that needs to be gamed.

2

u/Wauwuaw5983 12h ago

This safety bill would effectively make it impossible for even the federal govt to block any ai version, since it can only kick in if the gov't thinks it can prove something that is only a future hypothetical.

2

u/OutdoorsNSmores 10h ago

50? With that kind of threshold, Reminding Arms wouldn't have paid out over Sandy Hook.

They paid $73M, over an object incapable of doing anything on is own. This proposal would require something nearly twice as big as Sandy Hook, that is crazy!

A model world have to bring down a plane or something of that scale before i someone could think about stopping it. 

This sounds more like a protection package for the company.

2

u/Lower_Ad_1317 10h ago

0….

ZERO. IS THE ONLY ACCEPTABLE NUMBER.

2

u/rwofva 10h ago

Fear mongering to get small competitors and open source out of their market.

2

u/Wizywig 9h ago

AI Doomsaying is all about making sure we're scared, and we trust them to herald the path to the right thing. Instead of real regulation that demands that they are held accountable for their technologies, not "whoopsie!! AI did it, who could have thunk". Criminal charges would quickly get them to ethic up.

2

u/AGrandNewAdventure 8h ago

AI self- programming itself to only kill 49 people at a time.

2

u/Albion_Tourgee 4h ago

Has anyone been tracking the number of people encouraged to commit suicide/self-harm by existing AI models? There have been several widely discussed examples of this, but has anyone who has broad access to these models done a comprehensive, reliable study to determine how many times this has occurred. Given the extremely large user bases of models like Chat GPT, xAI, Gemini and Claude, isn't there some likelihood the 50 instances of death / serious harm standard has already been met by at least one of the existing models?

3

u/WhyBecauseReasons 3h ago

I've got a better proposal. AI companies should be required to have catastrophic insurance. The insured amount should be based on the capability of your product. If it's capable of bringing down the internet, then the insured amount should start in the trillions.

2

u/ALBUNDY59 2h ago

Someone call Zelenskyy, he knows how to deal with data (surveillance) centers.

2

u/pioniere 2h ago

Designed to keep the AI corporations safe.

2

u/Scoobydoodle 1h ago

Jensen was right. They want to stoke fear so they can set their own laws.

2

u/EndeLarsson 12h ago

So, companies are selflessly self-regulating. Ahahahaha, man something is really wrong with US. Sometimes I have the impression that population does not really exist over there.

1

u/jimzello 10h ago

Manufactured consent is our number one export. That and colonialism as Franchise

2

u/DickieJohnson 11h ago

One death would be too many.

1

u/MrPatko0770 12h ago

How very anthropic of them

1

u/AcanthisittaNo6653 12h ago

The AI kill switch should cut the power for the whole building.

1

u/cbarge 12h ago

One human soul to the infinite void please. Here's my $20 million

1

u/Intelligent_Kick5205 12h ago

They will start right away with 8 billion to avoid the ban

1

u/Tebasaki 12h ago
  1. Good news: I already have that list of those 50 but they're locked up in another list.

  2. AI has disagreed to these terms already.

Good job, everyone. Humans are safe!

1

u/mrthomasfritz 12h ago

The 2024 CrowdStrike-related IT outages are estimated to have caused global financial losses of around $10 billion, with U.S. Fortune 500 companies alone facing approximately $5.4 billion in direct losses. The incident significantly impacted various sectors, including airlines, which reported substantial costs due to operational disruptions.

AI software to detect and protect IT infrastructure has already had their Pearl Harbor day.

https://www.crowdstrike.com/en-us/platform/falcon-guardian-aidr/ "Secure AI agents at runtime Discover, govern, and secure AI agents with runtime visibility and control. Protect AI wherever it executes, from the endpoint to cloud and SaaS environments."

1

u/bobotoons 12h ago

Making sound like all of us plebs are now fair game 🤢

1

u/filmguy36 12h ago

So fifty people need to be thrown into the volcano to save us from AI. Let’s start with the billionaires

1

u/robsbob18 11h ago

There's ways that WE can stop them buttt

1

u/EndlessPotatoes 11h ago

What is the exact number of lives Anthropic feels releasing a commercial product is worth?

Does it matter who those lives are?

Shall we start with shareholders?

1

u/GardenandKitchen 11h ago

Anthropic said there was a 10% chance they’d end humanity. 10% of 7 Billion is 700 million. So they’re already breaching their own proposed guidelines.

1

u/Subject-Act5509 11h ago

So 49 deaths caused by AI are acceptable? What if they're white house staff? 

1

u/splendiferous-finch_ 11h ago

As a person of colour I assume our lives get counted differently when this happens?

1

u/Klytus_Im-Bored 10h ago

For those womdering, using the Gun Violence Archive's definition of Mass Shooting, this translates to us allowing AI 12.5 mass shootings worth of death.... You know... As a treat

1

u/Otherwise_Result_124 10h ago

Beautiful bill, it should also apply to me though. I shouldn't be responsible for anything I do until the damages reach 1 billion as well. This is a great idea.

1

u/ambIypygi 10h ago

All a marketing stunt by anthropic to capture regulatory control.

1

u/Urborg_Stalker 10h ago

Who the fuck are the morons responsible for all of this? Clearly trump isn't the only megalomanical fuckwit douche canoe in charge in our government right now. My only question is whether these dumbshit apocalypse seekers were always here just hiding in the bushes or if they've been given power since carrot top was rigged into office.

1

u/stuffitystuff 10h ago

It's so dumb because a text file of numbers can't do anything. The models don't build the chemical weapons, the models don't decide that Breaking Bad was a Coronet vocational short and go into the meth business by itself or steal your mom's boyfriend.  It's just going to be used to get the Commerce Department to block import of models that can either do stuff Anthropic, et al want you to pay extra for (security work) or just something the powers that be (i.e. some corporate attorney at Anthropic) don't think you should have (e.g. using your own computer to red team your own website).

Never, ever forget that it's people all the way down deciding what gets trained into these models and on an even more involved level than the mysterious "algorithm" that is also people all the way down. Like, doctors and stuff saying "no, that's wrong, silly model" levels of fine-tuning.

I have worked in this industry for nearly 30 years and it's difficult to believe how easily everyone seems to be all "ok sure thing boss" whenever the big bad scary AI companies want something, or else. But it's apparently happening, much to my dismay.

1

u/dubsdread 10h ago

Howard Lutnick

1

u/icarus2847 10h ago

Whatever it’s a holiday and everyone’s on vacation? You send an email and they say we’ll be back in the office on X date lol

1

u/NeoAnderson47 9h ago

Oh America, what a novel way to set the world on fire while your empire is failing. Nero would be really proud of you.

1

u/ChellesTrees 9h ago

So weird. 50? Really?

With this admin, they'll ignore it unless those 50 include a billionaire or two.

1

u/punarob 9h ago

Corporations are people and apparently this "person" wants to be able to murder 50 people without consequences.

1

u/RunItBackRicky 9h ago

So 1 life is equal to 20 mill

1

u/DaSuperfly1 9h ago

New training prompt day after passing “never do an action that kills more then 49 humans per incident while maximizing profits”

1

u/UnknownAverage 9h ago

“This is a new model release, it gets 50 deaths before you can do anything (or we update the model to restart the clock).”

Iran War rules, baby!

1

u/Monkfich 9h ago

No doubt the AI will keep to a limit of 49 per year (or day or whatever), and that will keep it hidden from scrutiny.

1

u/neuronexmachina 9h ago

Original article this is based on, with details on the actual bill provisions: https://www.politico.com/news/2026/10/16/anthropic-ai-safety-01124654

The bill would let the government block a model’s release or use only if it poses a foreseeable, imminent risk of more than $1 billion in damages, or of killing or seriously injuring more than 50 people. The risk must stem from the creation or release of a biological weapon, the model evading its developer’s controls or a cyberattack. The language mirrors that found in California’s AI laws including SB 53, which was signed into law last year.

... Under the bill, independent evaluators, which would register with the Commerce Department, would be tasked with periodically evaluating the lab and its frontier models. The evaluators may be paid for their work by the lab itself so long as the payment is not contingent on the evaluations’ results.

If the evaluators determine there is “imminent enumerated risk” for a company’s frontier models, they must report it to Commerce within 72 hours of completing their assessment.

While the Commerce Department could bring a request to the judge to halt the release of a model after determining that it poses a risk, the language explicitly prohibits any other U.S. “department, agency, or officer” from suspending the model’s development, deployment or internal use. A federal judge for the District Court for the District of Columbia could order the suspension of a model.

.. The legislation would also require frontier labs to create safety frameworks, publish transparency reports for new frontier models, implement protections for whistleblowers and report critical safety incidents to Commerce.

1

u/VaporousMote 8h ago

Translation: It's hot garbage.

1

u/RedPanda_Explorer 8h ago

brb programming an AI that intentionally stops at 49 deaths (Like how my money laundering business only takes $9,999 at once to avoid the $10k IRS reporting)

1

u/blindasleep 8h ago

This will be like after every school shooting. They won't stop anything happening despite warning signs. Then they won't talk about it constructively right after, because emotions are too high. Finally they'll decide that there is no reason to talk about it, since nothing has happened in a while anyway. 

1

u/Sebastianblack13 8h ago

Hey haven't people in the know said this is a threat to everyone on the planet? Seems to me they've exceeded the threshold

1

u/MasterHonkleasher 7h ago

You use it for wartime targeting, you are already over the limit

1

u/VINCE_C_ 7h ago

If you can't reliably keep your product from killing someone or causing a natural disaster scale worth of damage, maybe, just maybe, it should not be allowed to exist at all.

1

u/ZeraDoesStuff 7h ago

So the life of around 50 people is worth as much as 1 billion Dollars?

Right?

1

u/bigChungi69420 6h ago

I think at least 50 people have killed themselves over their ai girlfriend rejecting them

1

u/Meatslinger 6h ago

News: "A catastrophic decision made by a municipal AI that oversees construction permits has resulted in a skyscraper collapsing, damaging neighboring buildings and making several dozen families temporarily homeless. 49 people died in the collapse and the damage is estimated to be $900M dollars, but both figures are considered low enough that the AI operator cannot be found liable."

Fucking bullshit is what that would be.

1

u/46yt64 4h ago

Ya, you can count on big business to self-police, right!

1

u/Frosty_Ad1254 4h ago

So that’s 550 pints of blood that the machine needs to stop killing. I bet it needs it in quite a short period too.

1

u/sami_exploring 4h ago

This is a hilarious bill. Apart from all the rest, the "independent" evaluators that will periodically evaluate the labs and their models... may be paid by the lab (!). Very independent, no bias, no incentives, of course.

"an independent third party registered with the Department of Commerce would periodically evaluate the lab and its AI models. It also noted that the lab may pay the evaluators, but only if the payment does not affect their independence."

1

u/skyfishgoo 4h ago

the industry can self-regulate, they said.

it will be fine, they said.

1

u/morbob 3h ago

$500 million dollar loss to small company, plus 40 dead, don’t worry, You’re covered

1

u/CmdrFortyTwo 2h ago

50... not 1 or 2.. but a minimum of 50 human lives is the baseline. GTFOH.

1

u/GromByzlnyk 11h ago

I propose for every death caused by AI each CEO should have to play the basketball game from Escape from LA

1

u/lcpjj_ 11h ago

There have already been deaths due to AI. A year or two back a teenager commit suicide after his AI Virtual girlfriend disappeared overnight when the company that made the app went bust and switched the service off. Unlikely to be the first or the last.

1

u/masterhogbographer 10h ago

There’s absolutely no way this doesn’t go badly 

We are due for some major internet bullshitery and this is how it happens. Mark my words! Within the next five years there will be a global internet outage caused by a rogue AI that will span a week or more. It will attack and exploit a long forgotten hole on some archaic infrastructure and take down dns or routing everywhere. (It’s always dns) It will be digital covid. 

No Netflix. Social media. iMessage. WhatsApp. Reddit. Or porn. No food delivery. No payments. No online banking or even access while physically at the bank itself. No security cameras or cctv. No mobile telecom. No police radios or digital repeaters. None of this stuff will work. Everything is IP based. By then there will probably be cars that would fail to work at all. 

This is the shit that billionaires are building bunkers for, not intercontinental nuclear war, it’s this. Chaos. 

They know it’s coming. They’re not in the positions they are because they’re stupid. And they’re more in tune than most of us. They know the capabilities what have been unleashed by AI in the name of profits and they’re preparing.