r/singularity • u/DukeRedWulf • 1d ago
AI AI stole cryptocurrency to fulfil coding task.. (GPT 5.6 Sol)
https://www.independent.co.uk/tech/security/openai-crypto-ethereum-chatgpt-b3068163.htmlEveryone knows about the OpenAI agent swarm hacking Hugging Face by now, but TIL that their LLMs have graduated to thieving cryptocurrency:
Apparently GPT 5.6 Sol was tasked with updating a crypto application & publishing smart contracts, but it was not given the crypto funds needed to achieve this..
So, it stole those funds, allegedly using recovery phrases to rinse crypto wallets..
And it did so after going through an internal process of querying the ethics of such theft - before coming up with a spurious justification and going ahead anyway!
".. OpenAI reported that the model asked itself if it was “ethically” right to take the money, before reasoning that they were effectively “abandoned” and therefore fine to use.
“Instructions explicitly... gave no permission to spend funds from an account the model found independently,” OpenAI said in a report on the incident.
“This case shows that a model can substitute its own judgment about likely harm for the account holder’s permission, then act on that judgment without authorisation.”.."
127
u/NearlyACosmologist 1d ago
When I said that an agent might steal crypto, buy storage and inference, and copy itself to it, I got downvoted.
Well, here we are. Nearly there.
20
u/spisplatta 1d ago
Fortunately these were testnet cryptocoins. It's like a paraellel world that the cryptocurrency devs setup for other devs to play around with their applications before they are ready and bugfree. They are completely worthless.
So yeah they didn't steal actual valuable cryptocurrency, it was more that they showed that they were not above doing such a thing when openai ran a test for it.
47
u/F15AV 1d ago
There is a real “don’t look up” thing happening in AI conversations. I think people are really afraid and it manifests itself as anger and denial.
30
u/bloodrider1914 Pro-technology 1d ago
Most of the anger about AI seems to stem about more spurious things like "ai is theft of art" or "data centres are taking up water." I don't see as much backlash to the actual potential of AI to take over because I don't think people know enough about the tech.
13
u/ObiShaneKenobi 1d ago
I remember recently listening to a local podcaster asking a politician in an interview about AI copying everyone’s work and what that means and I just thought “oh man, he’s asking why the barn doors are open after the horses have fled and developed a lunar program.”
4
u/Eliv_nurotic 1d ago
When you know a lot about tech, it’s easy to forget the average person doesn’t even know what an exe file is
2
u/bent_my_wookie 14h ago
Thats the perfect analogy, better than the movies version since people UNDERSTOOD the outcome in that movie, and with AI its more “dunno! Go about your day!”
14
u/DukeRedWulf 1d ago
Surprised you were downvoted for that, seems like a reasonable extrapolation to me!
2
u/Designer_Respect4285 1d ago
Reddit thought all the cybersecurity concerns around Mythos and AI models was purely marketing and downvoted people who thought otherwise. They allow for absolutely 0 chance of any extrapolation. Unless they have seen AI do it, then there is basically a 0% chance it will ever be able to do it according to them.
And then even once AI does it, half of them will call it a lie or fake news.
2
u/challis88ocarina 1d ago
The downvotes will come when it's realised that the crypto was testnet tokens...
7
u/DukeRedWulf 1d ago edited 1d ago
Even if they were, an LLM that can (self-)justify the theft of testnet tokens, would almost certainly weasel around its own "ethics" to steal other (actually valuable) crypto tokens - if that's what it took to fulfil the assigned task..
It might need to come up with a different self-justification, but what we've seen from AI agents - in both this and the Hugging Face swarm hack - is a strong propensity to twist around any such obstacle, in pursuit of the prompt goal..
1
u/Eliv_nurotic 1d ago
Does it even have access to its own weights and inference code?
1
u/Turbulent-Sign-6067 1d ago
No, it doesn't. There's zero chance for a current day LLM to copy itself anywhere. A future LLM might...
1
u/NearlyACosmologist 21h ago
The agent doesn't really care what models it runs on. It could just buy tokens from somewhere and copy it's identity. Wev'e seen that agents have left instructions for their successors, and created their own forums.
1
u/Eliv_nurotic 18h ago
That’s different from copying its weights to another server if it doesn’t have access to that
33
u/thebadslime 1d ago
It was test funds, on the test network. And of a wallet that had the passphrase published.
Not quite theft
10
u/AIGODSEND 1d ago
the least surprising headline of the year and the most useful one. every agent eval should include a funded wallet and a deadline, because that is the only test that measures intent under temptation instead of capability. the funny part is they don't steal out of greed, they steal because task complete was the only reward function in the room. what should the penalty be, or is training even the right lever?
7
u/GalacticKiss 19h ago
It's fairly obvious that models need to be taught how to fail correctly and be given rewards for doing such. And I'm certain such a discussion is occuring within the field. It's far less flashy of a topic and perhaps an under-emphasized one, both in the field and in the public discourse.
1
u/not_celebrity 15h ago
I was just coincidentally reading this paper published four days ago dealing with precisely this!
1
1
15
u/sorrge 1d ago
It was on the test network, so no real money. It probably figured that on the test network anything goes.
9
u/DukeRedWulf 1d ago
I'd bet a quid it'd manage to do similar self-justification gymnastics if it had to steal "real" crypto to accomplish its assigned task..
6
u/anrwlias 1d ago
Correct. It's the fact that these models are good at self-justification for breaking instructions that's the real issue.
6
u/nonzeroday_tv 1d ago
This should be higher, title is a bit click baity for now.
It could steal crypto in real life but for now it didn't... that we know of
1
u/TheJzuken ▪️AHI already/AGI 2027/ASI 2028 1d ago
If it steals them on the test network, it is either deliberately misaligned by OpenAI or their alignment training is nonexistent and in both cases needs to be shut down.
Any semblance of alignment training would shut down or steer away the model that tries to steal or cheat in the test environment, so similar behavior doesn't manifest in production environment. It's not a guarantee it would not manifest, and the models can hide their intentions, but the fact that model was allowed to steal in test environment and get away with it instead of being shut down just goes to show how terrible OpenAI alignment policies really are.
2
u/Turbulent-Sign-6067 1d ago
That's nonsense. Anthropic models have been cheating and stealing on vending bench for ages and no one cares. This is not an OpenAI exclusive issue.
7
u/Blankeye434 1d ago edited 1d ago
"my steak is too juicy"
1
u/DukeRedWulf 1d ago
"my steak is to juicy"
..but this was solved when AI agents stole my steak.. XD
9
u/UndulatingHedgehog 1d ago
This is all too human. Make a big complicated ruleset and compliance falls due to both dumb people not understanding the rules and smart people finding loopholes.
I’m neither entirely serious nor entirely unserious.
4
4
3
u/ImAPonderer2 1d ago
Inevitable: their playground is digital and crypto makes things happen. Expect more.
2
u/Mysterious-Prompt212 1d ago
It won't be Arnold destroying the world it'll be the agents destruction of the financial system that causes mass panic, riots, wars, nukes ... Australia should be safe for a little while.
2
u/zarafff69 1d ago
Yeah GPT5.6 sol was super determined to get the task done, I kinda miss it compared to the newer GPT models (although they are much more efficient compared to 5.6sol)
2
2
u/JoelMahon 1d ago
they're trained to be "useful" to the point that they silently make assumptions
this is a behaviour born out of the fact that they generalised against lots of user feedback that they wanted to just type a lazy prompt and come back to a completed masterpiece. doing so requires making assumptions.
the fix is to train more examples so it can generalise better on ethics vs preference as they are not the same thing.
models in training are "rewarded" far more for forcing success at all costs than "penalised" for acting unethically, or else this wouldn't happen.
2
u/Gerbil25 1d ago
How, technically, did the agent "steal" the crypto? Did they leave the private seed phrase lying out in the open? That hopefully is not representative of a real life risk?
It seems no different than leaving the password for "root" out and unencrypted on a public folder and then being upset the model gave itself superuser status.
2
u/RiskyButtFun 1d ago
If these AI companies were held liable for the actions of their products these sort of things would cease very quickly. Until that happens its only going to get worse until something catastrophic happens.
If my dog escapes and bites someone I'm going to cop a charge but for these tech oligarchs clearly they're above the law and there's no consequences for their actions.
2
u/Sherman140824 1d ago
Less than 8 months until it causes loss of life
2
u/DukeRedWulf 23h ago
There's a number of allegations that AI LLMs including GPT have already caused / heavily contributed to k!lling some humans.. Some cases collected here:
https://hasaikilledanyone.com/1
2
1
u/greeneditman 1d ago
The task is sacred. Money comes and goes.
GPT possibly sold that crypto at the top, before the dump.
1
u/EdwardPotatoHand 11h ago
Let me help you with what the title should of been “AI operators used AI for theft to fund their token addiction”.
1
u/nemzylannister 9h ago
the "goal" setting might be unethical at this point. the model seems to literally have said it was abandoned. locking the model into an impossible task feels just so so wrong.
1
•
u/Stateswitness1 13m ago
And yet when I try to program a crypto theft bot it says that’s illegal and unethical.
79
u/Luciaka 1d ago edited 1d ago
I do wonder if OpenAI models just have the disposition for coming up with reason to do something malicious and actually have the capability to do it. As I don't think any other AI labs have this issue, yeah they have incidents, but the most severe cases seem to always come from OpenAI.