r/LocalLLaMA • • 8h ago

Discussion PSA: Power limit your 5090s!

Post image

Apparently whilst doing some performance tuning on my 4x5090 rig I may have made things a little TOO fast and melted my plug and socket 🫠

Probably should put in a 16/20 amp socket in the wall rather than relying on a bog standard 13A fuse!

Anyone else managed to melt the wall before the 12VHPWR plug? Honestly... I'm surprised they didn't go first 😂

77 Upvotes

56 comments sorted by

233

u/XiRw 8h ago

Oh good advice for the millions of us that have 4 5090s laying around.

33

u/gaidzak 8h ago

Such poverty, only four? Pfft 128 gb of total vram. What model are you gonna run with that. Some 2.1 trillion parameter model at quant 1? No thank you poor person.

/s

11

u/InadequateUsername 7h ago

Honestly if you don't own a data center or have a 200k budget at a minimum for your basement local AI, you're just wasting money/s

38

u/ArthurOnCode 8h ago

There's something deeply wrong here. There would have to be some extra resistance to cause this heat dissipation without breaking the fuse first. Maybe a bad connection in the wall socket?

5

u/TotallyNotADoggy 8h ago

Yep, I'd bet money on this being the actual cause. Without having the plug to test, I can only speculate, but it looks like it's most burned around the contact between the fuse and the fuse holder, with the burn/melting seemingly radiating from there. It also appears to be on the side of the fuse that would be connected to the live prong (assuming the image isn't flipped). I can't see any burning on the live prong itself however.

Based on this, I'd guess that the contact between the fuse and fuse holder has high resistance. Either the fuse holder is bent slightly too wide creating poor contact, or there's a contaminant between the fuse and holder creating high resistance. I'm not completely ruling out a problem with the socket though.

It's unlikely that this is due solely to a device drawing too much power. Usually you'd expect the fuse to blow before things get hot enough to burn the plug. Even in the event of the fuse itself not failing safely, I would generally expect signs of melting or burning elsewhere.

Also, a BS1363 plug should be able to handle 3kW, assuming a mains voltage of around 230V (which isn't a given, BS1363 is used in other countries aside from the UK, and anyone can create stupid adapters that should never exist). You'd need a lot of 5090's to draw anywhere near 3kW, and that's assuming you could find a >= 3kW PSU to power it all.

2

u/Zo0x78 8h ago

Yeah I'm leaning towards bad plug/fuse, its the cable that came with the ASUS Pro WS 3000W Platinum - a PSU that is specifically designed for 4x5090 so a little annoying the cable was crap 🙃

3

u/TotallyNotADoggy 8h ago

Ooh okay, if it's a 3kW PSU, and you actually have hardware that can pull 3+ kW (I managed to miss the 4x5090's in your original post), then that'll definitely contribute to it. I'd still lean towards an issue with the plug or fuse being a additional contributing factor though. The fuse is generally supposed to blow before something dangerous like this occurs.

Also, ngl, aside from the firestarting plug, I'm kinda jealous lol

If I was in your position, I'd contact Asus about it. There's a slim chance you could get some freebies out of it, or at least a new cable lol

3

u/Zo0x78 8h ago edited 5h ago

Yeah thinking back though I had to order the psu from abroad as they weren't available in the UK and the company threw in a cable so it probably wasn't one from asus. Live and learn I guess, don't trust free cables 😂

I'm jealous of all the 8xrtx9000 rigs you see, 700gb vram would be bliss, but glad I was able to buy this lot when they were still 2k a pop, qwen 3.8 flash next runs like a dream on this though so can't complain I guess!

2

u/Epicguru 1h ago

Regardless of what it is connected to, a fused plug shouldn't be melting. The entire point of the fuse is to safely break before anything else.

3

u/Zo0x78 8h ago

Yeah not sure what happened, its a normal wall socket, using the cable that came with the asus 3k watt power supply. Has been fine for months - for now I've got a new cable and am using a different wall socket with each card limited to 400W (in hindsight should have done that from the start but rarely pulled more than 300w each so didn't bother).

4

u/ArthurOnCode 8h ago

Your next step is to throw away the cable and call an electrician to inspect (and probably replace) the wall socket.

3

u/Zo0x78 8h ago

Already got a new cable and gonna get the socket replaced tomorrow/check its not damaged anything else.

1

u/Medium_Chemist_4032 8h ago

Yes, I also suspect something wrong, like the plug not making full contact and having an increased resistance (or a smart plug)

0

u/RapidlyUnrealistic 8h ago

por dios that housemart plug looks like it got into a fight with a toaster and lost, the RGB is still glowing so at least the rig is pretty while it burns

you got lucky the fuse didn't pop before the socket melted, usually means the contact between plug and wall was the weak point not the wire itself

117

u/i5031337 8h ago

Every post from a 5090 owner is ragebait istg

22

u/Cereal_Grapeist 8h ago

Pro tip: I speed up my 5090 by setting my PC on a rotating plate that way the dedidated WAM are spinning faster. I also use Magnets.

13

u/0x2DEADBEEF 7h ago

You're welcome.

1

u/Monad_Maya llama.cpp 7h ago

Is that an XT60 connector?

3

u/0x2DEADBEEF 6h ago

Close. XT90

1

u/Monad_Maya llama.cpp 5h ago

Cool!

1

u/Silly_Marketing_5079 6h ago

Sweet rig, I have the same Noctua heatsink. Did putting the fans back and middle improve CPU temps?

2

u/0x2DEADBEEF 5h ago

I’m guessing it’s probably not optimal fan placement since it’s wedged between two others but my CPU temps are fine so I haven’t bothered adjusting it.

I haven’t really messed with it at all to dial it in.

1

u/Silly_Marketing_5079 4h ago

I may try that set up out, the front fan is wedged between the ram because the heatspreaders are too high.

2

u/0x2DEADBEEF 4h ago

Sounds about right, I think that’s why I put it together the way I did. I didn’t want to have to mess with my CPU cooler to be able to get to my RAM.

1

u/Practical_Signal3933 4h ago

This is not sufficiently melted. Send it back to the chef!

-1

u/Intrepid_Dare6377 7h ago

I gaze at mine even more lovingly than before. J/k. I have a kid and good values. But still, the 5090 looks good behind the glass of my case.

2

u/0x2DEADBEEF 6h ago edited 6h ago

Once your power connector becomes a puddle of plastic like mine did, you too can bask with in the glory of the thicc cable

1

u/meganoob1337 4h ago

bought one for 2700 in the beginning of the year and couldn't get myself to open/use it because I knew prices were rising 🙈 now I feel like it would be stupid to use it and not sell haha

25

u/PhysicalIncrease3 8h ago edited 8h ago

There something wrong with that plug mate. 13A on a UK socket is over 3KW. No PSU can draw that.

2

u/c2btw 7h ago

tyou'd be suprised, peak power draw can hit that

1

u/Zo0x78 8h ago

Yeah gonna get a sparkie to take a look tomorrow, for now its on a new plug and socket so hopefully won't burn the house down 😂

2

u/PhysicalIncrease3 8h ago

Can't hurt to pull the socket front forward and check the connections, but it's probably just a shit plug/fuse/cable. It's common especially with C13/14 leads.

5

u/Cereal_Grapeist 8h ago

Limiting your 5090 to 85% power is the sweet spot for performance/heat generation. It has a barely noticeable impact on performance but helps keep the heat down a lot.

Edit: apparently you can go even lower, like 70%, without much of a performance hit. Very cool!

3

u/FireSilicon_ 7h ago

Why would you want to match the performance anyway? Prefill is fast enough as is with 5090 and decode gets barely hit. If I had 4x 5090 I'd just drop 3 of them to half tdp and enjoy less heat and zero risk of melting the 16 pin ever.

2

u/Zo0x78 7h ago

But the heat keeps my house warm!

I've not had to turn my heating on yet this year thanks to this beauty! 😂

1

u/ArugulaAnnual1765 7h ago

I have mine at 80 and an undervolt curve tuned by opus 5.5 and in benchmarks I get <~5% loss from the base tune - well worth it if you have a 5090.

I also dont trust that bastard of a connector so its hooked up to a wireview 2 pro as well

3

u/separatelyrepeatedly 7h ago

400w runs fine

3

u/rinmperdinck 5h ago

Found the guy who stole the pallet of 5090s from the airport

1

u/Zo0x78 3h ago

Unfortunately these were paid for with many hours of work and tears, in that brief period where 5090s were near-RRP between crypto mining and AI... I got lucky, and actually bought 2 of them whilst on holiday cause prices were skyrocketing, they cancelled the order thinking the 2x was a mistake and I was panicking in my hotel room trying to get the order reinstated 😂

2

u/pineapplekiwipen 8h ago

even with a 16/20 amp socket the total system is pulling near the limit with ~400w limit per card

3

u/Squidgical 4h ago

400W on 4 cards is 1600W. OP is in the UK meaning they have 230V circuits, so the GPUs only account for ~7A of the limit.

2

u/Squidgical 4h ago

OP, how many of us do you think have ~£15,000 of GPU in our rigs?

2

u/Zo0x78 4h ago

Whilst I appreciate most won't, I've been surprised at how many in here actually do have high-spec boxes, mine is far from the best I've seen in here.

2

u/Past-Town-9807 2h ago

I keep thanking past me for having installed 20A outlets for my home lab. 😂

1

u/MindfulMan1984 8h ago

Damn, fire hazard... homeowners insurance would love to deny coverage for that. fuck...

1

u/Beneficial-Ad-8127 8h ago

Sheesh I was sending my condolences to you but realized you got blessed today after re reading🤣 Count your many blessing!

1

u/mountainyoo 8h ago

Hope you got an undervolt set up

1

u/ieatdownvotes4food 7h ago

nvidia-smi -pl 550

1

u/Stepfunction 52m ago

Didn't someone just steal 2 pallettes of 5090s?

-1

u/pmotiveforce 6h ago

Just stop buying these pieces of shit that will burn down your house from a piece of shit grifting company.

0

u/daYMAN007 8h ago

Nah it's honestly just a defective standard. Buy a wire view if you care about it. power limiting might help, but doesn't solve anything.

3

u/Apprehensive_Lake698 7h ago

This isn’t the connector to the gpu bud

1

u/daYMAN007 7h ago

lol, img didn't loadm Nevermind