Yet another Hacker News

GPU-unlock: FP/BF16 instruction issue rate on Nvidia 3000's cards

20 points fragebogen 5 hours ago 12 comments

Tiberium 4 hours ago | parent

I've tested myself and the results are legit, this is incredible, even if a little risky (it relies on exploiting firmware): https://github.com/Zagitta/gpu-unlock/blob/main/docs/How%20A...

> Unfortunately for Nvidia the booter is vulnerable to a classic buffer overflow that writes straight into the stack which gives us the ability to execute code via Return Oriented Programming (ROP).

Repo for RTX 4090 and 4070 Ti Super: https://github.com/JustAResearcher/gpu-unlock-ada

jacquesm 31 minutes ago | parent

This one is real, but there are some caveats:

- you will need to run your card hotter than you might be comfortable with

- the real pay-off requires increasing the maximum power to 450W

- This is well above what the stock cooler was designed for so a bit more cooling would help

What's nice is that you can stack this mod on top of the p2p mod that has been out for a while, they don't seem to interfere with each other.

Tiberium 14 minutes ago | parent

> - you will need to run your card hotter than you might be comfortable with

Can you explain this one? On my GPU I didn't have to change the power limit. Obviously the card might spend more time at max power usage, but it's still within the limit.

jacquesm 8 minutes ago | parent

If you increase the power limit on the card you will be able to get much more mileage out of this modification. At 450W the difference is much, much larger but that is not something I'm comfortable with doing for longer runs. Note that the stock settings are already pretty extreme for that cooler, and that ambient temp and free airflow are going to be impacting the resulting GPU temp in a very direct way. I have my 3090's in an external box (with its own separate PCIe backplane) so they can talk p2p and have 'fresh air' rather than in the enclosure.

Tiberium 7 minutes ago | parent

Yes, but you can get up to a 40-50% increase in some workloads even with the same power limit, and some cards have it locked anyway. I just felt like it was a litle misleading for you to mention the power in your original post, since the unlock works regardless of that.

augment_me 12 minutes ago | parent

These number claim 240W locked power draw on 4070:

https://github.com/JustAResearcher/gpu-unlock-ada/releases/t...

jacquesm 11 minutes ago | parent

This is about the 30' series in principle, I don't have a 4070 here to test with (I do have a couple of modded 4090's) but I suspect that at 240W you are not getting the maximum out of that card. At the same time: that is probably pretty close to optimal in terms of tokens/W.

Scene_Cast2 25 minutes ago | parent

Really interesting.

On the other hand, this only applies to a limited set of workloads (training is often memory bandwidth bound for example).

I also wonder how this affects power consumption.

superkuh 22 minutes ago | parent

Uh, no. This headline is wrong. It's FP32 unlocked from being halved, not 16 bit ops. And to be clear, over the last week I've looked many places and I've not seen any reports of this unlock being applied to any 3xxx card except the 3090/3090ti series. Even if it might apply more generally in theory.

augment_me 15 minutes ago | parent

The repo has fp16 numbers.

Also this one for 4070s shows that its for fp8 as well:

https://github.com/JustAResearcher/gpu-unlock-ada/releases/t...

Tiberium 13 minutes ago | parent

The repo has a lot of reports of people applying it to other 3xxx card, see https://github.com/Zagitta/gpu-unlock/issues

There's at least 3080 Ti, 3070 Ti, 3070, 3060