THE CRUNCH
Consumer GeForce RTX cards share silicon with NVIDIA's professional GPUs, but firmware limits how fast they can issue instructions. Modders have now found ways to lift those restrictions on RTX 30-series and RTX 40-series cards, with reported gains of up to 2x in matrix multiplication workloads, the kind of maths that underpins running neural networks.
For the RTX 30-series "Ampere" cards, a researcher developed an unlock that effectively doubles the instruction issuing speed of FP/BF16 FMA into FP32 operations on GA102, GA103, GA104 and GA106-based cards. TechPowerUp reports that a GeForce RTX 3090 Ti with the unlock sees up to 80% faster BF16 matrix multiplication, a format crucial for running nearly all neural networks.
The RTX 40-series "Ada Lovelace" project goes further, demonstrating a 2x speedup in FP16 and BF16 tensor throughput with FP32 accumulation. In testing, a GeForce RTX 4090 measured 165.6 TeraFLOPS in FP16 tensor throughput with matrix accumulation, rising to 331.6 TeraFLOPS with the unlock applied. Those figures are normally the territory of higher-end professional cards such as the RTX 6000 "Ada Lovelace".
The implication, as the reporting frames it, is that NVIDIA artificially limited the AD102 silicon's performance in compute-sensitive matrix multiplication and accumulation, which affects how quickly large language models run on these cards. AD102, AD103 and AD106 are officially unlocked for the RTX 40-series project.
In numbers
RTX 4090 FP16 tensor throughput before and after unlock
165.6 TeraFLOPS
Before unlock
331.6 TeraFLOPS
After unlock
In testing, a GeForce RTX 4090 measured 165.6 TeraFLOPS in FP16 tensor throughput with matrix accumulation, rising to 331.6 TeraFLOPS with the unlock applied.
Figures from TechPowerUp, as quoted in this brief.