Every GPU and what it can do.
Memory bandwidth sets each card's ceiling; the board shows who gets close.
Memory bandwidth sets each card's ceiling; the board shows who gets close.
Cards with results come first. The ceiling is theoretical: bandwidth divided by the 3.4 GB that Qwen3.8 Flash-Next Q4_K_M reads per token.
| GPU | Bandwidth | Rigs | Best generation | Typical share of ceiling | Ceiling, Qwen3.8 Flash-Next Q4_K_M |
|---|---|---|---|---|---|
| CMP 170HX | 1,493 GB/s | 1 | 99.0tok/s | 19% | 435tok/s max |
| RTX 3090 | 936 GB/s | 1 | 99.0tok/s | 19% | 272tok/s max |
| Instinct MI300X | 5,300 GB/s | 0 | No runs | 1,543tok/s max | |
| H100 | 3,350 GB/s | 0 | No runs | 975tok/s max | |
| A100 80GB | 2,039 GB/s | 0 | No runs | 594tok/s max | |
| RTX PRO 6000 Blackwell | 1,792 GB/s | 0 | No runs | 522tok/s max | |
| RTX 5090 | 1,792 GB/s | 0 | No runs | 522tok/s max | |
| A100 40GB | 1,555 GB/s | 0 | No runs | 453tok/s max | |
| RTX 4090 | 1,008 GB/s | 0 | No runs | 293tok/s max | |
| RTX 3090 Ti | 1,008 GB/s | 0 | No runs | 293tok/s max | |
| RTX 5080 | 960 GB/s | 0 | No runs | 279tok/s max | |
| RX 7900 XTX | 960 GB/s | 0 | No runs | 279tok/s max | |
| RTX 3080 Ti | 912 GB/s | 0 | No runs | 266tok/s max | |
| RTX 5070 Ti | 896 GB/s | 0 | No runs | 261tok/s max | |
| L40S | 864 GB/s | 0 | No runs | 252tok/s max | |
| Apple M3 Ultra | 819 GB/s | 0 | No runs | 238tok/s max | |
| RX 7900 XT | 800 GB/s | 0 | No runs | 233tok/s max | |
| Apple M2 Ultra | 800 GB/s | 0 | No runs | 233tok/s max | |
| Apple M1 Ultra | 800 GB/s | 0 | No runs | 233tok/s max | |
| RTX A6000 | 768 GB/s | 0 | No runs | 224tok/s max | |
| RTX 3080 | 760 GB/s | 0 | No runs | 221tok/s max | |
| Tesla P100 | 732 GB/s | 0 | No runs | 213tok/s max | |
| RTX 4080 | 717 GB/s | 0 | No runs | 209tok/s max | |
| RTX 5070 | 672 GB/s | 0 | No runs | 196tok/s max | |
| RTX 4070 Ti Super | 672 GB/s | 0 | No runs | 196tok/s max | |
| Apple M5 Max | 614 GB/s | 0 | No runs | 179tok/s max | |
| Apple M4 Max | 546 GB/s | 0 | No runs | 159tok/s max | |
| RTX 4070 Ti | 504 GB/s | 0 | No runs | 147tok/s max | |
| RTX 4070 | 504 GB/s | 0 | No runs | 147tok/s max | |
| Apple M5 Max 32-core GPU | 460 GB/s | 0 | No runs | 134tok/s max | |
| RTX 5060 Ti | 448 GB/s | 0 | No runs | 130tok/s max | |
| Apple M4 Max 32-core GPU | 410 GB/s | 0 | No runs | 119tok/s max | |
| Apple M3 Max | 400 GB/s | 0 | No runs | 116tok/s max | |
| Apple M2 Max | 400 GB/s | 0 | No runs | 116tok/s max | |
| Apple M1 Max | 400 GB/s | 0 | No runs | 116tok/s max | |
| RTX 3060 | 360 GB/s | 0 | No runs | 105tok/s max | |
| Tesla P40 | 347 GB/s | 0 | No runs | 101tok/s max | |
| Apple M5 Pro | 307 GB/s | 0 | No runs | 89.4tok/s max | |
| Apple M3 Max 30-core GPU | 300 GB/s | 0 | No runs | 87.3tok/s max | |
| RTX 4060 Ti | 288 GB/s | 0 | No runs | 83.8tok/s max | |
| Apple M4 Pro | 273 GB/s | 0 | No runs | 79.5tok/s max | |
| NVIDIA GB10 | 273 GB/s | 0 | No runs | 79.5tok/s max | |
| RTX 4060 | 272 GB/s | 0 | No runs | 79.2tok/s max | |
| Ryzen AI Max+ 395 | 256 GB/s | 0 | No runs | 74.5tok/s max | |
| Apple M5 | 153.6 GB/s | 0 | No runs | 44.7tok/s max | |
| Apple M4 | 120 GB/s | 0 | No runs | 34.9tok/s max |