Hardware
Ranked by memory bandwidth, which is what actually governs token generation speed — not teraflops, and not core count.
| Accelerator | Memory○ | Bandwidth○ | Tensor FP16○ | TDP○ | Models fit◐ |
|---|---|---|---|---|---|
| B200 | 192 GB | 8000 | 2250 | 1000 | 1854 |
| Instinct MI325X | 256 GB | 6000 | 1307 | 1000 | 1861 |
| Instinct MI300X | 192 GB | 5300 | 1307 | 750 | 1854 |
| H200 SXM | 141 GB | 4800 | 989 | 700 | 1838 |
| H100 SXM 80GB | 80 GB | 3350 | 989 | 700 | 1809 |
| Instinct MI250X | 128 GB | 3277 | 383 | 560 | 1836 |
| A100 80GB | 80 GB | 2039 | 312 | 400 | 1809 |
| H100 PCIe 80GB | 80 GB | 2000 | 756 | 350 | 1809 |
| GeForce RTX 5090 | 32 GB | 1792 | 419 | 575 | 1663 |
| RTX PRO 6000 Blackwell Workstation Edition | 96 GB | 1792 | 504 | 600 | 1811 |
| RTX PRO 6000 Blackwell Max-Q Workstation Edition | 96 GB | 1792 | 441 | 300 | 1811 |
| GeForce RTX 5090 D | 32 GB | 1792 | 419 | 575 | 1663 |
| Instinct MI210 | 64 GB | 1638 | 181 | 300 | 1787 |
| A100 40GB | 40 GB | 1555 | 312 | 400 | 1694 |
| RTX PRO 5000 Blackwell | 48 GB | 1344 | 295 | 300 | 1748 |
| GeForce RTX 5090 D V2 | 24 GB | 1344 | 419 | 575 | 1557 |
| RTX PRO 5000 Blackwell | 72 GB | 1344 | 295 | 300 | 1800 |
| Instinct MI100 | 32 GB | 1229 | 185 | 300 | 1663 |
| Instinct MI60 | 32 GB | 1024 | — | 300 | 1663 |
| Radeon VII | 16 GB | 1024 | — | 300 | 1184 |
| Instinct MI50 32GB | 32 GB | 1024 | — | 300 | 1663 |
| GeForce RTX 3090 Ti | 24 GB | 1008 | 160 | 450 | 1557 |
| GeForce RTX 4090 D | 24 GB | 1008 | 294 | 425 | 1557 |
| GeForce RTX 4090 | 24 GB | 1008 | 330 | 450 | 1557 |
| GeForce RTX 5080 | 16 GB | 960 | 225 | 360 | 1184 |
| RTX 6000 Ada Generation | 48 GB | 960 | 364 | 300 | 1748 |
| RTX 5880 Ada Generation | 48 GB | 960 | 277 | 285 | 1748 |
| Radeon RX 7900 XTX | 24 GB | 960 | — | 355 | 1557 |
| GeForce RTX 3090 | 24 GB | 936 | 142 | 350 | 1557 |
| GeForce RTX 3080 Ti | 12 GB | 912 | 136 | 350 | 965 |
| GeForce RTX 3080 | 12 GB | 912 | 123 | 350 | 965 |
| Tesla V100 32GB | 32 GB | 900 | 125 | 300 | 1663 |
| Tesla V100 16GB | 16 GB | 900 | 125 | 300 | 1161 |
| GeForce RTX 5070 Ti | 16 GB | 896 | 176 | 300 | 1184 |
| GeForce RTX 5080 Laptop | 16 GB | 896 | — | 150 | 1184 |
| RTX PRO 4500 Blackwell | 32 GB | 896 | — | 200 | 1663 |
| GeForce RTX 5090 Laptop | 24 GB | 896 | — | 150 | 1557 |
| Titan V | 32 GB | 868 | 119 | 250 | 1663 |
| L40S | 48 GB | 864 | 366 | 350 | 1748 |
| Radeon Pro W7800 | 48 GB | 864 | — | 281 | 1748 |
| Radeon Pro W7900 | 48 GB | 864 | — | 295 | 1748 |
| Radeon Pro W7900 Dual Slot | 48 GB | 864 | — | 295 | 1748 |
| Apple M3 Ultraunified | 256 GB192 usable | 819 | — | — | 1857 |
| Apple M1 Ultraunified | 128 GB96 usable | 819 | — | — | 1811 |
| Apple M1 Ultraunified | 64 GB48 usable | 819 | — | — | 1770 |
| Apple M2 Ultraunified | 192 GB144 usable | 819 | — | — | 1843 |
| Apple M2 Ultraunified | 128 GB96 usable | 819 | — | — | 1811 |
| Apple M3 Ultraunified | 512 GB384 usable | 819 | — | — | 1874 |
| Apple M2 Ultraunified | 64 GB48 usable | 819 | — | — | 1770 |
| Apple M3 Ultraunified | 96 GB72 usable | 819 | — | — | 1809 |
| Radeon RX 7900 XT | 20 GB | 800 | — | 315 | 1430 |
| RTX A6000 | 48 GB | 768 | 155 | 300 | 1748 |
| RTX A5000 | 24 GB | 768 | 111 | 230 | 1557 |
| RTX A5500 | 24 GB | 768 | 136 | 230 | 1557 |
| GeForce RTX 3080 Ti | 20 GB | 760 | 136 | 350 | 1430 |
| GeForce RTX 3080 | 10 GB | 760 | 119 | 320 | 905 |
| GeForce RTX 4080 Super | 16 GB | 736 | 209 | 320 | 1184 |
| Tesla P100 16GB | 16 GB | 732 | — | 250 | 1161 |
| GeForce RTX 4080 | 16 GB | 717 | 195 | 320 | 1184 |
| A40 | 48 GB | 696 | 150 | 300 | 1748 |
| GeForce RTX 4070 Ti Super | 16 GB | 672 | 176 | 285 | 1184 |
| RTX PRO 4000 Blackwell | 24 GB | 672 | — | 140 | 1557 |
| Titan RTX | 24 GB | 672 | 131 | 280 | 1557 |
| GeForce RTX 5070 Ti Laptop | 12 GB | 672 | — | 115 | 965 |
| GeForce RTX 5070 | 12 GB | 672 | 123 | 250 | 965 |
| Titan V | 12 GB | 651 | 119 | 250 | 965 |
| Radeon RX 9070 | 16 GB | 640 | — | 220 | 1184 |
| RTX A4500 | 20 GB | 640 | 95 | 200 | 1428 |
| Radeon RX 9070 XT | 16 GB | 640 | — | 304 | 1184 |
| Radeon AI Pro R9600 | 32 GB | 640 | — | 150 | 1663 |
| Radeon AI Pro R9600D | 32 GB | 640 | — | 150 | 1663 |
| Radeon AI Pro R9700 | 32 GB | 640 | — | 300 | 1663 |
| Radeon AI Pro R9700S | 32 GB | 640 | — | 300 | 1663 |
| Radeon RX 7700 | 16 GB | 624 | — | 263 | 1184 |
| Radeon RX 7800 XT | 16 GB | 624 | — | 263 | 1184 |
| GeForce RTX 2080 Ti | 11 GB | 616 | 108 | 250 | 953 |
| Apple M5 Maxunified | 48 GB36 usable | 614 | — | — | 1693 |
| Apple M5 Maxunified | 128 GB96 usable | 614 | — | — | 1811 |
| Apple M5 Maxunified | 64 GB48 usable | 614 | — | — | 1770 |
| GeForce RTX 3060 Ti GDDR6X | 8 GB | 608 | 65 | 225 | 623 |
| GeForce RTX 3070 Ti | 8 GB | 608 | 87 | 290 | 623 |
| Arc Pro B70 32GB | 32 GB | 608 | 184 | 230 | 1663 |
| A10 | 24 GB | 600 | 125 | 150 | 1557 |
| Radeon RX 6950 XT | 16 GB | 576 | — | 335 | 1184 |
| Radeon Pro W7800 | 32 GB | 576 | — | 260 | 1663 |
| Radeon Pro W7700 | 16 GB | 576 | — | 190 | 1184 |
| RTX 5000 Ada Generation | 32 GB | 576 | 261 | 250 | 1663 |
| GeForce RTX 4090 Laptop | 16 GB | 576 | — | 150 | 1184 |
| Radeon RX 7900 GRE | 16 GB | 576 | — | 260 | 1184 |
| Arc A770 16GB | 16 GB | 560 | — | 225 | 1184 |
| Titan Xp | 12 GB | 548 | — | 250 | 965 |
| Apple M4 Maxunified | 128 GB96 usable | 546 | — | — | 1811 |
| Apple M4 Maxunified | 64 GB48 usable | 546 | — | — | 1770 |
| Apple M4 Maxunified | 48 GB36 usable | 546 | — | — | 1693 |
| Radeon RX 6900 XT | 16 GB | 512 | — | 300 | 1184 |
| Radeon RX 6800 | 16 GB | 512 | — | 250 | 1184 |
| GeForce RTX 3080 Ti Laptop | 16 GB | 512 | — | 150 | 1184 |
| Radeon RX 6800 XT | 16 GB | 512 | — | 300 | 1184 |
| Arc A580 8GB | 8 GB | 512 | — | 175 | 623 |
| Arc A750 8GB | 8 GB | 512 | — | 225 | 623 |
| Arc A770 8GB | 8 GB | 512 | — | 225 | 623 |
| Arc A770M 16GB | 16 GB | 512 | — | 150 | 1184 |
| Radeon Pro W6800 | 32 GB | 512 | — | 250 | 1663 |
| GeForce RTX 4070 Super | 12 GB | 504 | 142 | 220 | 965 |
| GeForce RTX 4070 | 12 GB | 504 | 117 | 200 | 965 |
| GeForce RTX 4070 Ti | 12 GB | 504 | 160 | 285 | 965 |
| GeForce RTX 2080 SUPER | 8 GB | 496 | 89 | 250 | 623 |
| GeForce GTX 1080 Ti | 11 GB | 484 | — | 250 | 953 |
| GeForce RTX 4070 GDDR6 | 12 GB | 480 | 117 | 200 | 965 |
| Apple M5 Maxunified | 36 GB27 usable | 461 | — | — | 1657 |
| Arc B580 12GB | 12 GB | 456 | — | 190 | 965 |
| Arc Pro B60 24GB | 24 GB | 456 | 99 | 200 | 1557 |
| GeForce RTX 2070 | 8 GB | 448 | 60 | 175 | 623 |
| GeForce RTX 2060 SUPER | 8 GB | 448 | 57 | 175 | 623 |
| GeForce RTX 2070 SUPER | 8 GB | 448 | 73 | 215 | 623 |
| GeForce RTX 2080 | 8 GB | 448 | 81 | 215 | 623 |
| GeForce RTX 3060 Ti | 8 GB | 448 | 65 | 200 | 623 |
| GeForce RTX 3070 | 8 GB | 448 | 81 | 220 | 623 |
| GeForce RTX 3080 Laptop | 16 GB | 448 | — | 150 | 1184 |
| GeForce RTX 5060 | 8 GB | 448 | 77 | 145 | 623 |
| GeForce RTX 5060 Ti | 16 GB | 448 | 95 | 180 | 1184 |
| GeForce RTX 5060 Ti | 8 GB | 448 | 95 | 180 | 623 |
| RTX A4000 | 16 GB | 448 | 77 | 140 | 1161 |
| GeForce RTX 4080 Laptop | 12 GB | 432 | — | 150 | 965 |
| Radeon RX 9070 GRE | 12 GB | 432 | — | 220 | 965 |
| Radeon RX 7700 XT | 12 GB | 432 | — | 245 | 965 |
| Radeon RX 6750 XT | 12 GB | 432 | — | 250 | 965 |
| RTX 4500 Ada Generation | 24 GB | 432 | 159 | 210 | 1557 |
| Apple M1 Maxunified | 32 GB24 usable | 410 | — | — | 1631 |
| Apple M2 Maxunified | 32 GB24 usable | 410 | — | — | 1631 |
| Apple M2 Maxunified | 64 GB48 usable | 410 | — | — | 1770 |
| Apple M2 Maxunified | 96 GB72 usable | 410 | — | — | 1809 |
| Apple M3 Maxunified | 128 GB96 usable | 410 | — | — | 1811 |
| Apple M3 Maxunified | 48 GB36 usable | 410 | — | — | 1693 |
| Apple M3 Maxunified | 64 GB48 usable | 410 | — | — | 1770 |
| Apple M4 Maxunified | 36 GB27 usable | 410 | — | — | 1657 |
| Apple M1 Maxunified | 64 GB48 usable | 410 | — | — | 1770 |
| GeForce RTX 5060 Laptop | 8 GB | 384 | — | 100 | 623 |
| Arc Pro A60 12GB | 12 GB | 384 | — | 130 | 965 |
| GeForce RTX 5070 Laptop | 8 GB | 384 | — | 100 | 623 |
| GeForce RTX 5050 Laptop | 8 GB | 384 | — | 100 | 623 |
| Radeon RX 6750 GRE | 12 GB | 384 | — | 230 | 965 |
| Radeon RX 6700 XT | 12 GB | 384 | — | 230 | 965 |
| Arc B570 10GB | 10 GB | 380 | — | 150 | 905 |
| GeForce RTX 3060 | 12 GB | 360 | 51 | 170 | 965 |
| RTX 4000 Ada Generation | 20 GB | 360 | 107 | 130 | 1428 |
| Tesla P40 | 24 GB | 346 | — | 250 | 1557 |
| GeForce RTX 3060 OEM | 6 GB | 336 | 57 | 185 | 369 |
| GeForce RTX 2060 | 6 GB | 336 | 52 | 160 | 369 |
| Arc A730M 12GB | 12 GB | 336 | — | 120 | 965 |
| GeForce RTX 2060 | 12 GB | 336 | 57 | 184 | 965 |
| Radeon RX 6750 GRE | 10 GB | 320 | — | 170 | 904 |
| GeForce RTX 5050 | 8 GB | 320 | 53 | 130 | 623 |
| Radeon RX 9060 XT | 16 GB | 320 | — | 160 | 1184 |
| Radeon RX 9060 XT | 8 GB | 320 | — | 150 | 617 |
| Tesla T4 | 16 GB | 320 | 65 | 70 | 1161 |
| Radeon RX 6700 | 10 GB | 320 | — | 175 | 904 |
| Apple M5 Prounified | 48 GB36 usable | 307 | — | — | 1693 |
| Apple M3 Maxunified | 96 GB72 usable | 307 | — | — | 1809 |
| Apple M3 Maxunified | 36 GB27 usable | 307 | — | — | 1657 |
| Apple M5 Prounified | 64 GB48 usable | 307 | — | — | 1770 |
| Apple M5 Prounified | 24 GB18 usable | 307 | — | — | 1424 |
| L4 | 24 GB | 300 | 121 | 72 | 1557 |
| GeForce RTX 4060 Ti | 8 GB | 288 | 88 | 160 | 623 |
| Radeon Pro W7500 | 8 GB | 288 | — | 70 | 617 |
| Radeon Pro W7600 | 8 GB | 288 | — | 130 | 617 |
| RTX PRO 2000 Blackwell | 16 GB | 288 | — | 70 | 1161 |
| RTX A2000 | 12 GB | 288 | 32 | 70 | 964 |
| GeForce RTX 4060 Ti | 16 GB | 288 | 88 | 165 | 1184 |
| RTX A2000 | 6 GB | 288 | 32 | 70 | 363 |
| Radeon RX 9060 | 8 GB | 288 | — | 132 | 617 |
| Radeon RX 7600 | 8 GB | 288 | — | 165 | 617 |
| Radeon RX 7600 XT | 16 GB | 288 | — | 190 | 1184 |
| Radeon RX 7650 GRE | 8 GB | 288 | — | 170 | 617 |
| RTX 4000 SFF Ada Generation | 20 GB | 280 | 77 | 70 | 1428 |
| Radeon RX 6650 XT | 8 GB | 280 | — | 180 | 617 |
| Apple M4 Prounified | 48 GB36 usable | 273 | — | — | 1693 |
| Apple M4 Prounified | 24 GB18 usable | 273 | — | — | 1424 |
| NVIDIA DGX Spark (GB10)unified | 128 GB | 273 | 125 | 140 | 1836 |
| Apple M4 Prounified | 64 GB48 usable | 273 | — | — | 1770 |
| GeForce RTX 4060 | 8 GB | 272 | 60 | 115 | 623 |
| AMD Ryzen AI Max 385 (Radeon 8050S)unified | 64 GB48 usable | 256 | — | 120 | 1770 |
| AMD Ryzen AI Max 385 (Radeon 8050S)unified | 32 GB24 usable | 256 | — | 120 | 1631 |
| AMD Ryzen AI Max+ 395 (Radeon 8060S)unified | 128 GB96 usable | 256 | — | 120 | 1811 |
| Radeon RX 6600 XT | 8 GB | 256 | — | 160 | 617 |
| AMD Ryzen AI Max+ 395 (Radeon 8060S)unified | 32 GB24 usable | 256 | — | 120 | 1631 |
| AMD Ryzen AI Max+ 395 (Radeon 8060S)unified | 64 GB48 usable | 256 | — | 120 | 1770 |
| AMD Ryzen AI Max 390 (Radeon 8050S)unified | 32 GB24 usable | 256 | — | 120 | 1631 |
| AMD Ryzen AI Max 390 (Radeon 8050S)unified | 64 GB48 usable | 256 | — | 120 | 1770 |
| Arc Pro A60M 8GB | 8 GB | 256 | — | 95 | 623 |
| GeForce RTX 3060 | 8 GB | 240 | 51 | 195 | 623 |
| Arc A530M 4GB | 4 GB | 224 | — | 95 | 242 |
| Radeon RX 6600 | 8 GB | 224 | — | 132 | 617 |
| Arc A550M 8GB | 8 GB | 224 | — | 80 | 623 |
| RTX 2000 Ada Generation | 16 GB | 224 | 48 | 70 | 1161 |
| Arc A570M 8GB | 8 GB | 224 | — | 95 | 623 |
| GeForce RTX 3050 | 8 GB | 224 | 36 | 130 | 623 |
| Arc Pro B50 16GB | 16 GB | 224 | 85 | 70 | 1184 |
| Arc A530M 8GB | 8 GB | 224 | — | 95 | 623 |
| Radeon Pro W6600 | 8 GB | 224 | — | 130 | 617 |
| GeForce RTX 3050 OEM | 8 GB | 224 | 32 | 130 | 623 |
| Apple M1 Prounified | 32 GB24 usable | 205 | — | — | 1631 |
| Apple M2 Prounified | 16 GB12 usable | 205 | — | — | 1063 |
| Apple M2 Prounified | 32 GB24 usable | 205 | — | — | 1631 |
| Apple M1 Prounified | 16 GB12 usable | 205 | — | — | 1063 |
| Arc Pro A50 6GB | 6 GB | 192 | — | 75 | 369 |
| RTX A1000 | 8 GB | 192 | 27 | 50 | 578 |
| Arc Pro A40 6GB | 6 GB | 192 | — | 50 | 369 |
| Arc A380 6GB | 6 GB | 186 | — | 75 | 369 |
| Radeon RX 7400 | 8 GB | 173 | — | 55 | 617 |
| GeForce RTX 3050 | 6 GB | 168 | 27 | 70 | 369 |
| Apple M5unified | 16 GB12 usable | 154 | — | — | 1063 |
| Apple M5unified | 12 GB9 usable | 154 | — | — | 904 |
| Apple M3 Prounified | 18 GB14 usable | 154 | — | — | 1142 |
| Apple M3 Prounified | 36 GB27 usable | 154 | — | — | 1657 |
| Apple M5unified | 32 GB24 usable | 154 | — | — | 1631 |
| Apple M5unified | 24 GB18 usable | 154 | — | — | 1424 |
| Radeon RX 6500 XT | 4 GB | 144 | — | 107 | 240 |
| Radeon RX 6500 XT | 8 GB | 144 | — | 113 | 617 |
| Radeon RX 6400 | 4 GB | 128 | — | 53 | 240 |
| Radeon Pro W6400 | 4 GB | 128 | — | 50 | 240 |
| Arc A310 4GB | 4 GB | 124 | — | 75 | 242 |
| Apple M4unified | 12 GB9 usable | 120 | — | — | 904 |
| Apple M4unified | 8 GB6 usable | 120 | — | — | 531 |
| Apple M4unified | 16 GB12 usable | 120 | — | — | 1063 |
| Apple M4unified | 24 GB18 usable | 120 | — | — | 1424 |
| Apple M4unified | 32 GB24 usable | 120 | — | — | 1631 |
| Arc A370M 4GB | 4 GB | 112 | — | 50 | 242 |
| Arc Pro A30M 4GB | 4 GB | 112 | — | 50 | 242 |
| Arc A350M 4GB | 4 GB | 112 | — | 35 | 242 |
| Apple M2unified | 8 GB6 usable | 102 | — | — | 531 |
| Apple M3unified | 24 GB18 usable | 102 | — | — | 1424 |
| Apple M3unified | 16 GB12 usable | 102 | — | — | 1063 |
| Apple M2unified | 16 GB12 usable | 102 | — | — | 1063 |
| Apple M3unified | 8 GB6 usable | 102 | — | — | 531 |
| Apple M2unified | 24 GB18 usable | 102 | — | — | 1424 |
| RTX A400 | 4 GB | 96 | 11 | 50 | 223 |
| Apple M1unified | 8 GB6 usable | 68 | — | — | 531 |
| Apple M1unified | 16 GB12 usable | 68 | — | — | 1063 |