GPUs
This page was written and translated automatically by AI and checked by software against its sources, with no human editing. Specifications come from the sources listed; memory requirements and speeds are estimates calculated with published formulas. How we use AI
Every entry below has its memory size confirmed by two independent open sources. Other specifications are shown only when two sources agree.
| GPU | Type | Memory | Bandwidth | Run profile | Models that fit (8,192 context) |
|---|---|---|---|---|---|
| AMD Instinct MI350X | Data-center GPU | 288 GB | 8,000 GB/s | ROCm on Linux (AMD) (only “Doesn’t fit” is judged) | — |
| AMD Instinct MI355X | Data-center GPU | 288 GB | 8,000 GB/s | ROCm on Linux (AMD) (only “Doesn’t fit” is judged) | — |
| AMD Instinct MI325X | Data-center GPU | 256 GB | 6,000 GB/s | ROCm on Linux (AMD) (only “Doesn’t fit” is judged) | — |
| AMD Instinct MI300X | Data-center GPU | 192 GB | 5,300 GB/s | ROCm on Linux (AMD) (only “Doesn’t fit” is judged) | — |
| NVIDIA B100 | Data-center GPU | 192 GB | 8,000 GB/s | CUDA (NVIDIA) (only “Doesn’t fit” is judged) | — |
| NVIDIA B200 | Data-center GPU | 180 GB | — | CUDA (NVIDIA) (only “Doesn’t fit” is judged) | — |
| NVIDIA H200 SXM | Data-center GPU | 141 GB | — | CUDA (NVIDIA) (only “Doesn’t fit” is judged) | — |
| NVIDIA H100 NVL | Data-center GPU | 94 GB | 3,938 GB/s | CUDA (NVIDIA) (only “Doesn’t fit” is judged) | — |
| NVIDIA H100 PCIe | Data-center GPU | 80 GB | — | CUDA (NVIDIA) (only “Doesn’t fit” is judged) | — |
| AMD Instinct MI210 | Data-center GPU | 64 GB | 1,638 GB/s | ROCm on Linux (AMD) (only “Doesn’t fit” is judged) | — |
| AMD Radeon PRO W7800 48GB | Workstation GPU | 48 GB | — | ROCm on Linux (AMD) (only “Doesn’t fit” is judged) | — |
| AMD Radeon PRO W7900 | Workstation GPU | 48 GB | — | ROCm on Linux (AMD) (only “Doesn’t fit” is judged) | — |
| NVIDIA L40 | Data-center GPU | 48 GB | 864 GB/s | CUDA (NVIDIA) (only “Doesn’t fit” is judged) | — |
| AMD Radeon AI PRO R9700 | Workstation GPU | 32 GB | — | ROCm on Linux (AMD) (only “Doesn’t fit” is judged) | — |
| AMD Radeon PRO W7800 32GB | Workstation GPU | 32 GB | — | ROCm on Linux (AMD) (only “Doesn’t fit” is judged) | — |
| NVIDIA GeForce RTX 5090 | Desktop GPU | 32 GB | 1,792 GB/s | CUDA (NVIDIA) | 311 |
| NVIDIA Tesla V100 32GB | Data-center GPU | 32 GB | 900 GB/s | CUDA (NVIDIA) (only “Doesn’t fit” is judged) | — |
| AMD Radeon RX 7900 XTX | Desktop GPU | 24 GB | — | ROCm on Linux (AMD) | 309 |
| NVIDIA GeForce RTX 3090 | Desktop GPU | 24 GB | 936 GB/s | CUDA (NVIDIA) | 305 |
| NVIDIA GeForce RTX 3090 Ti | Desktop GPU | 24 GB | 1,008 GB/s | CUDA (NVIDIA) | 305 |
| NVIDIA L4 | Data-center GPU | 24 GB | 300 GB/s | CUDA (NVIDIA) (only “Doesn’t fit” is judged) | — |
| NVIDIA Tesla P40 | Data-center GPU | 24 GB | 346 GB/s | CUDA (NVIDIA) (only “Doesn’t fit” is judged) | — |
| AMD Radeon RX 7900 XT | Desktop GPU | 20 GB | — | ROCm on Linux (AMD) | 295 |
| AMD Radeon RX 7800 XT | Desktop GPU | 16 GB | — | ROCm on Linux (AMD) | 281 |
| AMD Radeon RX 7900 GRE | Desktop GPU | 16 GB | — | ROCm on Linux (AMD) | 279 |
| NVIDIA GeForce RTX 4060 Ti 16GB | Desktop GPU | 16 GB | 288 GB/s | CUDA (NVIDIA) | 272 |
| NVIDIA GeForce RTX 4070 Ti SUPER | Desktop GPU | 16 GB | 672 GB/s | CUDA (NVIDIA) | 260 |
| NVIDIA GeForce RTX 4080 | Desktop GPU | 16 GB | 717 GB/s | CUDA (NVIDIA) | 260 |
| NVIDIA GeForce RTX 5060 Ti 16GB | Desktop GPU | 16 GB | — | CUDA (NVIDIA) | 260 |
| NVIDIA GeForce RTX 5070 Ti | Desktop GPU | 16 GB | 896 GB/s | CUDA (NVIDIA) | 260 |
| NVIDIA GeForce RTX 5080 | Desktop GPU | 16 GB | 960 GB/s | CUDA (NVIDIA) | 260 |
| NVIDIA P100 | Data-center GPU | 16 GB | 732 GB/s | CUDA (NVIDIA) (only “Doesn’t fit” is judged) | — |
| AMD Radeon RX 7700 XT | Desktop GPU | 12 GB | — | ROCm on Linux (AMD) | 234 |
| NVIDIA GeForce RTX 3080 Ti | Desktop GPU | 12 GB | 912 GB/s | CUDA (NVIDIA) | 223 |
| NVIDIA GeForce RTX 4070 | Desktop GPU | 12 GB | 504 GB/s | CUDA (NVIDIA) | 223 |
| NVIDIA GeForce RTX 4070 Ti | Desktop GPU | 12 GB | 504 GB/s | CUDA (NVIDIA) | 223 |
| NVIDIA GeForce RTX 5070 | Desktop GPU | 12 GB | 672 GB/s | CUDA (NVIDIA) | 223 |
| NVIDIA GeForce GTX 1080 Ti | Desktop GPU | 11 GB | 484 GB/s | CUDA (NVIDIA) | 208 |
| NVIDIA GeForce RTX 2080 Ti | Desktop GPU | 11 GB | 616 GB/s | CUDA (NVIDIA) | 208 |
| NVIDIA GeForce RTX 3080 10GB | Desktop GPU | 10 GB | 760 GB/s | CUDA (NVIDIA) | 189 |
| AMD Radeon Pro V520 | Data-center GPU | 8 GB | 512 GB/s | ROCm on Linux (AMD) (only “Doesn’t fit” is judged) | — |
| NVIDIA GeForce GTX 1070 | Desktop GPU | 8 GB | 256 GB/s | CUDA (NVIDIA) | 137 |
| NVIDIA GeForce GTX 1070 Ti | Desktop GPU | 8 GB | 256 GB/s | CUDA (NVIDIA) | 137 |
| NVIDIA GeForce GTX 1080 | Desktop GPU | 8 GB | 320 GB/s | CUDA (NVIDIA) | 137 |
| NVIDIA GeForce RTX 2060 SUPER | Desktop GPU | 8 GB | 448 GB/s | CUDA (NVIDIA) | 137 |
| NVIDIA GeForce RTX 2070 | Desktop GPU | 8 GB | 448 GB/s | CUDA (NVIDIA) | 137 |
| NVIDIA GeForce RTX 2080 | Desktop GPU | 8 GB | 448 GB/s | CUDA (NVIDIA) | 137 |
| NVIDIA GeForce RTX 3050 8GB | Desktop GPU | 8 GB | 224 GB/s | CUDA (NVIDIA) | 137 |
| NVIDIA GeForce RTX 3060 Ti | Desktop GPU | 8 GB | 448 GB/s | CUDA (NVIDIA) | 137 |
| NVIDIA GeForce RTX 3070 | Desktop GPU | 8 GB | 448 GB/s | CUDA (NVIDIA) | 137 |
| NVIDIA GeForce RTX 3070 Ti | Desktop GPU | 8 GB | 608 GB/s | CUDA (NVIDIA) | 137 |
| NVIDIA GeForce RTX 4060 | Desktop GPU | 8 GB | 272 GB/s | CUDA (NVIDIA) | 137 |
| NVIDIA GeForce RTX 5060 | Desktop GPU | 8 GB | 448 GB/s | CUDA (NVIDIA) | 156 |
| NVIDIA P4 | Data-center GPU | 8 GB | 192 GB/s | CUDA (NVIDIA) (only “Doesn’t fit” is judged) | — |
| NVIDIA GeForce GTX 1060 | Desktop GPU | 6 GB | 192 GB/s | CUDA (NVIDIA) | 53 |
| NVIDIA GeForce GTX 1660 | Desktop GPU | 6 GB | 192 GB/s | CUDA (NVIDIA) | 53 |
| NVIDIA GeForce GTX 1660 SUPER | Desktop GPU | 6 GB | 336 GB/s | CUDA (NVIDIA) | 53 |
| NVIDIA GeForce RTX 2060 6GB | Desktop GPU | 6 GB | 336 GB/s | CUDA (NVIDIA) | 53 |
| NVIDIA GeForce GTX 1050 Ti | Desktop GPU | 4 GB | 112 GB/s | CUDA (NVIDIA) | 0 |
| NVIDIA GeForce GTX 1650 | Desktop GPU | 4 GB | 128 GB/s | CUDA (NVIDIA) | 0 |