Can AMD Radeon RX 7900 XTX run Qwen2.5-32B-Instruct?

This page was written and translated automatically by AI and checked by software against its sources, with no human editing. Specifications come from the sources listed; memory requirements and speeds are estimates calculated with published formulas. How we use AI

Fits By llama.cpp’s default load check, Qwen2.5-32B-Instruct fits on AMD Radeon RX 7900 XTX at Q4_K_M with a context of 8,192 tokens. One card needs 22.5 GiB available for “Fits”.

At Q4_K_M, the longest context that fits is 9,216 tokens.

“Fits” means llama.cpp's default --fit load check would place all layers on the GPU: our estimate of llama.cpp's buffers, plus its error margin, plus llama.cpp's 1 GiB per-card reserve, is within the memory counted as available. It does not promise that peak memory while generating fits. How results are decided

Values updated Oct 19, 2026 · Calculator

Run profile: llama.cpp, ROCm back end on Linux (AMD discrete GPU), all layers on the GPU, Flash Attention on, micro-batch (-ub) 512, F16 KV cache, context (-c) as shown; “Fits” is judged with llama-server’s default 4 parallel sequences (-np auto, unified KV cache) and “Doesn’t fit” with 1 sequence; idle GPU with no other programs using its memory.

Memory counted as available for “Fits” (C_low): 22.75 GiB per card = 24 GiB nominal − 16 MiB (nominal minus the total that llama.cpp reported for this GPU in public logs) − 1,216 MiB (the largest start-up and runtime use seen in this profile’s public logs), rounded down to a multiple of 64 MiB.

These results apply to the configuration seen in public logs, where llama.cpp reported 24,560 MiB of total memory for this GPU. Settings that reduce usable memory (for example ECC, where the card supports it) are not covered.

Memory used for “Doesn’t fit” (C_high): the full 24 GiB per card. Run profiles and their measurements

Memory needed by quantization and context

Available memory needed for “Fits” (X, GiB per card) and the result on AMD Radeon RX 7900 XTX.
Quantization	4,096	8,192	32,768
Q8_0	35.7 GiB Doesn't fit	36.7 GiB Doesn't fit	43.1 GiB Doesn't fit
Q6_K	28.1 GiB Tight	29.2 GiB Doesn't fit	35.5 GiB Doesn't fit
Q5_K_M	24.6 GiB Tight	25.7 GiB Tight	32.1 GiB Doesn't fit
Q4_K_M	21.4 GiB Fits	22.5 GiB Fits	28.8 GiB Doesn't fit
Q4_0	20.2 GiB Fits	21.3 GiB Fits	27.6 GiB Tight
Q3_K_M	17.7 GiB Fits	18.7 GiB Fits	25.1 GiB Tight
Q2_K	14.2 GiB Fits	15.2 GiB Fits	21.6 GiB Fits

No speed ceiling is given: the memory bandwidth of AMD Radeon RX 7900 XTX is not confirmed by two independent open sources.

Alternatives

Other 24 GB GPUs where Qwen2.5-32B-Instruct fits

NVIDIA GeForce RTX 3090 24 GB
NVIDIA GeForce RTX 3090 Ti 24 GB

Sources

Model page, with every quantization and its files: Qwen2.5-32B-Instruct
GPU page, with each specification and its sources: AMD Radeon RX 7900 XTX
Model files on Hugging Face (pinned revisions): Qwen/Qwen2.5-32B-Instruct@5ede1c9, Qwen/Qwen2.5-32B-Instruct-GGUF@a15e3cc
Memory size of AMD Radeon RX 7900 XTX from: Wikipedia: Radeon RX 7000 series (CC BY-SA 4.0), ROCm documentation gpu-specs.rst (MIT)
Methodology