This page was written and translated automatically by AI and checked by software against its sources, with no human editing. Specifications come from the sources listed; memory requirements and speeds are estimates calculated with published formulas. How we use AI
UI-Mate-27B has 26.9B parameters. At Q4_K_M with a context of 8,192 tokens, a single GPU needs 19.1 GiB of available memory for the result to be “Fits”.
File sizes are the real files on Hugging Face. The other columns give X, the available memory one GPU needs for the result to be “Fits” (largest buffer estimate ÷ 0.95 + 1 GiB), with llama.cpp, Flash Attention, F16 KV cache, 4 parallel sequences and micro-batch 512. They do not depend on any particular GPU.
Quantization
File size
X at 4,096 context
X at 8,192 context
X at 32,768 context
X at 131,072 context
BF16
50.11 GiB (2 files)
52.6 GiB
52.9 GiB
54.5 GiB
61.0 GiB
Q8_0
26.70 GiB
29.2 GiB
29.4 GiB
31.1 GiB
37.6 GiB
Q6_K
21.63 GiB
24.1 GiB
24.4 GiB
26.0 GiB
32.5 GiB
Q5_K_M
19.10 GiB
21.6 GiB
21.9 GiB
23.5 GiB
30.0 GiB
Q4_K_M
16.33 GiB
18.9 GiB
19.1 GiB
20.8 GiB
27.3 GiB
Q4_0
15.00 GiB
17.5 GiB
17.7 GiB
19.4 GiB
25.9 GiB
IQ4_XS
14.28 GiB
16.7 GiB
17.0 GiB
18.6 GiB
25.2 GiB
Q3_K_M
13.38 GiB
15.9 GiB
16.2 GiB
17.8 GiB
24.3 GiB
IQ3_XXS
11.54 GiB
14.0 GiB
14.3 GiB
15.9 GiB
22.4 GiB
Q2_K
10.80 GiB
13.3 GiB
13.6 GiB
15.2 GiB
21.8 GiB
IQ2_M
9.90 GiB
12.3 GiB
12.5 GiB
14.2 GiB
20.7 GiB
IQ2_XXS
8.53 GiB
10.9 GiB
11.2 GiB
12.8 GiB
19.4 GiB
Smallest GPU for each quantization
For each quantization: the listed GPU with the least memory whose estimate is “Fits” on one card at 8,192 tokens of context. GPUs whose result is “Unknown” (no runtime profile, workstation and data-center cards, unified memory) are not considered.
Quantization
Smallest GPU that fits (one card)
Memory
BF16
No listed GPU fits on one card. For several cards, use the Calculator
—
Q8_0
No listed GPU fits on one card. For several cards, use the Calculator
“Fits” means llama.cpp's default --fit load check would place all layers on the GPU: our estimate of llama.cpp's buffers, plus its error margin, plus llama.cpp's 1 GiB per-card reserve, is within the memory counted as available. It does not promise that peak memory while generating fits. How results are decided
Context 8,192 tokens. Highest quantization that fits on one card, its longest context and the speed ceiling.
Unknown: memory configuration not established for this GPU; only “Doesn’t fit” would be shown.
Unknown
CUDA (NVIDIA): llama.cpp, CUDA back end (NVIDIA discrete GPU), all layers on the GPU, Flash Attention on, micro-batch (-ub) 512, F16 KV cache, context (-c) as shown; “Fits” is judged with llama-server’s default 4 parallel sequences (-np auto, unified KV cache) and “Doesn’t fit” with 1 sequence; idle GPU with no other programs using its memory.
ROCm on Linux (AMD): llama.cpp, ROCm back end on Linux (AMD discrete GPU), all layers on the GPU, Flash Attention on, micro-batch (-ub) 512, F16 KV cache, context (-c) as shown; “Fits” is judged with llama-server’s default 4 parallel sequences (-np auto, unified KV cache) and “Doesn’t fit” with 1 sequence; idle GPU with no other programs using its memory.
The speed ceiling is memory bandwidth divided by the bytes read per generated token. It is a theoretical upper bound; public llama.cpp benchmarks typically reach about half to four-fifths of it, and small models and mixture-of-experts models often less. Public benchmarks
Original weights
The model’s original weight files (safetensors, counted as described in the methodology) total 51.0 GiB. They are listed for reference only; every estimate on this page is for the GGUF files above, run with llama.cpp. tencent/UI-Mate-27B · Methodology