The Matrix
All community submissions, filterable and sortable
Showing 50 of 50 submissions
No setups match the current filters.
| Actions | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Under 16 GB | ||||||||||||
| Novasty β | RTX 3060 12GB | NVIDIA | Other | HauhauCS/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive | GGUF Q4_K_M | RAG | 14.8 | 196608 | Default | 170w | Hybrid (VRAM+RAM) | |
| 16β23 GB | ||||||||||||
| Burhan β | 1x RTX-4000 Ada SFF (20 GB) | NVIDIA | Ollama | Qwen3.6-27b-q4_KM | GGUF Q4_K_M | Document Q&A, Deep Research, Brainstorming, Coding | ~20-30 t/s | 52000 | q8_0 | 75 W | Hybrid (VRAM+RAM) | |
| Burhan β | 1x RTX-4000 Ada SFF (20 GB) | NVIDIA | Ollama | LFM2.5-thinking:1.2b-q8_0 | GGUF Q8_0 | Q&A, Brainstorming, Complex Tasks | ~35 t/s | 64000 | q8_0 | 75 W | Hybrid (VRAM+RAM) | |
| Burhan β | 1x RTX-4000 Ada SFF (20 GB) | NVIDIA | Ollama | Gemma4:26b-a4b-it-mtp-q4_K_M | GGUF Q4_K_M | Document Q&A, Deep Research, Creative Writing | ~25-30 t/s | 64000 | q8_0 | 75 W | Hybrid (VRAM+RAM) | |
| Burhan β | 1x RTX-4000 Ada SFF (20 GB) | NVIDIA | Ollama | Qwen3.5:9b-q8_0 | GGUF Q8_0 | Document Q&A, Deep Research, Creative Writing, Brainstorming, Planning | ~30 t/s | 128000 | q8_0 | 75 W | Hybrid (VRAM+RAM) | |
| s1ugh34d β | Radeon VII 16GB HBM2 (gfx906), 32-core EPYC host, 67GB RAM | AMD | llama-swap | samuelcardillo/Carnice-Qwen3.6-MoE-35B-A3B (I-Nano) | GGUF I-Nano | LLM assited coding / technical documentation | 54.4 | 65536 tokens native (NIAH hit YES at 65k) | quantized KV (observed near-flat VRAM growth w/ context) | ~150 W peak, 99% GPU util | ||
| s1ugh34d β | Radeon VII 16GB HBM2 (gfx906) | AMD | llama-swap | deadbydawn101/gemma-4-E4B-Agentic-Opus-Reasoning-GeminiCLI-GGUF | GGUF Q4_K_M | LLM assited coding / technical documentation / large repo generation | 86.7 | 49152 tokens (NIAH hit YES at 49k; 65k not probed) | quantized KV (VRAM ~flat: 4618MB @49k vs 4619MB @4k) | ~214 W peak, 99% GPU util | ||
| s1ugh34d β | Radeon VII 16GB HBM2 (gfx906) | AMD | llama-swap | unsloth/Qwen3.8-27B-GGUF (UD-Q2_K_XL) | GGUF UD-Q2_K_XL | LLM assited coding / technical documentation | 13.9 | 49152 tokens (NIAH hit YES at 49k) | quantized KV (VRAM 13770β13798MB across 8kβ32k, +28MB) | ~233 W peak, 100% GPU util | ||
| s1ugh34d β | Radeon VII 16GB HBM2 (gfx906) | AMD | llama-swap | MaziyarPanahi/DeepSeek-R1-0528-Qwen3-8B-GGUF | GGUF Q4_K_M | LLM assited coding / technical documentation | 84.4 | 16384 tokens (native server cap) | quantized KV (7327β7299 MB, flat) | ~268 W peak, 93% GPU util | ||
| s1ugh34d β | Radeon VII 16GB HBM2 (gfx906) | AMD | llama-swap | bartowski/Qwen_Qwen3.5-4B-GGUF | GGUF Q4_K_M | LLM assited coding / technical documentation | 96.4 | 65536 tokens (native cap; NIAH hit YES; 96k rejected, n_ctx=65536 enforced) | quantized KV (VRAM 5312MB @4k β 5312MB @65k, +0) | ~225 W peak, 99% GPU util | ||
| Yehya_Karout β | 1x RTX 5080 16GB + 64GB DDR5-6400, Core Ultra 9 285K | NVIDIA | Strata | Qwen3.8-Flash-Next (GSQ-RCO GGUF) | IQ3_XXS, int8 KV | agentic coding (opencode, pi), full 262K context | ~170 tok/s tg / ~3,000 pp at 4K, ~150 tg / ~3,950 pp at 180K, 152 tg / 3,800 pp at 250K. Real opencode agent session: ~165-170 tok/s avg | |||||
| f41n β | 1xRX 6800 XT 16GB + Ryzen 7 7700X + 32GB DDR5-5200 | llama.cpp | gpt-oss 20B | Q8_0 | coding, agents | 131 t/s generation, 2762 t/s prompt | ||||||
| f41n β | 1x RX 6800 XT 16GB + Ryzen 7 7700X + 32GB DDR5-5200 | llama.cpp | Nemotron3 Nano 4B | Q4_K_M | quick tasks, small helper model | 123 t/s generation, 2412 t/s prompt | ||||||
| f41n β | 1x RX 6800 XT 16GB + Ryzen 7 7700X + 32GB DDR5-5200 | llama.cpp | Gemma 4 26B-A4B | UD-IQ4_XS | daily chat, general | 73 t/s generation, 2278 t/s prompt | ||||||
| f41n β | 1x RX 6800 XT 16GB + Ryzen 7 7700X + 32GB DDR5-5200 | llama.cpp | Qwen3.8-27B | 3-bit GSQ-RCO (mixed IQ3_S etc.) + MTP head | coding, agents | \~51 t/s generation | ||||||
| f41n β | 1x RX 6800 XT 16GB + Ryzen 7 7700X + 32GB DDR5-5200 | llama.cpp | Qwen3.8-27B | 3-bit GSQ-RCO (mixed IQ3_S etc.) | coding, agents | 27 t/s generation, 441 t/s prompt | ||||||
| f41n β | 1x RX 6800 XT 16GB + Ryzen 7 7700X + 32GB DDR5-5200 | llama.cpp | Qwen3.8-Flash-Next (125B MoE) | GSQ-RCO Q2_0 (37.6 GB shard 1 + 29 GB n-gram table shard 2) | coding, agents | \~12 t/s generation (warm), \~12 t/s prompt, \~5 t/s first cold prompt | ||||||
| f41n β | 1x Radeon Instinct MI25 16GB HBM2 @ 170W (WX 9100 BIOS) + Ryzen 5 3600 + 32GB DDR4-3200 | AMD | llama.cpp | gpt-oss 20B | Q8_0 | coding, agents | 89 t/s generation, 858 t/s prompt | |||||
| f41n β | 1x Radeon Instinct MI25 16GB HBM2 @ 170W (WX 9100 BIOS) + Ryzen 5 3600 + 32GB DDR4-3200 | AMD | llama.cpp | Nemotron3 Nano 4B | Q4_K_M | quick tasks, small helper model | 77 t/s generation, 783 t/s prompt | |||||
| f41n β | 1x Radeon Instinct MI25 16GB HBM2 @ 170W (WX 9100 BIOS) + Ryzen 5 3600 + 32GB DDR4-3200 | AMD | llama.cpp | Qwen3.6 35B-A3B | IQ3_S (Unsloth UD) | coding, agents | 68 t/s generation, 586 t/s prompt | |||||
| f41n β | 1x Radeon Instinct MI25 16GB HBM2 @ 170W (WX 9100 BIOS) + Ryzen 5 3600 + 32GB DDR4-3200 | AMD | llama.cpp | Gemma 4 26B-A4B | UD-IQ4_XS | daily chat, general | 57 t/s generation, 678 t/s prompt | |||||
| f41n β | 1x Radeon Instinct MI25 16GB HBM2 @ 170W (WX 9100 BIOS) + Ryzen 5 3600 + 32GB DDR4-3200 | AMD | llama.cpp | Qwen3.8-27B | 3-bit GSQ-RCO (mixed IQ3_S etc.) + MTP head | coding, agents | \~23 t/s generation | |||||
| f41n β | 1x Radeon Instinct MI25 16GB HBM2 @ 170W (WX 9100 BIOS) + Ryzen 5 3600 + 32GB DDR4-3200 | AMD | llama.cpp | Qwen3.8-27B | 3-bit GSQ-RCO (mixed IQ3_S etc.) | coding, agents | 17 t/s generation, 134 t/s prompt | |||||
| f41n β | 1x Radeon Instinct MI25 16GB HBM2 @ 170W (WX 9100 BIOS) + Ryzen 5 3600 + 32GB DDR4-3200 | AMD | llama.cpp | Qwen3.8-Flash-Next (125B MoE) | GSQ-RCO Q2_0 (37.6 GB shard 1 + 29 GB n-gram table shard 2) | coding, agents | \~17 t/s generation (warm, 15.5-18), \~14 t/s prompt, \~10 t/s first cold prompt | |||||
| f41n β | 1x RX 6800 XT 16GB + Ryzen 7 7700X + 32GB DDR5-5200 | llama.cpp | LFM2.5 8B-A1B | Q4_K_M | quick tasks, small helper model | 221 t/s generation, 4718 t/s prompt | ||||||
| f41n β | 1x RX 6800 XT 16GB + Ryzen 7 7700X + 32GB DDR5-5200 | llama.cpp | LFM2.5 8B-A1B | Q4_K_M + DSpark draft (Q4_K_M) | quick tasks, small helper model | 240 t/s generation | ||||||
| 24β31 GB | ||||||||||||
| Kif β | 1x RTX 4090 24GB, Ryzen 9 5900X, 128GB DDR4-3600 CL18 (2x CMK64GX4M2D3600C18, 4x32GB double-rank, holding 3600 MT/s XMP), Gigabyte Aorus X570 Master, Samsung 990 Pro NVMe (PCIe 4.0) | NVIDIA | Strata | Qwen/Qwen3.8-Flash-Next (GGUF IQ3_S) | GGUF IQ3_S (ISTA-DASLab GSQ-RCO) | Agentic coding / agent harness via OpenAI-compatible API | 128 | 131072 configured; >=106,493 tokens observed in live agent sessions, no OOM, no coherence collapse | int8 (engine-native quantized KV) | 450W stock cap, 12W idle at P8; peak under load not measured | ||
| Kif β | 1x RTX 4090 24GB, Ryzen 9 5900X, 128GB DDR4-3600 CL18 (4x32GB double-rank), Gigabyte Aorus X570 Master, Samsung 990 Pro NVMe | NVIDIA | Strata | Qwen/Qwen3.8-Flash-Next (GGUF IQ3_S) | GGUF IQ3_S (ISTA-DASLab GSQ-RCO) | Agentic coding / agent harness via OpenAI-compatible API (+ vision via mmproj) | 107 | 262144 configured; 228,553 tokens observed in live sessions (65 s cold prefill), no OOM, no truncation, no coherence collapse | int8 | 450W stock cap, 12W idle at P8 | ||
| Kif β | 1x RTX 4090 24GB, Ryzen 9 5900X, 128GB DDR4-3600 CL18 (4x32GB double-rank), Gigabyte Aorus X570 Master, Samsung 990 Pro NVMe | NVIDIA | Strata | Qwen/Qwen3.8-Flash-Next (GGUF IQ3_S) | GGUF IQ3_S (ISTA-DASLab GSQ-RCO) | Agentic coding, 2 parallel streams via OpenAI-compatible API | 39.9 | 262144 per slot x 2 slots; 2x41k-token parallel fresh streams measured with KV in host RAM, plus a single 255k-token request (74 s cold prefill) | k8v4 + --kv-resident 32768 (32,768 positions per layer in VRAM, rest in host RAM; injected via config user_args since the engine declines it under WSL2) | 450W stock cap, 12W idle at P8 | ||
| 48β95 GB | ||||||||||||
| digitalscream β | 2 x R9700 | AMD | vLLM | d0xin/Swift-Qwen3.8-27B-Uncensored-FP8 | FP8 | Coding, Research (OpenCode) | 120t/s code, 70t/s chat | ~240k | FP16 | |||
| manoreken2 β | RTX 5090 + IceLake Xeon | NVIDIA | llama.cpp | nemotron-3-super:120b | ||||||||
| manoreken2 β | RTX 5090 + IceLake Xeon | NVIDIA | llama.cpp | GPT-OSS:120b | ||||||||
| PawingUnplug β | 2 x R9700 | AMD | llama.cpp | bartowski/Kwaipilot_KAT-Coder-V2.5-Dev-Q5_K_M.gguf, Qwen3-Embedding-8B-Q8_0.gguf, Qwen3.5-9B-The-Defiant-Fable-Uncnr-Heretic-NEO-MAX-MTP-Q8_0.gguf | Coding Judge, Research (OpenCode and own harness), Embeddings | ~50t/s chat | ~262k | FP16 | ~600w | |||
| PawingUnplug β | 2 x w7900 | AMD | llama.cpp | Qwen3.8-Flash-Next-Q4_K_M | Coding | ~35t/s chat | ~262k | FP16 | ~750w | |||
| digitalscream β | 2 x R9700 32GB, Threadripper 3960X, 128GB DDR4 | AMD | Radiance | Swift 1.5 Flash Next | IQ4 weights, FP8 PLE | General purpose / agentic coding / agent harness via OpenAI-compatible API | 200 | 262,144 configured; >=200,000 tokens observed in live agent sessions, no OOM, no coherence collapse | FP8 | 500-600W during inference, GPUs at 330W cap each | ||
| mattlach β | 1x AMD Instinct MI210 passed through to VM w. 32GB of RAM | AMD | llama.cpp | Gemma4:26B | Q8_0 | Daily chat, basic scripting, summarization | Pre-fill: ~2,200 tok/s. Eval ~75-80 tok/s | |||||
| mattlach β | 1x AMD Instinct MI210 passed through to VM w. 32GB of RAM | AMD | llama.cpp | Gemma4:12B | Q8_0 | Model dedicated to context compaction for other models. | Pre-fill: ~1,300 tok/s. Eval ~40-45 tok/s | |||||
| mattlach β | 1x AMD Instinct MI210 passed through to VM w. 32GB of RAM | AMD | llama.cpp | Gemma4:E4B | Q4_K_M | Model dedicated to small front end tasks (title generation, tags, etc.) | Pre-fill: ~2,500 - 3,000 tok/s. Eval ~85-90 tok/s | |||||
| mattlach β | 1x AMD Instinct MI210 passed through to VM w. 32GB of RAM | AMD | llama.cpp | Jina Embeddings V3 | F16 | Model dedicated for embeddings | ||||||
| 96+ GB | ||||||||||||
| manoreken2 β | RTX 5090 + IceLake Xeon | NVIDIA | llama.cpp | DeepSeek-R1:671b-0528_q8_0 | GGUF Q8_0 | 1 tokens/sec | 8192 | default | ||||
| manoreken2 β | RTX 5090 + IceLake Xeon | NVIDIA | llama.cpp | DeepSeek-V3.1 Terminus:671b_q8_0 | GGUF Q8_0 | 1 tokens/sec | ||||||
| manoreken2 β | RTX 5090 + IceLake Xeon | NVIDIA | llama.cpp | DeepSeek-V3.2:671b_UD-Q8_K_XL | 1 tokens/sec | |||||||
| manoreken2 β | RTX 5090 + IceLake Xeon | NVIDIA | llama.cpp | GLM-5.2-UD-Q6_K_XL | 1 tokens/sec | |||||||
| manoreken2 β | RTX 5090 + IceLake Xeon | NVIDIA | llama.cpp | Kimi-K2.6-UD-Q8_K_XL | 1 tokens/sec | |||||||
| manoreken2 β | RTX 5090 + IceLake Xeon | NVIDIA | llama.cpp | Qwen3:235b-a22b-thinking-2507-q4 | ||||||||
| manoreken2 β | 1x RTX5090 (32 GB) + IceLake Xeon CPU (1024 GB) | NVIDIA | llama.cpp | DeepSeek-V4-Pro-0813-UD_K_XL.gguf | translation of government journal from 481 BCE China | 16384 | ||||||
| SgtAwesomesauce β | 4x AMD v620 | AMD | vLLM | qwen3.8-flash-next | W4A16 | Agentic coding, chatbot tomfoolery | 68-92 | 245K | FP16/BF16 | ~900w, full system power from wall | ||
| manoreken2 β | Intel Arc B60 24GB x4 | Intel | llama.cpp | gemma-4-31B-it-BF16.gguf | BF16 | CtoJ translation of Chinese history θ³ζ²»ιι | 2.5 | |||||
| Unclassified (no VRAM tier) | ||||||||||||
| manoreken2 β | Colibri | DeepSeek-V4.1-Flash | translation of government journal from 481 BCE China | 12288 | ||||||||
| mattlach β | AMD Epyc 7543 (Milan, 32C/64T), LXC container with 128GB of RAM (no GPU) | AMD | Ollama | Poolside Laguna S2.1 (118B) | Q4_K_M | Coding, and long form reasoning beyond what Gemma4:26B is capable of | Pre-fill 60-62 tok/s. Evaluation 12-14 tok/s | |||||
The data at a glance
Three views of the 50 community submissions. The charts are generated from the data on every update β new submissions appear automatically.
VRAM distribution
Backend usage
- llama.cpp 31
- llama-swap 5
- Ollama 5
- Strata 4
- vLLM 2
- Colibri 1
- Other 1
- Radiance 1
Model size vs VRAM
Reading the matrix
Click any column header to sort (within each tier), or use the Sort dropdown for VRAM, model size, speed, or date across the whole table. Cells show the full value on hover. Speed is community-reported decode throughput; Context is the stable context tested; KV is the KV-cache precision; Mode is how the engine splits weights across VRAM/RAM/SSD. All data is community-reported β treat numbers as user-experienced, not lab-tested. Discussion lives in the forum thread β
The search box covers every field β model name, notes, hardware, backend, quant, use case. Backend and use case are multi-select: tick several, and they combine as any-of within the list and all-of across the other filters. Active filters appear as chips below the bar (each has an β), and the count line tells you exactly what you're seeing.
Click any row (or its β button) to open the full submission β complete hardware, the submitter's notes, a launch command, and a deep link straight to the forum post. The π Share button copies a link to this exact view β every filter, the sort, and the search included β so you can paste it in the forum and others land on the same table.
The Actions column: β opens the detail panel, tick the checkbox to add a row to a comparison, or hit π to copy a launch command for that setup.