The Local AI Stack

Your community-driven guide to running AI models on your own hardware

50
Submissions
11
Contributors
3
Hardware Vendors
8
Backends

Not sure what to run?

Tell us your VRAM and what you need โ€” we'll rank the community's real, working setups that fit your hardware, with honest tradeoffs. No guesswork, no server.

Find My Model โ†’

Community Highlights

Trending Models

Qwen3.8-Flash-Next

7 submissions

Up to 170 t/s decode

Qwen3.8-27B

6 submissions

Up to 120 t/s decode

Gemma 4 26B-A4B

2 submissions

Up to 73 t/s decode

Popular Backends

31
llama.cpp
5
llama-swap
5
Ollama

Fastest Setups

240 t/s

LFM2.5 8B-A1B

1x RX 6800 XT 16GB + Ryzen 7 7700X + 32GB DDR5-5200

by f41n ยท See in the matrix โ†’

221 t/s

LFM2.5 8B-A1B

1x RX 6800 XT 16GB + Ryzen 7 7700X + 32GB DDR5-5200

by f41n ยท See in the matrix โ†’

200 t/s

Swift 1.5 Flash Next

2 x R9700 32GB, Threadripper 3960X, 128GB DDR4

by digitalscream ยท See in the matrix โ†’

By the numbers

The whole dataset at a glance โ€” 50 submissions, updated from the forum thread every week. The matrix has three more views โ†’

VRAM distribution

VRAM distribution: 50 submissions across 6 tiers Bar chart of community submissions per VRAM tier. Under 16 GB: 1; 16โ€“23 GB: 25; 24โ€“31 GB: 3; 32โ€“47 GB: 0; 48โ€“95 GB: 10; 96+ GB: 9 0 10 20 30 Under 16 GB: 1 submission 1 Under 16 GB 16โ€“23 GB: 25 submissions 25 16โ€“23 GB 24โ€“31 GB: 3 submissions 3 24โ€“31 GB 32โ€“47 GB: 0 submissions 0 32โ€“47 GB 48โ€“95 GB: 10 submissions 10 48โ€“95 GB 96+ GB: 9 submissions 9 96+ GB VRAM tier

Backend usage

Backend usage: 50 submissions across 8 backends Donut chart of submissions per inference backend. llama.cpp: 31 (62%); llama-swap: 5 (10%); Ollama: 5 (10%); Strata: 4 (8%); vLLM: 2 (4%); Colibri: 1 (2%); Other: 1 (2%); Radiance: 1 (2%) llama.cpp: 31 submissions (62%) llama-swap: 5 submissions (10%) Ollama: 5 submissions (10%) Strata: 4 submissions (8%) vLLM: 2 submissions (4%) Colibri: 1 submission (2%) Other: 1 submission (2%) Radiance: 1 submission (2%) 50 submissions
  • llama.cpp 31
  • llama-swap 5
  • Ollama 5
  • Strata 4
  • vLLM 2
  • Colibri 1
  • Other 1
  • Radiance 1

How It Works

1. Submit

Head to the forum thread and use the Quick Submit template in Post #2. 30 seconds: tier, hardware, model, speed.

2. We Parse

The pipeline pulls the thread, parses every submission, and organizes it by VRAM tier, hardware vendor, backend, and model.

3. You Browse

Filter the matrix by your hardware. See exactly what the community runs, at what speed, in what context.

Related Resources