Run AI on your own computer.

Tell us what you want it to do. We'll pick the models, work out the hardware they need, and show the machines you can buy today.

Step 1:What do you want it to do?

Pick one or more. 3 selected

How quick should answers be?4.7 or more tokens a second (your number)

How much should it read at once?32,768 tokens

What's your budget?Optional. Models that only run on pricier machines stay in the list, dimmed.

Step 2:The models we'd use

Ranking every model for your jobs and checking which machines run them…

Step 3:What it takes, and machines you can buy

Working out the memory your plan needs…

Loading this page…

Run AI on your own computer.

Tell us what you want it to do. We'll pick the models, work out the hardware they need, and show the machines you can buy today.

Step 1:What do you want it to do?

Pick one or more. 3 selected

How quick should answers be?4.7 or more tokens a second (your number)

How much should it read at once?32,768 tokens

What's your budget?Optional. Models that only run on pricier machines stay in the list, dimmed.

Step 2:The models we'd use

Every model we track for each job, best score first. Pick any one a machine can run; we start you on the best of those that run together. Models too big for any desktop, and models too slow on every machine at your speed setting, stay in the list, with ways to use them in the cloud. Sizes include working space for what you chose in Step 1.

Code 19 models, ranked by SciCode

Size · Runs on

1Kimi K3SciCode 59.5 · kimi-k3 (own terms)1,631 GBCloud only

Ways to run it (3)

No desktop or off-the-shelf build we track has 1,630 GB for AI — the most any has is 1,066 GB (1× RTX PRO 6000 Blackwell Workstation Edition, 96GB · EPYC 9015 · 8× Server Premier KSM64R52BD4-128MB 128GB DDR5-6400 RDIMM).

kimi-k3 license (its own terms — read before commercial use).

Use it as a service, or rent a server that holds it:

OpenRouter — pay per use (opens in a new tab)$0.249 in · $11.936 out per 1M tokens
Use it as a service: OpenRouter runs Kimi K3 and charges per token. Its cheapest live route, Sail Research (fp4), charges $0.504 in · $8.10 out per 1M tokens, not the listed price.
Price checked 1 h ago· OpenRouter states the Hugging Face id moonshotai/Kimi-K3 · price source for OpenRouter — pay per use (opens in a new tab)
Together AI — pay per use

2GLM-5.3SciCode 59 · other (own terms)Not sized yet

Ways to run it (2)

Not sized yet, so it can't be checked against machines: GLM-5.3 is not fully onboarded yet (missing: Plan asks for quant "4bit" but no sized quantization of GLM-5.3 has that name).

other license (its own terms — read before commercial use).

Use it as a service, or rent a server that holds it:

OpenRouter — pay per use (opens in a new tab)$0.079 in · $1.00 out per 1M tokens
Use it as a service: OpenRouter runs GLM-5.3 and charges per token. Its cheapest live route, Inceptron (fp4), charges $0.55 in · $1.00 out per 1M tokens, not the listed price.

3Qwen3.8-2.4T-A95B (open Qwen3.8-Max)SciCode 54.1 · other (own terms)1,419 GBCloud only

Ways to run it (3)

No desktop or off-the-shelf build we track has 1,419 GB for AI — the most any has is 1,066 GB (1× RTX PRO 6000 Blackwell Workstation Edition, 96GB · EPYC 9015 · 8× Server Premier KSM64R52BD4-128MB 128GB DDR5-6400 RDIMM).

other license (its own terms — read before commercial use).

Use it as a service, or rent a server that holds it:

OpenRouter — pay per use (opens in a new tab)$2.00 in · $6.00 out per 1M tokens

4DeepSeek-V4.1-FlashSciCode 51.9 · MITNot sized yet

Ways to run it (2)

Not sized yet, so it can't be checked against machines: DeepSeek-V4.1-Flash is not fully onboarded yet (missing: no quantized repos listed (GGUF/MLX) — only the native checkpoint can be sized; Plan asks for quant "4bit" but no sized quantization of DeepSeek-V4.1-Flash has that name).

MIT license (permissive).

Use it as a service, or rent a server that holds it:

OpenRouter — pay per use (opens in a new tab)$0.0616 in · $0.144 out per 1M tokens
Use it as a service: OpenRouter runs DeepSeek-V4.1-Flash and charges per token (cheapest route OpenInference, fp4).

Notes, version and download (1)

Not confirmed to run together: the sizes of Qwen3.8-Flash-Next's and Qwen3-ASR 1.7B's projector files (mmproj) are not recorded, so they are not counted. 1× Radeon AI PRO R9700 32GB (32358-01-20G) · EPYC 9015 · 8× M321R8GA0BB0-CQK 64GB DDR5-4800 RDIMM (at least $6,328), the cheapest machine you can buy that holds the rest, has 212.43 GB left for them; the most room is 760.91 GB, on 1× RTX PRO 6000 Blackwell Workstation Edition, 96GB · EPYC 9015 · 8× Server Premier KSM64R52BD4-128MB 128GB DDR5-6400 RDIMM.

Show all 19 code models

How the Code list is ranked and sized

Every model we track for each job is listed, best on-task score first, unscored ones after with why. You can choose a model when a machine you can buy today runs it at your speed — within your budget when you set one. We start you on the best set that runs together on one machine you can buy: job by job, the best-scoring model that still leaves room for the jobs after it. A better model passed over says why; a job that can't fit alongside the others is marked, and the rest are chosen without it. A GGUF model used for Images or Speech also loads a projector file whose size is not recorded yet: it is not counted, so a set that needs one is marked not confirmed, with the room each machine leaves for it.

Ranked by SciCode (Artificial Analysis run) (% of subproblems solved, higher is better), run by Artificial Analysis. Code that has to run and pass tests, scored by an independent board on every coding model we track.

Checked against 89 machines: 40 ready-made (priced or not) and 49 builds put together from parts with a current price. A part with no current price is in no build, so a graphics card we have no price for does not count here. Judged at ≥ 4.7 tok/s.

Cloud options under a locked model: Up to 3, cheapest first: services that run this exact model and charge per token (matched on its Hugging Face id, or the model name the provider lists), then rented GPU servers whose GPU memory holds it at your settings — on-demand before spot, since a spot server can be taken back. Each at its latest observed price; prices checked in the last 36 hours come before older ones.

46 GPU server offers don't state GPU memory (Together AI, Azure, Nebius, Oracle OCI, Hyperbolic, GCP, Paperspace (DigitalOcean), DigitalOcean), so they were not considered.

A spot server costs less — Azure Standard_ND96isr_MI300X_v5 at $8.8704 an hour — but the provider can take it back mid-run, so on-demand servers are listed first.

Images

Qwen3.8-Flash-NextMMMU-Pro 79.8 — best a machine runs; Kimi K3 (80.5) is cloud only · 35 models

130 GBShow list

Speech

Qwen3-ASR 1.7B4.31% word errors on Open ASR Leaderboard — best a machine runs · 8 models

5.8 GBShow list

Together: 306 GB kept cached — 36 machines you can buy run all three, from at least $6,328.

Set a budget in Step 1 to see what fits it.

Show the work

Memory for all three together: estimate — Arithmetic that also depends on a tunable assumption, listed below.

Σ required_gb over every model in the stack

GLM-5.3-Flash 169.74 + Qwen3.8-Flash-Next 129.49 + Qwen3-ASR 1.7B 5.78 = 305 GB

Fast follow-ups

Keep your conversations cached?

The first reply to a long prompt is the slow one — the machine has to read all of it. Kept cached, every model stays loaded with its conversation, so follow-ups only read what is new and switching jobs reloads nothing.

First reply to a long promptWorking it out on the first machine below…

Follow-up repliesOnly new text is read

Memory for AI306 GB, 4.7 GB of it cache

Show the work

Step 3:What it takes, and machines you can buy

What it takes

Your setup needs 306 GB of memory for AI, and 4.7 or more tokens a second.

That includes 4.7 GB of conversation cache. A Mac with 512 GB or more has room for all three at once.

Qwen3.8-Flash-Next reads images through a separate projector file (mmproj); its size is not recorded yet, so the memory shown leaves it out.

Qwen3-ASR 1.7B hears audio through a separate projector file (mmproj); its size is not recorded yet, so the memory shown leaves it out.

256 GB512 GB

Step 2:The models we'd use

Ranking every model for your jobs and checking which machines run them…

Step 3:What it takes, and machines you can buy

Working out the memory your plan needs…

Step 2:The models we'd use

Code 19 models, ranked by SciCode

Size · Runs on

1Kimi K3SciCode 59.5 · kimi-k3 (own terms)1,631 GBCloud only

Ways to run it (3)

kimi-k3 license (its own terms — read before commercial use).

Use it as a service, or rent a server that holds it:

OpenRouter — pay per use (opens in a new tab)$0.249 in · $11.936 out per 1M tokens
Use it as a service: OpenRouter runs Kimi K3 and charges per token. Its cheapest live route, Sail Research (fp4), charges $0.504 in · $8.10 out per 1M tokens, not the listed price.
Price checked 1 h ago· OpenRouter states the Hugging Face id moonshotai/Kimi-K3 · price source for OpenRouter — pay per use (opens in a new tab)
Together AI — pay per use

2GLM-5.3SciCode 59 · other (own terms)Not sized yet

Ways to run it (2)

Not sized yet, so it can't be checked against machines: GLM-5.3 is not fully onboarded yet (missing: Plan asks for quant "4bit" but no sized quantization of GLM-5.3 has that name).

other license (its own terms — read before commercial use).

Use it as a service, or rent a server that holds it:

OpenRouter — pay per use (opens in a new tab)$0.079 in · $1.00 out per 1M tokens
Use it as a service: OpenRouter runs GLM-5.3 and charges per token. Its cheapest live route, Inceptron (fp4), charges $0.55 in · $1.00 out per 1M tokens, not the listed price.

3Qwen3.8-2.4T-A95B (open Qwen3.8-Max)SciCode 54.1 · other (own terms)1,419 GBCloud only

Ways to run it (3)

other license (its own terms — read before commercial use).

Use it as a service, or rent a server that holds it:

OpenRouter — pay per use (opens in a new tab)$2.00 in · $6.00 out per 1M tokens

4DeepSeek-V4.1-FlashSciCode 51.9 · MITNot sized yet

Ways to run it (2)

MIT license (permissive).

Use it as a service, or rent a server that holds it:

OpenRouter — pay per use (opens in a new tab)$0.0616 in · $0.144 out per 1M tokens
Use it as a service: OpenRouter runs DeepSeek-V4.1-Flash and charges per token (cheapest route OpenInference, fp4).

Notes, version and download (1)

Show all 19 code models

How the Code list is ranked and sized

46 GPU server offers don't state GPU memory (Together AI, Azure, Nebius, Oracle OCI, Hyperbolic, GCP, Paperspace (DigitalOcean), DigitalOcean), so they were not considered.

A spot server costs less — Azure Standard_ND96isr_MI300X_v5 at $8.8704 an hour — but the provider can take it back mid-run, so on-demand servers are listed first.

Images

Qwen3.8-Flash-NextMMMU-Pro 79.8 — best a machine runs; Kimi K3 (80.5) is cloud only · 35 models

130 GBShow list

Speech

Qwen3-ASR 1.7B4.31% word errors on Open ASR Leaderboard — best a machine runs · 8 models

5.8 GBShow list

Together: 306 GB kept cached — 36 machines you can buy run all three, from at least $6,328.

Set a budget in Step 1 to see what fits it.

Show the work

Memory for all three together: estimate — Arithmetic that also depends on a tunable assumption, listed below.

Σ required_gb over every model in the stack

GLM-5.3-Flash 169.74 + Qwen3.8-Flash-Next 129.49 + Qwen3-ASR 1.7B 5.78 = 305 GB

Fast follow-ups

Keep your conversations cached?

First reply, 32,768-token prompt56.7 secondsQwen3.8-Flash-Next (Images) on 1× Radeon AI PRO R9700 32GB (32358-01-20G) · EPYC 9015 · 8× M321R8GA0BB0-CQK 64GB DDR5-4800 RDIMM, the first machine below.

Show the work

First reply: estimate — Arithmetic that also depends on a tunable assumption, listed below.

ttft_s = prompt_tokens / prefill_tok_s

= 32,768 / 577.57 = 56.7 s

prompt_tokens: 32768 tokens
prefill_tok_s: 577.57 tok/s

No published prefill measurement for this hardware; scaled from peak compute (estimate).
Model q_dim unknown — attention FLOPs omitted, so long-prompt TTFT is understated.

First reply to a long promptWorking it out on the first machine below…

Follow-up repliesOnly new text is read

Memory for AI306 GB, 4.7 GB of it cache

Show the work

Step 3:What it takes, and machines you can buy

What it takes

Your setup needs 306 GB of memory for AI, and 4.7 or more tokens a second.

That includes 4.7 GB of conversation cache. A Mac with 512 GB or more has room for all three at once.

Qwen3.8-Flash-Next reads images through a separate projector file (mmproj); its size is not recorded yet, so the memory shown leaves it out.

Qwen3-ASR 1.7B hears audio through a separate projector file (mmproj); its size is not recorded yet, so the memory shown leaves it out.

256 GB512 GB

Show the work

SciCode (Artificial Analysis run): 51.6 (% of subproblems solved). The best we track on it is Kimi K3 at 59.5; 19 models for this job have a score on it. Score source (opens in a new tab)

Memory it needs: estimate — Arithmetic that also depends on a tunable assumption, listed below.

required_gb = weights_gb + kv_cache_gb + overhead_gb

required_gb = 156.82 + 0.37 + 12.55 = 169.74 GB

weights_gb: 156.8221112 GB — real file size
kv_cache_gb: 0.369098752 GB — kv formula
overhead_gb: 12.545768896 GB — overhead fraction

Real file size used instead of the params × bits formula (SPEC §8.1).
11 layers keep a full KV cache, 34 linear-attention/SSM layers keep only a constant-size state (treated as negligible).
Runtime buffers, activations, and allocator slack approximated as a fraction of weights.

Weights: observed — Measured or published, passed through unchanged — not computed here.

weights_gb = real file size (Hugging Face API)

weights_gb = 156.82 GB (UD-IQ4_XS)

file_size_gb: 156.8221112 GB — https://huggingface.co/unsloth/GLM-5.3-Flash-GGUF/tree/main

Real file size used instead of the params × bits formula (SPEC §8.1).

Conversation cache: derived — Arithmetic on observed inputs only. No tunable assumption enters it.

kv_cache_gb = Σ over full-attention layers of per-token cache width × kv_bytes (from tensor shapes) × context / 1e9

kv_cache_gb = 5,632 elements × 2 × 32,768 / 1e9 = 0.37 GB

num_layers: 45 — model config
context: 32768 tokens — Plan context setting
kv_bytes: 2 bytes — Settings (fp16 cache)

11 layers keep a full KV cache, 34 linear-attention/SSM layers keep only a constant-size state (treated as negligible).

Runtime overhead: estimate — Arithmetic that also depends on a tunable assumption, listed below.

overhead_gb = weights_gb × overhead_fraction

overhead_gb = 156.82 × 0.08 = 12.55 GB

overhead_fraction: 0.08 — Settings (SPEC §8.1 default 0.08)

Runtime buffers, activations, and allocator slack approximated as a fraction of weights.

Runs on 52 machines you can buy today, from at least $4,061 (CPU only · EPYC 9015 · 8× M321R4GA3BB6-CQK 32GB DDR5-4800 RDIMM). 1 Mac and 1 mini PC also run it but have no current price.

1	Kimi K3	59.5 on SciCode (opens in a new tab)	UD-Q4_K_XL · 32,768 tokens	1,508.7 + 0.9 + 120.7 = 1,630.3	0 (0 priced)	None with a current price
2	GLM-5.3	59 on SciCode (opens in a new tab)	GLM-5.3 is not fully onboarded yet (missing: Plan asks for quant "4bit" but no sized quantization of GLM-5.3 has that name).	Not sized	Not judged	None with a current price
3	Qwen3.8-2.4T-A95B (open Qwen3.8-Max)	54.1 on SciCode (opens in a new tab)	UD-IQ4_XS · 32,768 tokens	1,310.9 + 3.1 + 104.9 = 1,418.8	0 (0 priced)	None with a current price
4	DeepSeek-V4.1-Flash	51.9 on SciCode (opens in a new tab)	DeepSeek-V4.1-Flash is not fully onboarded yet (missing: no quantized repos listed (GGUF/MLX) — only the native checkpoint can be sized; Plan asks for quant "4bit" but no sized quantization of DeepSeek-V4.1-Flash has that name).	Not sized	Not judged	None with a current price
5	GLM-5.3-Flash	51.6 on SciCode (opens in a new tab)	UD-IQ4_XS · 32,768 tokens	156.8 + 0.4 + 12.5 = 169.7	54 (52 priced)	at least $4,061 — CPU only · EPYC 9015 · 8× M321R4GA3BB6-CQK 32GB DDR5-4800 RDIMM, 11.7 tok/s
6	DeepSeek-V4-Pro-0813	51 on SciCode (opens in a new tab)	UD-Q4_K_XL · 32,768 tokens	849.7 + 2 + 68 = 919.7	9 (9 priced)	at least $33,772 — CPU only · EPYC 9015 · 8× Server Premier KSM64R52BD4-128MB 128GB DDR5-6400 RDIMM, 5.7 tok/s
7	Qwen3.8-Flash-Next	50.6 on SciCode (opens in a new tab)	Q4_K_M · 32,768 tokens	119.2 + 0.8 + 9.5 = 129.5	49 (48 priced)	at least $5,665 — 1× Radeon AI PRO R9700 32GB (32358-01-20G) · EPYC 9015 · 8× M321R4GA3BB6-CQK 32GB DDR5-4800 RDIMM, 4.9 tok/s
8	DeepSeek-V4-Flash-0731	50.3 on SciCode (opens in a new tab)	UD-Q4_K_XL · 32,768 tokens	155.1 + 1.4 + 12.4 = 168.9	54 (52 priced)	at least $4,061 — CPU only · EPYC 9015 · 8× M321R4GA3BB6-CQK 32GB DDR5-4800 RDIMM, 11.8 tok/s
9	MiniMax-M2.7	50.1 on SciCode (opens in a new tab)	Q4_K_M · 32,768 tokens	138.3 + 8.3 + 11.1 = 157.7	54 (52 priced)	at least $4,061 — CPU only · EPYC 9015 · 8× M321R4GA3BB6-CQK 32GB DDR5-4800 RDIMM, 8.9 tok/s
10	Kimi-K2.7-Code	47.8 on SciCode (opens in a new tab)	UD-Q4_K_XL · 32,768 tokens	583.7 + 2.3 + 46.7 = 632.7	19 (19 priced)	at least $31,363 — CPU only · EPYC 9015 · 8× Server Premier KSM64R52BD4-96MC 96GB DDR5-6400 RDIMM, 7.2 tok/s
11	MiniMax-M3	47.1 on SciCode (opens in a new tab)	Q4_K_M · 32,768 tokens	261.3 + 4 + 20.9 = 286.2	42 (41 priced)	at least $4,723 — CPU only · EPYC 9015 · 8× M321R8GA0BB0-CQK 64GB DDR5-4800 RDIMM, 7.4 tok/s
12	Qwen3.8-27B	46.6 on SciCode (opens in a new tab)	Q4_K_M · 32,768 tokens	16.8 + 2.1 + 1.3 = 20.3	86 (79 priced)	at least $1,269 — Framework Desktop DIY Edition (Ryzen AI Max 385), 32GB, 10.6 tok/s
13	gemma-4-31B-it	45.5 on SciCode (opens in a new tab)	Q4_K_M · 32,768 tokens	18.3 + 2.2 + 1.5 = 22	86 (79 priced)	at least $1,269 — Framework Desktop DIY Edition (Ryzen AI Max 385), 32GB, 9.8 tok/s
14	Muse Glimmer 30B	44.9 on SciCode (opens in a new tab)	Q4_K_M · 32,768 tokens	17.3 + 0.5 + 1.4 = 19.2	86 (79 priced)	at least $1,269 — Framework Desktop DIY Edition (Ryzen AI Max 385), 32GB, 11.3 tok/s
15	Qwen3.6-27B	42.8 on SciCode (opens in a new tab)	Q4_K_M · 32,768 tokens	16.5 + 2.1 + 1.3 = 20	86 (79 priced)	at least $1,269 — Framework Desktop DIY Edition (Ryzen AI Max 385), 32GB, 10.7 tok/s
16	Mistral-Medium-3.5-128B	40.2 on SciCode (opens in a new tab)	Q4_K_M · 32,768 tokens	74.9 + 11.8 + 6 = 92.7	19 (18 priced)	at least $5,399 — Apple Mac Studio (2026), M5 Max 18-core CPU / 40-core GPU, 128GB, 5.6 tok/s
17	Qwen3.6-35B-A3B	36.6 on SciCode (opens in a new tab)	Q4_K_M · 32,768 tokens	20.4 + 0.7 + 1.6 = 22.7	86 (79 priced)	at least $1,269 — Framework Desktop DIY Edition (Ryzen AI Max 385), 32GB, 59.6 tok/s
18	Qwen3-Coder-Next	36.2 on SciCode (opens in a new tab)	Q4_K_M · 32,768 tokens	48.5 + 0.8 + 3.9 = 53.2	78 (72 priced)	at least $1,959 — Framework Desktop DIY Edition (Ryzen AI Max+ 395), 64GB, 48.2 tok/s
19	Devstral-Small-2-24B-Instruct-2512	32.4 on SciCode (opens in a new tab)	Q4_K_M · 32,768 tokens	14.3 + 5.4 + 1.1 = 20.8	86 (79 priced)	at least $1,269 — Framework Desktop DIY Edition (Ryzen AI Max 385), 32GB, 10 tok/s

How we worked this out

Each model needs its weights at the version shown in Step 2, a conversation cache for the context it is sized at, and the runtime's overhead.

GLM-5.3-Flash (Code), sized for 32,768 tokens

Total: estimate — Arithmetic that also depends on a tunable assumption, listed below.

required_gb = weights_gb + kv_cache_gb + overhead_gb

required_gb = 156.82 + 0.37 + 12.55 = 169.74 GB

weights_gb: 156.8221112 GB — real file size
kv_cache_gb: 0.369098752 GB — kv formula
overhead_gb: 12.545768896 GB — overhead fraction

Real file size used instead of the params × bits formula (SPEC §8.1).
11 layers keep a full KV cache, 34 linear-attention/SSM layers keep only a constant-size state (treated as negligible).
Runtime buffers, activations, and allocator slack approximated as a fraction of weights.

Qwen3.8-Flash-Next (Images), sized for 32,768 tokens

Total: estimate — Arithmetic that also depends on a tunable assumption, listed below.

required_gb = weights_gb + kv_cache_gb + overhead_gb

required_gb = 119.15 + 0.81 + 9.53 = 129.49 GB

weights_gb: 119.15072224 GB — real file size
kv_cache_gb: 0.805306368 GB — kv formula
overhead_gb: 9.5320577792 GB — overhead fraction

Real file size used instead of the params × bits formula (SPEC §8.1).
12 layers keep a full KV cache, 36 linear-attention/SSM layers keep only a constant-size state (treated as negligible).
Runtime buffers, activations, and allocator slack approximated as a fraction of weights.

Qwen3-ASR 1.7B (Speech), sized for 30,000 tokens

Total: estimate — Arithmetic that also depends on a tunable assumption, listed below.

required_gb = weights_gb + kv_cache_gb + overhead_gb

required_gb = 2.17 + 3.44 + 0.17 = 5.78 GB

weights_gb: 2.165034944 GB — real file size
kv_cache_gb: 3.44064 GB — kv formula
overhead_gb: 0.17320279551999998 GB — overhead fraction

Real file size used instead of the params × bits formula (SPEC §8.1).
28 layers keep a full KV cache.
Runtime buffers, activations, and allocator slack approximated as a fraction of weights.

Every model kept cached: estimate — Arithmetic that also depends on a tunable assumption, listed below.

Σ required_gb over every model in the stack

GLM-5.3-Flash 169.74 + Qwen3.8-Flash-Next 129.49 + Qwen3-ASR 1.7B 5.78 = 305 GB

GLM-5.3-Flash: 169.736978848 GB
Qwen3.8-Flash-Next: 129.48808638719998 GB
Qwen3-ASR 1.7B: 5.77887773952 GB

All models loaded at once (Keep all resident).

Conversation cache: derived — Arithmetic on observed inputs only. No tunable assumption enters it.

cache_gb = Σ kv_cache_gb

GLM-5.3-Flash 0.37 + Qwen3.8-Flash-Next 0.81 + Qwen3-ASR 1.7B 3.44 = 4.62 GB

GLM-5.3-Flash: 0.369098752 GB — lib/perf kv_cache_gb
Qwen3.8-Flash-Next: 0.805306368 GB — lib/perf kv_cache_gb
Qwen3-ASR 1.7B: 3.44064 GB — lib/perf kv_cache_gb

Every model stays loaded with its own conversation cache (Keep cached).

16 GB Mac (Apple Mac mini (2026), M6, 16GB): derived — Arithmetic on observed inputs only. No tunable assumption enters it.

available_gb = Σ usable_gb over memory pools

unified memory 12 = 12 GB

unified memory.usable_gb: 12 GB — 16 GB physical

Usable memory is below physical memory for at least one pool (OS/driver reservation; editable in Settings).

24 GB Mac (Apple Mac mini (2026), M5 Pro 15-core CPU / 16-core GPU, 24GB, and 1 more of this size): derived — Arithmetic on observed inputs only. No tunable assumption enters it.

available_gb = Σ usable_gb over memory pools

unified memory 18 = 18 GB

unified memory.usable_gb: 18 GB — 24 GB physical

Usable memory is below physical memory for at least one pool (OS/driver reservation; editable in Settings).

32 GB Mac (Apple Mac mini (2026), M6, 32GB): derived — Arithmetic on observed inputs only. No tunable assumption enters it.

available_gb = Σ usable_gb over memory pools

unified memory 24 = 24 GB

unified memory.usable_gb: 24 GB — 32 GB physical

Usable memory is below physical memory for at least one pool (OS/driver reservation; editable in Settings).

36 GB Mac (Apple Mac Studio (2026), M5 Max 18-core CPU / 32-core GPU, 36GB): derived — Arithmetic on observed inputs only. No tunable assumption enters it.

available_gb = Σ usable_gb over memory pools

unified memory 27 = 27 GB

unified memory.usable_gb: 27 GB — 36 GB physical

Usable memory is below physical memory for at least one pool (OS/driver reservation; editable in Settings).

48 GB Mac (Apple Mac Studio (2026), M5 Max 18-core CPU / 40-core GPU, 48GB, and 1 more of this size): derived — Arithmetic on observed inputs only. No tunable assumption enters it.

available_gb = Σ usable_gb over memory pools

unified memory 36 = 36 GB

unified memory.usable_gb: 36 GB — 48 GB physical

Usable memory is below physical memory for at least one pool (OS/driver reservation; editable in Settings).

64 GB Mac (Apple Mac Studio (2026), M5 Max 18-core CPU / 40-core GPU, 64GB, and 2 more of this size): derived — Arithmetic on observed inputs only. No tunable assumption enters it.

available_gb = Σ usable_gb over memory pools

unified memory 48 = 48 GB

unified memory.usable_gb: 48 GB — 64 GB physical

Usable memory is below physical memory for at least one pool (OS/driver reservation; editable in Settings).

96 GB Mac (Apple Mac Studio (2025), M3 Ultra 28-core CPU / 60-core GPU, 96GB, and 3 more of this size): derived — Arithmetic on observed inputs only. No tunable assumption enters it.

available_gb = Σ usable_gb over memory pools

unified memory 72 = 72 GB

unified memory.usable_gb: 72 GB — 96 GB physical

Usable memory is below physical memory for at least one pool (OS/driver reservation; editable in Settings).

128 GB Mac (Apple Mac Studio (2026), M5 Max 18-core CPU / 40-core GPU, 128GB): derived — Arithmetic on observed inputs only. No tunable assumption enters it.

available_gb = Σ usable_gb over memory pools

unified memory 96 = 96 GB

unified memory.usable_gb: 96 GB — 128 GB physical

Usable memory is below physical memory for at least one pool (OS/driver reservation; editable in Settings).

256 GB Mac (Apple Mac Studio (2026), M5 Ultra 30-core CPU / 64-core GPU, 256GB, and 1 more of this size): derived — Arithmetic on observed inputs only. No tunable assumption enters it.

available_gb = Σ usable_gb over memory pools

unified memory 192 = 192 GB

unified memory.usable_gb: 192 GB — 256 GB physical

Usable memory is below physical memory for at least one pool (OS/driver reservation; editable in Settings).

512 GB Mac (Apple Mac Studio (2026), M5 Ultra 36-core CPU / 80-core GPU, 512GB): derived — Arithmetic on observed inputs only. No tunable assumption enters it.

available_gb = Σ usable_gb over memory pools

unified memory 384 = 384 GB

unified memory.usable_gb: 384 GB — 512 GB physical

Usable memory is below physical memory for at least one pool (OS/driver reservation; editable in Settings).

36 machines run your setup. Speeds are tokens a second; prices are before sales tax (see Deliver to, above). 1 more machine would run it but has no current price.

Best value

1× Radeon AI PRO R9700 32GB (32358-01-20G) · EPYC 9015 · 8× M321R8GA0BB0-CQK 64GB DDR5-4800 RDIMM

Build your own · 517 GB for AI · 212 GB to spare

Code: 13.3
Images: 4.9
Speech: 89.6

At least$6,328

SabrePC, Newegg and 1 more Out of stock

Checked 15 h ago

via access tier 2, route sabrepc_ngstate; access tier 3, route newegg_realtime; access tier 1, route arctic_jsonld (landed, before sales tax) · SabrePC, Newegg, ARCTIC don't show shipping until checkout.

View

Show the work

Speeds are tokens a second for each model on this machine, estimated from its memory bandwidth and calibrated against published benchmarks; each one expands below to its formula and inputs.

GLM-5.3-Flash (Code)

Speed: estimate — Arithmetic that also depends on a tunable assumption, listed below.

tok_s = 1 / (Σ_pool bytes_per_token × share_pool / (bandwidth_pool × efficiency_pool) + Σ_pool share_pool × overhead_pool)

GPU VRAM: 1.6 GB ÷ (640 GB/s × 0.48) + System RAM: 7.149 GB ÷ (307.2 GB/s × 0.7) + 36.74 ms fixed per-token overhead (share-weighted) → 13.3 tok/s

GPU VRAM.bandwidth_gbs: 640 GB/s — component spec
GPU VRAM.efficiency_moe: 0.48 — Settings efficiency.gpu_vram — Discrete GPU: 10 dense + 3 MoE points; fit on published benchmarks, leave-one-out validated — docs/research/12b-estimator-validation.md
GPU VRAM.active_share: 0.1829 — placement
System RAM.bandwidth_gbs: 307.2 GB/s — component spec
System RAM.efficiency_moe: 0.7 — Settings efficiency.server_ddr5 — Server DDR5/MRDIMM single socket: 6 dense + 7 MoE points (small n); fit on published benchmarks, leave-one-out validated — docs/research/12b-estimator-validation.md. The 44.61 ms per-token overhead is a CPU-engine constant, not a property of DDR5: its fit is set by a 128-core Xeon 6980P running llama.cpp (55.3 ms/token on a 3B model) and SGLang (17.0 ms) with the CPU executing every layer (research 12 §6). A GPU spilling into this memory pays it only for the share of each token read from RAM (lib/perf/decode.ts refinement 3).

1× GeForce RTX 5080 16G SHADOW 3X OC · EPYC 9015 · 8× M321R8GA0BB0-CQK 64GB DDR5-4800 RDIMM

Build your own · 501 GB for AI · 196 GB to spare

Code: 12.6
Images: 4.7
Speech: 125

At least$6,414

SabrePC, Newegg and 1 more Out of stock

Checked 15 h ago

View

Show the work

Speeds are tokens a second for each model on this machine, estimated from its memory bandwidth and calibrated against published benchmarks; each one expands below to its formula and inputs.

1× GeForce RTX 5080 WINDFORCE OC 16G (GV-N5080WF3OC-16GD) · EPYC 9015 · 8× M321R8GA0BB0-CQK 64GB DDR5-4800 RDIMM

Build your own · 501 GB for AI · 196 GB to spare

Code: 12.6
Images: 4.7
Speech: 125

At least$6,424

SabrePC, Newegg and 1 more Out of stock

Checked 15 h ago

View

Show the work

Speeds are tokens a second for each model on this machine, estimated from its memory bandwidth and calibrated against published benchmarks; each one expands below to its formula and inputs.

1× Radeon AI PRO R9700 Creator 32GB (R9700 CT) · EPYC 9015 · 8× M321R8GA0BB0-CQK 64GB DDR5-4800 RDIMM

Build your own · 517 GB for AI · 212 GB to spare

Code: 13.3
Images: 4.9
Speech: 89.6

At least$6,424

SabrePC, Newegg and 1 more Out of stock

Stale · checked 29 d ago

via access tier 2, route sabrepc_ngstate; access tier 2, route seed:research; access tier 3, route newegg_realtime; access tier 1, route arctic_jsonld (landed, before sales tax) · SabrePC, Newegg, ARCTIC don't show shipping until checkout.

View

1× AI TOP Radeon AI PRO R9700 32GB (GV-R9700AI TOP-32GD) · EPYC 9015 · 8× M321R8GA0BB0-CQK 64GB DDR5-4800 RDIMM

Build your own · 517 GB for AI · 212 GB to spare

Code: 13.3
Images: 4.9
Speech: 89.6

At least$6,435

SabrePC, Newegg and 1 more Out of stock

Checked 15 h ago

View

Show the work

Speeds are tokens a second for each model on this machine, estimated from its memory bandwidth and calibrated against published benchmarks; each one expands below to its formula and inputs.

2× Radeon AI PRO R9700 32GB (32358-01-20G) · EPYC 9015 · 8× M321R4GA3BB6-CQK 32GB DDR5-4800 RDIMM

Build your own · 305 GB for AI · 0.2 GB to spare

Code: 15.4
Images: 5.5
Speech: 89.6

At least$7,270

SabrePC, Newegg and 1 more Out of stock

Checked 15 h ago

View

Show the work

Speeds are tokens a second for each model on this machine, estimated from its memory bandwidth and calibrated against published benchmarks; each one expands below to its formula and inputs.

2× Radeon AI PRO R9700 Creator 32GB (R9700 CT) · EPYC 9015 · 8× M321R4GA3BB6-CQK 32GB DDR5-4800 RDIMM

Build your own · 305 GB for AI · 0.2 GB to spare

Code: 15.4
Images: 5.5
Speech: 89.6

At least$7,462

Newegg, SabrePC and 1 more Out of stock

Stale · checked 29 d ago

via access tier 2, route seed:research; access tier 2, route sabrepc_ngstate; access tier 3, route newegg_realtime; access tier 1, route arctic_jsonld (landed, before sales tax) · Newegg, SabrePC, ARCTIC don't show shipping until checkout.

View

2× AI TOP Radeon AI PRO R9700 32GB (GV-R9700AI TOP-32GD) · EPYC 9015 · 8× M321R4GA3BB6-CQK 32GB DDR5-4800 RDIMM

Build your own · 305 GB for AI · 0.2 GB to spare

Code: 15.4
Images: 5.5
Speech: 89.6

At least$7,483

SabrePC, Newegg and 1 more Out of stock

Checked 15 h ago

View

Show the work

Speeds are tokens a second for each model on this machine, estimated from its memory bandwidth and calibrated against published benchmarks; each one expands below to its formula and inputs.

Show 28 more machines

Close, but not quite

CPU only · EPYC 9015 · 8× M321R8GA0BB0-CQK 64GB DDR5-4800 RDIMM (at least $4,723) fits everything, but Qwen3.8-Flash-Next (Images) runs at 4.4 tokens a second — below ≥ 4.7 tok/s. That counts measured memory only: the sizes of Qwen3.8-Flash-Next's and Qwen3-ASR 1.7B's projector files (mmproj) are not recorded. Show slower machines (at least 4.35 tokens a second)
1× Radeon AI PRO R9700 32GB (32358-01-20G) · EPYC 9015 · 8× M321R4GA3BB6-CQK 32GB DDR5-4800 RDIMM (at least $5,665) fits only if you load models when needed: keeping all of them cached needs 305 GB, it has 274.2 GB for AI. That counts measured memory only: the sizes of Qwen3.8-Flash-Next's and Qwen3-ASR 1.7B's projector files (mmproj) are not recorded. Switch to load when needed
1× GeForce RTX 5080 16G SHADOW 3X OC · EPYC 9015 · 8× M321R4GA3BB6-CQK 32GB DDR5-4800 RDIMM (at least $5,751) fits only if you load models when needed: keeping all of them cached needs 305 GB, it has 258.7 GB for AI. That counts measured memory only: the sizes of Qwen3.8-Flash-Next's and Qwen3-ASR 1.7B's projector files (mmproj) are not recorded. Switch to load when needed
1× GeForce RTX 5080 WINDFORCE OC 16G (GV-N5080WF3OC-16GD) · EPYC 9015 · 8× M321R4GA3BB6-CQK 32GB DDR5-4800 RDIMM (at least $5,761) fits only if you load models when needed: keeping all of them cached needs 305 GB, it has 258.7 GB for AI. That counts measured memory only: the sizes of Qwen3.8-Flash-Next's and Qwen3-ASR 1.7B's projector files (mmproj) are not recorded. Switch to load when needed
CPU only · EPYC 9015 · 8× Server Premier KSM64R52BD4-96MC 96GB DDR5-6400 RDIMM (at least $31,363) fits everything, but Qwen3.8-Flash-Next (Images) runs at 4.4 tokens a second — below ≥ 4.7 tok/s. That counts measured memory only: the sizes of Qwen3.8-Flash-Next's and Qwen3-ASR 1.7B's projector files (mmproj) are not recorded. Show slower machines (at least 4.35 tokens a second)
CPU only · EPYC 9015 · 8× Server Premier KSM64R52BD4-128MB 128GB DDR5-6400 RDIMM (at least $33,772) fits everything, but Qwen3.8-Flash-Next (Images) runs at 4.4 tokens a second — below ≥ 4.7 tok/s. That counts measured memory only: the sizes of Qwen3.8-Flash-Next's and Qwen3-ASR 1.7B's projector files (mmproj) are not recorded.

About this search

89 machines checked (89 evaluated); 83 part combinations skipped as incompatible. 36 priced machines of any kind run your plan, budget aside.

1333 platform combinations were collapsed. For each socket and memory size the search carries the CHEAPEST build and, when it is a different part, the one with the most CPU compute (which is what prefill depends on); the rest differ only in price.

8 CPU+GPU hybrid candidates were the same parts as a GPU rig already in the list and are listed once.

1 more machine of this kind would run your setup but has no current price, so it is not listed.

Machine results are reused within the same minute while no price, part, offer, model or setting has changed. Prices are aged as of the start of that minute, so a price passing the 36-hour freshness line is flagged up to 60 seconds late. Evaluated at Oct 18, 6:01 PM ET, in 595 ms.

Checking every machine against your plan…

36 machines run your setup. Speeds are tokens a second; prices are before sales tax (see Deliver to, above). 1 more machine would run it but has no current price.

Best value

1× Radeon AI PRO R9700 32GB (32358-01-20G) · EPYC 9015 · 8× M321R8GA0BB0-CQK 64GB DDR5-4800 RDIMM

Build your own · 517 GB for AI · 212 GB to spare

Code: 13.3
Images: 4.9
Speech: 89.6

At least$6,328

SabrePC, Newegg and 1 more Out of stock

Checked 15 h ago

View

Show the work

Speeds are tokens a second for each model on this machine, estimated from its memory bandwidth and calibrated against published benchmarks; each one expands below to its formula and inputs.

GLM-5.3-Flash (Code)

Speed: estimate — Arithmetic that also depends on a tunable assumption, listed below.

tok_s = 1 / (Σ_pool bytes_per_token × share_pool / (bandwidth_pool × efficiency_pool) + Σ_pool share_pool × overhead_pool)

GPU VRAM: 1.6 GB ÷ (640 GB/s × 0.48) + System RAM: 7.149 GB ÷ (307.2 GB/s × 0.7) + 36.74 ms fixed per-token overhead (share-weighted) → 13.3 tok/s

GPU VRAM.bandwidth_gbs: 640 GB/s — component spec
GPU VRAM.efficiency_moe: 0.48 — Settings efficiency.gpu_vram — Discrete GPU: 10 dense + 3 MoE points; fit on published benchmarks, leave-one-out validated — docs/research/12b-estimator-validation.md
GPU VRAM.active_share: 0.1829 — placement
System RAM.bandwidth_gbs: 307.2 GB/s — component spec
System RAM.efficiency_moe: 0.7 — Settings efficiency.server_ddr5 — Server DDR5/MRDIMM single socket: 6 dense + 7 MoE points (small n); fit on published benchmarks, leave-one-out validated — docs/research/12b-estimator-validation.md. The 44.61 ms per-token overhead is a CPU-engine constant, not a property of DDR5: its fit is set by a 128-core Xeon 6980P running llama.cpp (55.3 ms/token on a 3B model) and SGLang (17.0 ms) with the CPU executing every layer (research 12 §6). A GPU spilling into this memory pays it only for the share of each token read from RAM (lib/perf/decode.ts refinement 3).

1× GeForce RTX 5080 16G SHADOW 3X OC · EPYC 9015 · 8× M321R8GA0BB0-CQK 64GB DDR5-4800 RDIMM

Build your own · 501 GB for AI · 196 GB to spare

Code: 12.6
Images: 4.7
Speech: 125

At least$6,414

SabrePC, Newegg and 1 more Out of stock

Checked 15 h ago

View

Show the work

Speeds are tokens a second for each model on this machine, estimated from its memory bandwidth and calibrated against published benchmarks; each one expands below to its formula and inputs.

1× GeForce RTX 5080 WINDFORCE OC 16G (GV-N5080WF3OC-16GD) · EPYC 9015 · 8× M321R8GA0BB0-CQK 64GB DDR5-4800 RDIMM

Build your own · 501 GB for AI · 196 GB to spare

Code: 12.6
Images: 4.7
Speech: 125

At least$6,424

SabrePC, Newegg and 1 more Out of stock

Checked 15 h ago

View

Show the work

Speeds are tokens a second for each model on this machine, estimated from its memory bandwidth and calibrated against published benchmarks; each one expands below to its formula and inputs.

1× Radeon AI PRO R9700 Creator 32GB (R9700 CT) · EPYC 9015 · 8× M321R8GA0BB0-CQK 64GB DDR5-4800 RDIMM

Build your own · 517 GB for AI · 212 GB to spare

Code: 13.3
Images: 4.9
Speech: 89.6

At least$6,424

SabrePC, Newegg and 1 more Out of stock

Stale · checked 29 d ago

View

1× AI TOP Radeon AI PRO R9700 32GB (GV-R9700AI TOP-32GD) · EPYC 9015 · 8× M321R8GA0BB0-CQK 64GB DDR5-4800 RDIMM

Build your own · 517 GB for AI · 212 GB to spare

Code: 13.3
Images: 4.9
Speech: 89.6

At least$6,435

SabrePC, Newegg and 1 more Out of stock

Checked 15 h ago

View

Show the work

Speeds are tokens a second for each model on this machine, estimated from its memory bandwidth and calibrated against published benchmarks; each one expands below to its formula and inputs.

2× Radeon AI PRO R9700 32GB (32358-01-20G) · EPYC 9015 · 8× M321R4GA3BB6-CQK 32GB DDR5-4800 RDIMM

Build your own · 305 GB for AI · 0.2 GB to spare

Code: 15.4
Images: 5.5
Speech: 89.6

At least$7,270

SabrePC, Newegg and 1 more Out of stock

Checked 15 h ago

View

Show the work

Speeds are tokens a second for each model on this machine, estimated from its memory bandwidth and calibrated against published benchmarks; each one expands below to its formula and inputs.

2× Radeon AI PRO R9700 Creator 32GB (R9700 CT) · EPYC 9015 · 8× M321R4GA3BB6-CQK 32GB DDR5-4800 RDIMM

Build your own · 305 GB for AI · 0.2 GB to spare

Code: 15.4
Images: 5.5
Speech: 89.6

At least$7,462

Newegg, SabrePC and 1 more Out of stock

Stale · checked 29 d ago

View

2× AI TOP Radeon AI PRO R9700 32GB (GV-R9700AI TOP-32GD) · EPYC 9015 · 8× M321R4GA3BB6-CQK 32GB DDR5-4800 RDIMM

Build your own · 305 GB for AI · 0.2 GB to spare

Code: 15.4
Images: 5.5
Speech: 89.6

At least$7,483

SabrePC, Newegg and 1 more Out of stock

Checked 15 h ago

View

Show the work

Speeds are tokens a second for each model on this machine, estimated from its memory bandwidth and calibrated against published benchmarks; each one expands below to its formula and inputs.

Show 28 more machines

Close, but not quite

CPU only · EPYC 9015 · 8× M321R8GA0BB0-CQK 64GB DDR5-4800 RDIMM (at least $4,723) fits everything, but Qwen3.8-Flash-Next (Images) runs at 4.4 tokens a second — below ≥ 4.7 tok/s. That counts measured memory only: the sizes of Qwen3.8-Flash-Next's and Qwen3-ASR 1.7B's projector files (mmproj) are not recorded. Show slower machines (at least 4.35 tokens a second)
1× Radeon AI PRO R9700 32GB (32358-01-20G) · EPYC 9015 · 8× M321R4GA3BB6-CQK 32GB DDR5-4800 RDIMM (at least $5,665) fits only if you load models when needed: keeping all of them cached needs 305 GB, it has 274.2 GB for AI. That counts measured memory only: the sizes of Qwen3.8-Flash-Next's and Qwen3-ASR 1.7B's projector files (mmproj) are not recorded. Switch to load when needed
1× GeForce RTX 5080 16G SHADOW 3X OC · EPYC 9015 · 8× M321R4GA3BB6-CQK 32GB DDR5-4800 RDIMM (at least $5,751) fits only if you load models when needed: keeping all of them cached needs 305 GB, it has 258.7 GB for AI. That counts measured memory only: the sizes of Qwen3.8-Flash-Next's and Qwen3-ASR 1.7B's projector files (mmproj) are not recorded. Switch to load when needed
1× GeForce RTX 5080 WINDFORCE OC 16G (GV-N5080WF3OC-16GD) · EPYC 9015 · 8× M321R4GA3BB6-CQK 32GB DDR5-4800 RDIMM (at least $5,761) fits only if you load models when needed: keeping all of them cached needs 305 GB, it has 258.7 GB for AI. That counts measured memory only: the sizes of Qwen3.8-Flash-Next's and Qwen3-ASR 1.7B's projector files (mmproj) are not recorded. Switch to load when needed
CPU only · EPYC 9015 · 8× Server Premier KSM64R52BD4-96MC 96GB DDR5-6400 RDIMM (at least $31,363) fits everything, but Qwen3.8-Flash-Next (Images) runs at 4.4 tokens a second — below ≥ 4.7 tok/s. That counts measured memory only: the sizes of Qwen3.8-Flash-Next's and Qwen3-ASR 1.7B's projector files (mmproj) are not recorded. Show slower machines (at least 4.35 tokens a second)
CPU only · EPYC 9015 · 8× Server Premier KSM64R52BD4-128MB 128GB DDR5-6400 RDIMM (at least $33,772) fits everything, but Qwen3.8-Flash-Next (Images) runs at 4.4 tokens a second — below ≥ 4.7 tok/s. That counts measured memory only: the sizes of Qwen3.8-Flash-Next's and Qwen3-ASR 1.7B's projector files (mmproj) are not recorded.

About this search

89 machines checked (89 evaluated); 83 part combinations skipped as incompatible. 36 priced machines of any kind run your plan, budget aside.

8 CPU+GPU hybrid candidates were the same parts as a GPU rig already in the list and are listed once.

1 more machine of this kind would run your setup but has no current price, so it is not listed.

Qwen3.8-Flash-Next (Images)

Speed: estimate — Arithmetic that also depends on a tunable assumption, listed below.

tok_s = 1 / (Σ_pool bytes_per_token × share_pool / (bandwidth_pool × efficiency_pool) + Σ_pool share_pool × overhead_pool)

GPU VRAM: 9.535 GB ÷ (640 GB/s × 0.48) + System RAM: 30.24 GB ÷ (307.2 GB/s × 0.7) + 34.3 ms fixed per-token overhead (share-weighted) → 4.86 tok/s

GPU VRAM.bandwidth_gbs: 640 GB/s — component spec
GPU VRAM.efficiency_moe: 0.48 — Settings efficiency.gpu_vram — Discrete GPU: 10 dense + 3 MoE points; fit on published benchmarks, leave-one-out validated — docs/research/12b-estimator-validation.md
GPU VRAM.active_share: 0.2397 — placement
System RAM.bandwidth_gbs: 307.2 GB/s — component spec
System RAM.efficiency_moe: 0.7 — Settings efficiency.server_ddr5 — Server DDR5/MRDIMM single socket: 6 dense + 7 MoE points (small n); fit on published benchmarks, leave-one-out validated — docs/research/12b-estimator-validation.md. The 44.61 ms per-token overhead is a CPU-engine constant, not a property of DDR5: its fit is set by a 128-core Xeon 6980P running llama.cpp (55.3 ms/token on a 3B model) and SGLang (17.0 ms) with the CPU executing every layer (research 12 §6). A GPU spilling into this memory pays it only for the share of each token read from RAM (lib/perf/decode.ts refinement 3).
System RAM.active_share: 0.7603 — placement
per_token_overhead_ms: 34.29514218361304 ms — derived: 0.24 × 1.58 ms (GPU VRAM) + 0.76 × 44.61 ms (System RAM) = 34.3 ms
bytes_per_token_gb: 39.7749 GB

MoE: only active parameters are read per token (SPEC §3).
Effective bits include embeddings and higher-precision tensors in mixed quants.
Assumes active weights are quantized like the file average.
KV cache is re-read on every token (DECISIONS D-008).
Pools are read sequentially per token, so per-pool times add (harmonic combination).
Per-token overhead on a split placement is DERIVED, not measured: each pool's own single-pool figure is charged in proportion to the share of each token it serves, because the fixed cost is per operation and the split puts that share of the operations on each device.
Calibrated with that pool serving the whole model (D-013): GPU VRAM (1.58 ms), System RAM (44.61 ms).
Batch size 1 (single user, interactive).

First reply: estimate — Arithmetic that also depends on a tunable assumption, listed below.

ttft_s = prompt_tokens / prefill_tok_s

= 32,768 / 577.57 = 56.7 s

prompt_tokens: 32768 tokens
prefill_tok_s: 577.57 tok/s

No published prefill measurement for this hardware; scaled from peak compute (estimate).
Model q_dim unknown — attention FLOPs omitted, so long-prompt TTFT is understated.

GLM-5.3-Flash (Code)

Speed: estimate — Arithmetic that also depends on a tunable assumption, listed below.

tok_s = 1 / (Σ_pool bytes_per_token × share_pool / (bandwidth_pool × efficiency_pool) + Σ_pool share_pool × overhead_pool)

GPU VRAM: 0.8 GB ÷ (960 GB/s × 0.48) + System RAM: 7.949 GB ÷ (307.2 GB/s × 0.7) + 40.68 ms fixed per-token overhead (share-weighted) → 12.6 tok/s

GPU VRAM.bandwidth_gbs: 960 GB/s — component spec
GPU VRAM.efficiency_moe: 0.48 — Settings efficiency.gpu_vram — Discrete GPU: 10 dense + 3 MoE points; fit on published benchmarks, leave-one-out validated — docs/research/12b-estimator-validation.md
GPU VRAM.active_share: 0.0914 — placement
System RAM.bandwidth_gbs: 307.2 GB/s — component spec
System RAM.efficiency_moe: 0.7 — Settings efficiency.server_ddr5 — Server DDR5/MRDIMM single socket: 6 dense + 7 MoE points (small n); fit on published benchmarks, leave-one-out validated — docs/research/12b-estimator-validation.md. The 44.61 ms per-token overhead is a CPU-engine constant, not a property of DDR5: its fit is set by a 128-core Xeon 6980P running llama.cpp (55.3 ms/token on a 3B model) and SGLang (17.0 ms) with the CPU executing every layer (research 12 §6). A GPU spilling into this memory pays it only for the share of each token read from RAM (lib/perf/decode.ts refinement 3).
System RAM.active_share: 0.9086 — placement
per_token_overhead_ms: 40.67552676657429 ms — derived: 0.091 × 1.58 ms (GPU VRAM) + 0.909 × 44.61 ms (System RAM) = 40.68 ms
bytes_per_token_gb: 8.7487 GB

MoE: only active parameters are read per token (SPEC §3).
Effective bits include embeddings and higher-precision tensors in mixed quants.
Assumes active weights are quantized like the file average.
KV cache is re-read on every token (DECISIONS D-008).
Pools are read sequentially per token, so per-pool times add (harmonic combination).
Per-token overhead on a split placement is DERIVED, not measured: each pool's own single-pool figure is charged in proportion to the share of each token it serves, because the fixed cost is per operation and the split puts that share of the operations on each device.
Calibrated with that pool serving the whole model (D-013): GPU VRAM (1.58 ms), System RAM (44.61 ms).
Batch size 1 (single user, interactive).

First reply: estimate — Arithmetic that also depends on a tunable assumption, listed below.

ttft_s = prompt_tokens / prefill_tok_s

= 32,768 / 1,176.96 = 27.8 s

prompt_tokens: 32768 tokens
prefill_tok_s: 1176.96 tok/s

No published prefill measurement for this hardware; scaled from peak compute (estimate).
Model q_dim unknown — attention FLOPs omitted, so long-prompt TTFT is understated.

Qwen3.8-Flash-Next (Images)

Speed: estimate — Arithmetic that also depends on a tunable assumption, listed below.

tok_s = 1 / (Σ_pool bytes_per_token × share_pool / (bandwidth_pool × efficiency_pool) + Σ_pool share_pool × overhead_pool)

GPU VRAM: 4.767 GB ÷ (960 GB/s × 0.48) + System RAM: 35.008 GB ÷ (307.2 GB/s × 0.7) + 39.45 ms fixed per-token overhead (share-weighted) → 4.7 tok/s

GPU VRAM.bandwidth_gbs: 960 GB/s — component spec
GPU VRAM.efficiency_moe: 0.48 — Settings efficiency.gpu_vram — Discrete GPU: 10 dense + 3 MoE points; fit on published benchmarks, leave-one-out validated — docs/research/12b-estimator-validation.md
GPU VRAM.active_share: 0.1199 — placement
System RAM.bandwidth_gbs: 307.2 GB/s — component spec
System RAM.efficiency_moe: 0.7 — Settings efficiency.server_ddr5 — Server DDR5/MRDIMM single socket: 6 dense + 7 MoE points (small n); fit on published benchmarks, leave-one-out validated — docs/research/12b-estimator-validation.md. The 44.61 ms per-token overhead is a CPU-engine constant, not a property of DDR5: its fit is set by a 128-core Xeon 6980P running llama.cpp (55.3 ms/token on a 3B model) and SGLang (17.0 ms) with the CPU executing every layer (research 12 §6). A GPU spilling into this memory pays it only for the share of each token read from RAM (lib/perf/decode.ts refinement 3).
System RAM.active_share: 0.8801 — placement
per_token_overhead_ms: 39.45257109180652 ms — derived: 0.12 × 1.58 ms (GPU VRAM) + 0.88 × 44.61 ms (System RAM) = 39.45 ms
bytes_per_token_gb: 39.7749 GB

MoE: only active parameters are read per token (SPEC §3).
Effective bits include embeddings and higher-precision tensors in mixed quants.
Assumes active weights are quantized like the file average.
KV cache is re-read on every token (DECISIONS D-008).
Pools are read sequentially per token, so per-pool times add (harmonic combination).
Per-token overhead on a split placement is DERIVED, not measured: each pool's own single-pool figure is charged in proportion to the share of each token it serves, because the fixed cost is per operation and the split puts that share of the operations on each device.
Calibrated with that pool serving the whole model (D-013): GPU VRAM (1.58 ms), System RAM (44.61 ms).
Batch size 1 (single user, interactive).

First reply: estimate — Arithmetic that also depends on a tunable assumption, listed below.

ttft_s = prompt_tokens / prefill_tok_s

= 32,768 / 340.5 = 96.2 s

prompt_tokens: 32768 tokens
prefill_tok_s: 340.5 tok/s

No published prefill measurement for this hardware; scaled from peak compute (estimate).
Model q_dim unknown — attention FLOPs omitted, so long-prompt TTFT is understated.

GLM-5.3-Flash (Code)

Speed: estimate — Arithmetic that also depends on a tunable assumption, listed below.

tok_s = 1 / (Σ_pool bytes_per_token × share_pool / (bandwidth_pool × efficiency_pool) + Σ_pool share_pool × overhead_pool)

GPU VRAM: 0.8 GB ÷ (960 GB/s × 0.48) + System RAM: 7.949 GB ÷ (307.2 GB/s × 0.7) + 40.68 ms fixed per-token overhead (share-weighted) → 12.6 tok/s

GPU VRAM.bandwidth_gbs: 960 GB/s — component spec
GPU VRAM.efficiency_moe: 0.48 — Settings efficiency.gpu_vram — Discrete GPU: 10 dense + 3 MoE points; fit on published benchmarks, leave-one-out validated — docs/research/12b-estimator-validation.md
GPU VRAM.active_share: 0.0914 — placement
System RAM.bandwidth_gbs: 307.2 GB/s — component spec
System RAM.efficiency_moe: 0.7 — Settings efficiency.server_ddr5 — Server DDR5/MRDIMM single socket: 6 dense + 7 MoE points (small n); fit on published benchmarks, leave-one-out validated — docs/research/12b-estimator-validation.md. The 44.61 ms per-token overhead is a CPU-engine constant, not a property of DDR5: its fit is set by a 128-core Xeon 6980P running llama.cpp (55.3 ms/token on a 3B model) and SGLang (17.0 ms) with the CPU executing every layer (research 12 §6). A GPU spilling into this memory pays it only for the share of each token read from RAM (lib/perf/decode.ts refinement 3).
System RAM.active_share: 0.9086 — placement
per_token_overhead_ms: 40.67552676657429 ms — derived: 0.091 × 1.58 ms (GPU VRAM) + 0.909 × 44.61 ms (System RAM) = 40.68 ms
bytes_per_token_gb: 8.7487 GB

MoE: only active parameters are read per token (SPEC §3).
Effective bits include embeddings and higher-precision tensors in mixed quants.
Assumes active weights are quantized like the file average.
KV cache is re-read on every token (DECISIONS D-008).
Pools are read sequentially per token, so per-pool times add (harmonic combination).
Per-token overhead on a split placement is DERIVED, not measured: each pool's own single-pool figure is charged in proportion to the share of each token it serves, because the fixed cost is per operation and the split puts that share of the operations on each device.
Calibrated with that pool serving the whole model (D-013): GPU VRAM (1.58 ms), System RAM (44.61 ms).
Batch size 1 (single user, interactive).

First reply: estimate — Arithmetic that also depends on a tunable assumption, listed below.

ttft_s = prompt_tokens / prefill_tok_s

= 32,768 / 1,176.96 = 27.8 s

prompt_tokens: 32768 tokens
prefill_tok_s: 1176.96 tok/s

No published prefill measurement for this hardware; scaled from peak compute (estimate).
Model q_dim unknown — attention FLOPs omitted, so long-prompt TTFT is understated.

Qwen3.8-Flash-Next (Images)

Speed: estimate — Arithmetic that also depends on a tunable assumption, listed below.

tok_s = 1 / (Σ_pool bytes_per_token × share_pool / (bandwidth_pool × efficiency_pool) + Σ_pool share_pool × overhead_pool)

GPU VRAM: 4.767 GB ÷ (960 GB/s × 0.48) + System RAM: 35.008 GB ÷ (307.2 GB/s × 0.7) + 39.45 ms fixed per-token overhead (share-weighted) → 4.7 tok/s

GPU VRAM.bandwidth_gbs: 960 GB/s — component spec
GPU VRAM.efficiency_moe: 0.48 — Settings efficiency.gpu_vram — Discrete GPU: 10 dense + 3 MoE points; fit on published benchmarks, leave-one-out validated — docs/research/12b-estimator-validation.md
GPU VRAM.active_share: 0.1199 — placement
System RAM.bandwidth_gbs: 307.2 GB/s — component spec
System RAM.efficiency_moe: 0.7 — Settings efficiency.server_ddr5 — Server DDR5/MRDIMM single socket: 6 dense + 7 MoE points (small n); fit on published benchmarks, leave-one-out validated — docs/research/12b-estimator-validation.md. The 44.61 ms per-token overhead is a CPU-engine constant, not a property of DDR5: its fit is set by a 128-core Xeon 6980P running llama.cpp (55.3 ms/token on a 3B model) and SGLang (17.0 ms) with the CPU executing every layer (research 12 §6). A GPU spilling into this memory pays it only for the share of each token read from RAM (lib/perf/decode.ts refinement 3).
System RAM.active_share: 0.8801 — placement
per_token_overhead_ms: 39.45257109180652 ms — derived: 0.12 × 1.58 ms (GPU VRAM) + 0.88 × 44.61 ms (System RAM) = 39.45 ms
bytes_per_token_gb: 39.7749 GB

MoE: only active parameters are read per token (SPEC §3).
Effective bits include embeddings and higher-precision tensors in mixed quants.
Assumes active weights are quantized like the file average.
KV cache is re-read on every token (DECISIONS D-008).
Pools are read sequentially per token, so per-pool times add (harmonic combination).
Per-token overhead on a split placement is DERIVED, not measured: each pool's own single-pool figure is charged in proportion to the share of each token it serves, because the fixed cost is per operation and the split puts that share of the operations on each device.
Calibrated with that pool serving the whole model (D-013): GPU VRAM (1.58 ms), System RAM (44.61 ms).
Batch size 1 (single user, interactive).

First reply: estimate — Arithmetic that also depends on a tunable assumption, listed below.

ttft_s = prompt_tokens / prefill_tok_s

= 32,768 / 340.5 = 96.2 s

prompt_tokens: 32768 tokens
prefill_tok_s: 340.5 tok/s

No published prefill measurement for this hardware; scaled from peak compute (estimate).
Model q_dim unknown — attention FLOPs omitted, so long-prompt TTFT is understated.

Speeds are tokens a second for each model on this machine, estimated from its memory bandwidth and calibrated against published benchmarks; each one expands below to its formula and inputs.

GLM-5.3-Flash (Code)

Speed: estimate — Arithmetic that also depends on a tunable assumption, listed below.

tok_s = 1 / (Σ_pool bytes_per_token × share_pool / (bandwidth_pool × efficiency_pool) + Σ_pool share_pool × overhead_pool)

GPU VRAM: 1.6 GB ÷ (640 GB/s × 0.48) + System RAM: 7.149 GB ÷ (307.2 GB/s × 0.7) + 36.74 ms fixed per-token overhead (share-weighted) → 13.3 tok/s

GPU VRAM.bandwidth_gbs: 640 GB/s — component spec
GPU VRAM.efficiency_moe: 0.48 — Settings efficiency.gpu_vram — Discrete GPU: 10 dense + 3 MoE points; fit on published benchmarks, leave-one-out validated — docs/research/12b-estimator-validation.md
GPU VRAM.active_share: 0.1829 — placement
System RAM.bandwidth_gbs: 307.2 GB/s — component spec
System RAM.efficiency_moe: 0.7 — Settings efficiency.server_ddr5 — Server DDR5/MRDIMM single socket: 6 dense + 7 MoE points (small n); fit on published benchmarks, leave-one-out validated — docs/research/12b-estimator-validation.md. The 44.61 ms per-token overhead is a CPU-engine constant, not a property of DDR5: its fit is set by a 128-core Xeon 6980P running llama.cpp (55.3 ms/token on a 3B model) and SGLang (17.0 ms) with the CPU executing every layer (research 12 §6). A GPU spilling into this memory pays it only for the share of each token read from RAM (lib/perf/decode.ts refinement 3).
System RAM.active_share: 0.8171 — placement
per_token_overhead_ms: 36.74105353314861 ms — derived: 0.183 × 1.58 ms (GPU VRAM) + 0.817 × 44.61 ms (System RAM) = 36.74 ms
bytes_per_token_gb: 8.7487 GB

MoE: only active parameters are read per token (SPEC §3).
Effective bits include embeddings and higher-precision tensors in mixed quants.
Assumes active weights are quantized like the file average.
KV cache is re-read on every token (DECISIONS D-008).
Pools are read sequentially per token, so per-pool times add (harmonic combination).
Per-token overhead on a split placement is DERIVED, not measured: each pool's own single-pool figure is charged in proportion to the share of each token it serves, because the fixed cost is per operation and the split puts that share of the operations on each device.
Calibrated with that pool serving the whole model (D-013): GPU VRAM (1.58 ms), System RAM (44.61 ms).
Batch size 1 (single user, interactive).

First reply: estimate — Arithmetic that also depends on a tunable assumption, listed below.

ttft_s = prompt_tokens / prefill_tok_s

= 32,768 / 1,996.44 = 16.4 s

prompt_tokens: 32768 tokens
prefill_tok_s: 1996.44 tok/s

No published prefill measurement for this hardware; scaled from peak compute (estimate).
Model q_dim unknown — attention FLOPs omitted, so long-prompt TTFT is understated.

Qwen3.8-Flash-Next (Images)

Speed: estimate — Arithmetic that also depends on a tunable assumption, listed below.

tok_s = 1 / (Σ_pool bytes_per_token × share_pool / (bandwidth_pool × efficiency_pool) + Σ_pool share_pool × overhead_pool)

GPU VRAM: 9.535 GB ÷ (640 GB/s × 0.48) + System RAM: 30.24 GB ÷ (307.2 GB/s × 0.7) + 34.3 ms fixed per-token overhead (share-weighted) → 4.86 tok/s

GPU VRAM.bandwidth_gbs: 640 GB/s — component spec
GPU VRAM.efficiency_moe: 0.48 — Settings efficiency.gpu_vram — Discrete GPU: 10 dense + 3 MoE points; fit on published benchmarks, leave-one-out validated — docs/research/12b-estimator-validation.md
GPU VRAM.active_share: 0.2397 — placement
System RAM.bandwidth_gbs: 307.2 GB/s — component spec
System RAM.efficiency_moe: 0.7 — Settings efficiency.server_ddr5 — Server DDR5/MRDIMM single socket: 6 dense + 7 MoE points (small n); fit on published benchmarks, leave-one-out validated — docs/research/12b-estimator-validation.md. The 44.61 ms per-token overhead is a CPU-engine constant, not a property of DDR5: its fit is set by a 128-core Xeon 6980P running llama.cpp (55.3 ms/token on a 3B model) and SGLang (17.0 ms) with the CPU executing every layer (research 12 §6). A GPU spilling into this memory pays it only for the share of each token read from RAM (lib/perf/decode.ts refinement 3).
System RAM.active_share: 0.7603

Qwen3-ASR 1.7B (Speech)

Speed: estimate — Arithmetic that also depends on a tunable assumption, listed below.

tok_s = 1 / (bytes_per_token / (bandwidth × efficiency) + overhead)

GPU VRAM: 5.274 GB ÷ (640 GB/s × 0.86) + 1.58 ms fixed per-token overhead → 89.59 tok/s

GPU VRAM.bandwidth_gbs: 640 GB/s — component spec
GPU VRAM.efficiency: 0.86 — Settings efficiency.gpu_vram — Discrete GPU: 10 dense + 3 MoE points; fit on published benchmarks, leave-one-out validated — docs/research/12b-estimator-validation.md
GPU VRAM.active_share: 1 — placement
per_token_overhead_ms: 1.58 ms — Settings efficiency.gpu_vram.overheadMs — calibrated fit (D-013): Discrete GPU: 10 dense + 3 MoE points; fit on published benchmarks, leave-one-out validated — docs/research/12b-estimator-validation.md
bytes_per_token_gb: 5.2741 GB

Dense: every parameter is read per token.
Effective bits include embeddings and higher-precision tensors in mixed quants.
Assumes active weights are quantized like the file average.
KV cache is re-read on every token (DECISIONS D-008).
Per-token overhead of System RAM (44.61 ms) is not charged: this model does not read from that pool.

Your plan needs: estimate — Arithmetic that also depends on a tunable assumption, listed below.

Σ required_gb over every model in the stack

GLM-5.3-Flash 169.74 + Qwen3.8-Flash-Next 129.49 + Qwen3-ASR 1.7B 5.78 = 305 GB

GLM-5.3-Flash: 169.736978848 GB
Qwen3.8-Flash-Next: 129.48808638719998 GB
Qwen3-ASR 1.7B: 5.77887773952 GB

All models loaded at once (Keep all resident).
Not counted, because it is not measured yet: Qwen3.8-Flash-Next reads images through a separate projector file (mmproj); its size is not recorded yet, so the memory shown leaves it out. Qwen3-ASR 1.7B hears audio through a separate projector file (mmproj); its size is not recorded yet, so the memory shown leaves it out.

Memory for AI: derived — Arithmetic on observed inputs only. No tunable assumption enters it.

available_gb = Σ usable_gb over memory pools

GPU VRAM 31.04 + System RAM 486.4 = 517.44 GB

GPU VRAM.usable_gb: 31.04 GB — 32 GB physical
System RAM.usable_gb: 486.4 GB — 512 GB physical

Usable memory is below physical memory for at least one pool (OS/driver reservation; editable in Settings).

To spare: estimate — Arithmetic that also depends on a tunable assumption, listed below.

spare_gb = usable_gb − all-resident need

= 517.44 − 305 = 212.44 GB

usable_gb: 517.4399999999999 GB
need_gb: 305.00394297471996 GB

All models loaded at once (Keep all resident).
Not counted, because it is not measured yet: Qwen3.8-Flash-Next reads images through a separate projector file (mmproj); its size is not recorded yet, so the memory shown leaves it out. Qwen3-ASR 1.7B hears audio through a separate projector file (mmproj); its size is not recorded yet, so the memory shown leaves it out.
So the real room to spare is smaller than shown, by the size of what is not counted: the sizes of Qwen3.8-Flash-Next's and Qwen3-ASR 1.7B's projector files (mmproj) are not recorded.

Slowest model (what Fastest and Best value rank on): estimate — Arithmetic that also depends on a tunable assumption, listed below.

slowest = min(decode tok/s) over the Plan's text models

GLM-5.3-Flash 13.3, Qwen3.8-Flash-Next 4.9, Qwen3-ASR 1.7B 89.6 → Qwen3.8-Flash-Next 4.9 tok/s

GLM-5.3-Flash: 13.299051837036174 tok/s — lib/perf decode estimate
Qwen3.8-Flash-Next: 4.855340208220114 tok/s — lib/perf decode estimate
Qwen3-ASR 1.7B: 89.58730496849219 tok/s — lib/perf decode estimate

MoE: only active parameters are read per token (SPEC §3).
Effective bits include embeddings and higher-precision tensors in mixed quants.
Assumes active weights are quantized like the file average.
KV cache is re-read on every token (DECISIONS D-008).
Pools are read sequentially per token, so per-pool times add (harmonic combination).
Per-token overhead on a split placement is DERIVED, not measured: each pool's own single-pool figure is charged in proportion to the share of each token it serves, because the fixed cost is per operation and the split puts that share of the operations on each device.
Calibrated with that pool serving the whole model (D-013): GPU VRAM (1.58 ms), System RAM (44.61 ms).
Batch size 1 (single user, interactive).

Price per token a second: derived — Arithmetic on observed inputs only. No tunable assumption enters it.

cost_per_tok_s = landed_cost / decode_tok_s

= $6,424 / 4.86 tok/s = $1,323.07/tok/s

landed_cost: 6423.96 USD
decode_tok_s: 4.855340208220114 tok/s

Where each part is bought

Samsung M321R8GA0BB0-CQK 64GB DDR5-4800 RDIMM: SabrePC (opens in a new tab), shipped, checked 14 h ago, access tier 2, route sabrepc_ngstate
ASRock Radeon AI PRO R9700 Creator 32GB (R9700 CT): Newegg (opens in a new tab), shipped, checked 29 d ago, access tier 2, route seed:research
ASRock Rack GENOAD8QM3-2T/BCM: Newegg (opens in a new tab), shipped, checked 1 h ago, access tier 3, route newegg_realtime
Seasonic PRIME PX-2200: Newegg (opens in a new tab), shipped, checked 1 h ago, access tier 3, route newegg_realtime
AMD EPYC 9015: SabrePC (opens in a new tab), shipped, checked 14 h ago, access tier 2, route sabrepc_ngstate
Phanteks Enthoo Pro 2 Server Edition: Newegg (opens in a new tab), shipped, checked 1 h ago, access tier 3, route newegg_realtime
ARCTIC Freezer 4U-SP5: , shipped, checked , access tier 1, route arctic_jsonld

GLM-5.3-Flash (Code)

Speed: estimate — Arithmetic that also depends on a tunable assumption, listed below.

tok_s = 1 / (Σ_pool bytes_per_token × share_pool / (bandwidth_pool × efficiency_pool) + Σ_pool share_pool × overhead_pool)

GPU VRAM: 1.6 GB ÷ (640 GB/s × 0.48) + System RAM: 7.149 GB ÷ (307.2 GB/s × 0.7) + 36.74 ms fixed per-token overhead (share-weighted) → 13.3 tok/s

GPU VRAM.bandwidth_gbs: 640 GB/s — component spec
GPU VRAM.efficiency_moe: 0.48 — Settings efficiency.gpu_vram — Discrete GPU: 10 dense + 3 MoE points; fit on published benchmarks, leave-one-out validated — docs/research/12b-estimator-validation.md
GPU VRAM.active_share: 0.1829 — placement
System RAM.bandwidth_gbs: 307.2 GB/s — component spec
System RAM.efficiency_moe: 0.7 — Settings efficiency.server_ddr5 — Server DDR5/MRDIMM single socket: 6 dense + 7 MoE points (small n); fit on published benchmarks, leave-one-out validated — docs/research/12b-estimator-validation.md. The 44.61 ms per-token overhead is a CPU-engine constant, not a property of DDR5: its fit is set by a 128-core Xeon 6980P running llama.cpp (55.3 ms/token on a 3B model) and SGLang (17.0 ms) with the CPU executing every layer (research 12 §6). A GPU spilling into this memory pays it only for the share of each token read from RAM (lib/perf/decode.ts refinement 3).
System RAM.active_share: 0.8171 — placement
per_token_overhead_ms: 36.74105353314861 ms — derived: 0.183 × 1.58 ms (GPU VRAM) + 0.817 × 44.61 ms (System RAM) = 36.74 ms
bytes_per_token_gb: 8.7487 GB

MoE: only active parameters are read per token (SPEC §3).
Effective bits include embeddings and higher-precision tensors in mixed quants.
Assumes active weights are quantized like the file average.
KV cache is re-read on every token (DECISIONS D-008).
Pools are read sequentially per token, so per-pool times add (harmonic combination).
Per-token overhead on a split placement is DERIVED, not measured: each pool's own single-pool figure is charged in proportion to the share of each token it serves, because the fixed cost is per operation and the split puts that share of the operations on each device.
Calibrated with that pool serving the whole model (D-013): GPU VRAM (1.58 ms), System RAM (44.61 ms).
Batch size 1 (single user, interactive).

First reply: estimate — Arithmetic that also depends on a tunable assumption, listed below.

ttft_s = prompt_tokens / prefill_tok_s

= 32,768 / 1,996.44 = 16.4 s

prompt_tokens: 32768 tokens
prefill_tok_s: 1996.44 tok/s

No published prefill measurement for this hardware; scaled from peak compute (estimate).
Model q_dim unknown — attention FLOPs omitted, so long-prompt TTFT is understated.

Qwen3.8-Flash-Next (Images)

Speed: estimate — Arithmetic that also depends on a tunable assumption, listed below.

tok_s = 1 / (Σ_pool bytes_per_token × share_pool / (bandwidth_pool × efficiency_pool) + Σ_pool share_pool × overhead_pool)

GPU VRAM: 9.535 GB ÷ (640 GB/s × 0.48) + System RAM: 30.24 GB ÷ (307.2 GB/s × 0.7) + 34.3 ms fixed per-token overhead (share-weighted) → 4.86 tok/s

GPU VRAM.bandwidth_gbs: 640 GB/s — component spec
GPU VRAM.efficiency_moe: 0.48 — Settings efficiency.gpu_vram — Discrete GPU: 10 dense + 3 MoE points; fit on published benchmarks, leave-one-out validated — docs/research/12b-estimator-validation.md
GPU VRAM.active_share: 0.2397 — placement
System RAM.bandwidth_gbs: 307.2 GB/s — component spec
System RAM.efficiency_moe: 0.7 — Settings efficiency.server_ddr5 — Server DDR5/MRDIMM single socket: 6 dense + 7 MoE points (small n); fit on published benchmarks, leave-one-out validated — docs/research/12b-estimator-validation.md. The 44.61 ms per-token overhead is a CPU-engine constant, not a property of DDR5: its fit is set by a 128-core Xeon 6980P running llama.cpp (55.3 ms/token on a 3B model) and SGLang (17.0 ms) with the CPU executing every layer (research 12 §6). A GPU spilling into this memory pays it only for the share of each token read from RAM (lib/perf/decode.ts refinement 3).
System RAM.active_share: 0.7603 — placement
per_token_overhead_ms: 34.29514218361304 ms — derived: 0.24 × 1.58 ms (GPU VRAM) + 0.76 × 44.61 ms (System RAM) = 34.3 ms
bytes_per_token_gb: 39.7749 GB

MoE: only active parameters are read per token (SPEC §3).
Effective bits include embeddings and higher-precision tensors in mixed quants.
Assumes active weights are quantized like the file average.
KV cache is re-read on every token (DECISIONS D-008).
Pools are read sequentially per token, so per-pool times add (harmonic combination).
Per-token overhead on a split placement is DERIVED, not measured: each pool's own single-pool figure is charged in proportion to the share of each token it serves, because the fixed cost is per operation and the split puts that share of the operations on each device.
Calibrated with that pool serving the whole model (D-013): GPU VRAM (1.58 ms), System RAM (44.61 ms).
Batch size 1 (single user, interactive).

First reply: estimate — Arithmetic that also depends on a tunable assumption, listed below.

ttft_s = prompt_tokens / prefill_tok_s

= 32,768 / 577.57 = 56.7 s

prompt_tokens: 32768 tokens
prefill_tok_s: 577.57 tok/s

No published prefill measurement for this hardware; scaled from peak compute (estimate).
Model q_dim unknown — attention FLOPs omitted, so long-prompt TTFT is understated.

GLM-5.3-Flash (Code)

Speed: estimate — Arithmetic that also depends on a tunable assumption, listed below.

tok_s = 1 / (Σ_pool bytes_per_token × share_pool / (bandwidth_pool × efficiency_pool) + Σ_pool share_pool × overhead_pool)

GPU VRAM: 3.2 GB ÷ (640 GB/s × 0.48) + System RAM: 5.549 GB ÷ (307.2 GB/s × 0.7) + 28.87 ms fixed per-token overhead (share-weighted) → 15.36 tok/s

GPU VRAM.bandwidth_gbs: 640 GB/s — component spec
GPU VRAM.efficiency_moe: 0.48 — Settings efficiency.gpu_vram — Discrete GPU: 10 dense + 3 MoE points; fit on published benchmarks, leave-one-out validated — docs/research/12b-estimator-validation.md
GPU VRAM.active_share: 0.3657 — placement
System RAM.bandwidth_gbs: 307.2 GB/s — component spec
System RAM.efficiency_moe: 0.7 — Settings efficiency.server_ddr5 — Server DDR5/MRDIMM single socket: 6 dense + 7 MoE points (small n); fit on published benchmarks, leave-one-out validated — docs/research/12b-estimator-validation.md. The 44.61 ms per-token overhead is a CPU-engine constant, not a property of DDR5: its fit is set by a 128-core Xeon 6980P running llama.cpp (55.3 ms/token on a 3B model) and SGLang (17.0 ms) with the CPU executing every layer (research 12 §6). A GPU spilling into this memory pays it only for the share of each token read from RAM (lib/perf/decode.ts refinement 3).
System RAM.active_share: 0.6343 — placement
per_token_overhead_ms: 28.872107066297204 ms — derived: 0.366 × 1.58 ms (GPU VRAM) + 0.634 × 44.61 ms (System RAM) = 28.87 ms
bytes_per_token_gb: 8.7487 GB

MoE: only active parameters are read per token (SPEC §3).
Effective bits include embeddings and higher-precision tensors in mixed quants.
Assumes active weights are quantized like the file average.
KV cache is re-read on every token (DECISIONS D-008).
Pools are read sequentially per token, so per-pool times add (harmonic combination).
layer split across 2 GPUs: capacity adds, per-token bandwidth does not (layers run sequentially)
Per-token overhead on a split placement is DERIVED, not measured: each pool's own single-pool figure is charged in proportion to the share of each token it serves, because the fixed cost is per operation and the split puts that share of the operations on each device.
Calibrated with that pool serving the whole model (D-013): GPU VRAM (1.58 ms), System RAM (44.61 ms).
Batch size 1 (single user, interactive).

First reply: estimate — Arithmetic that also depends on a tunable assumption, listed below.

ttft_s = prompt_tokens / prefill_tok_s

= 32,768 / 1,996.44 = 16.4 s

prompt_tokens: 32768 tokens
prefill_tok_s: 1996.44 tok/s

No published prefill measurement for this hardware; scaled from peak compute (estimate).
Model q_dim unknown — attention FLOPs omitted, so long-prompt TTFT is understated.

Qwen3.8-Flash-Next (Images)

Speed: estimate — Arithmetic that also depends on a tunable assumption, listed below.

tok_s = 1 / (Σ_pool bytes_per_token × share_pool / (bandwidth_pool × efficiency_pool) + Σ_pool share_pool × overhead_pool)

GPU VRAM: 19.069 GB ÷ (640 GB/s × 0.48) + System RAM: 20.706 GB ÷ (307.2 GB/s × 0.7) + 23.98 ms fixed per-token overhead (share-weighted) → 5.48 tok/s

GPU VRAM.bandwidth_gbs: 640 GB/s — component spec
GPU VRAM.efficiency_moe: 0.48 — Settings efficiency.gpu_vram — Discrete GPU: 10 dense + 3 MoE points; fit on published benchmarks, leave-one-out validated — docs/research/12b-estimator-validation.md
GPU VRAM.active_share: 0.4794 — placement
System RAM.bandwidth_gbs: 307.2 GB/s — component spec
System RAM.efficiency_moe: 0.7 — Settings efficiency.server_ddr5 — Server DDR5/MRDIMM single socket: 6 dense + 7 MoE points (small n); fit on published benchmarks, leave-one-out validated — docs/research/12b-estimator-validation.md. The 44.61 ms per-token overhead is a CPU-engine constant, not a property of DDR5: its fit is set by a 128-core Xeon 6980P running llama.cpp (55.3 ms/token on a 3B model) and SGLang (17.0 ms) with the CPU executing every layer (research 12 §6). A GPU spilling into this memory pays it only for the share of each token read from RAM (lib/perf/decode.ts refinement 3).
System RAM.active_share: 0.5206 — placement
per_token_overhead_ms: 23.980284367226076 ms — derived: 0.479 × 1.58 ms (GPU VRAM) + 0.521 × 44.61 ms (System RAM) = 23.98 ms
bytes_per_token_gb: 39.7749 GB

MoE: only active parameters are read per token (SPEC §3).
Effective bits include embeddings and higher-precision tensors in mixed quants.
Assumes active weights are quantized like the file average.
KV cache is re-read on every token (DECISIONS D-008).
Pools are read sequentially per token, so per-pool times add (harmonic combination).
layer split across 2 GPUs: capacity adds, per-token bandwidth does not (layers run sequentially)
Per-token overhead on a split placement is DERIVED, not measured: each pool's own single-pool figure is charged in proportion to the share of each token it serves, because the fixed cost is per operation and the split puts that share of the operations on each device.
Calibrated with that pool serving the whole model (D-013): GPU VRAM (1.58 ms), System RAM (44.61 ms).
Batch size 1 (single user, interactive).

First reply: estimate — Arithmetic that also depends on a tunable assumption, listed below.

ttft_s = prompt_tokens / prefill_tok_s

= 32,768 / 577.57 = 56.7 s

prompt_tokens: 32768 tokens
prefill_tok_s: 577.57 tok/s

No published prefill measurement for this hardware; scaled from peak compute (estimate).
Model q_dim unknown — attention FLOPs omitted, so long-prompt TTFT is understated.

Speeds are tokens a second for each model on this machine, estimated from its memory bandwidth and calibrated against published benchmarks; each one expands below to its formula and inputs.

GLM-5.3-Flash (Code)

Speed: estimate — Arithmetic that also depends on a tunable assumption, listed below.

tok_s = 1 / (Σ_pool bytes_per_token × share_pool / (bandwidth_pool × efficiency_pool) + Σ_pool share_pool × overhead_pool)

GPU VRAM: 3.2 GB ÷ (640 GB/s × 0.48) + System RAM: 5.549 GB ÷ (307.2 GB/s × 0.7) + 28.87 ms fixed per-token overhead (share-weighted) → 15.36 tok/s

GPU VRAM.bandwidth_gbs: 640 GB/s — component spec
GPU VRAM.efficiency_moe: 0.48 — Settings efficiency.gpu_vram — Discrete GPU: 10 dense + 3 MoE points; fit on published benchmarks, leave-one-out validated — docs/research/12b-estimator-validation.md
GPU VRAM.active_share: 0.3657 — placement
System RAM.bandwidth_gbs: 307.2 GB/s — component spec
System RAM.efficiency_moe: 0.7 — Settings efficiency.server_ddr5 — Server DDR5/MRDIMM single socket: 6 dense + 7 MoE points (small n); fit on published benchmarks, leave-one-out validated — docs/research/12b-estimator-validation.md. The 44.61 ms per-token overhead is a CPU-engine constant, not a property of DDR5: its fit is set by a 128-core Xeon 6980P running llama.cpp (55.3 ms/token on a 3B model) and SGLang (17.0 ms) with the CPU executing every layer (research 12 §6). A GPU spilling into this memory pays it only for the share of each token read from RAM (lib/perf/decode.ts refinement 3).
System RAM.active_share: 0.6343 — placement
per_token_overhead_ms: 28.872107066297204 ms — derived: 0.366 × 1.58 ms (GPU VRAM) + 0.634 × 44.61 ms (System RAM) = 28.87 ms
bytes_per_token_gb: 8.7487 GB

MoE: only active parameters are read per token (SPEC §3).
Effective bits include embeddings and higher-precision tensors in mixed quants.
Assumes active weights are quantized like the file average.
KV cache is re-read on every token (DECISIONS D-008).
Pools are read sequentially per token, so per-pool times add (harmonic combination).
layer split across 2 GPUs: capacity adds, per-token bandwidth does not (layers run sequentially)
Per-token overhead on a split placement is DERIVED, not measured: each pool's own single-pool figure is charged in proportion to the share of each token it serves, because the fixed cost is per operation and the split puts that share of the operations on each device.
Calibrated with that pool serving the whole model (D-013): GPU VRAM (1.58 ms), System RAM (44.61 ms).
Batch size 1 (single user, interactive).

First reply: estimate — Arithmetic that also depends on a tunable assumption, listed below.

ttft_s = prompt_tokens / prefill_tok_s

= 32,768 / 1,996.44 = 16.4 s

prompt_tokens: 32768 tokens
prefill_tok_s: 1996.44 tok/s

No published prefill measurement for this hardware; scaled from peak compute (estimate).
Model q_dim unknown — attention FLOPs omitted, so long-prompt TTFT is understated.

Qwen3.8-Flash-Next (Images)

Speed: estimate — Arithmetic that also depends on a tunable assumption, listed below.

tok_s = 1 / (Σ_pool bytes_per_token × share_pool / (bandwidth_pool × efficiency_pool) + Σ_pool share_pool × overhead_pool)

GPU VRAM: 19.069 GB ÷ (640 GB/s × 0.48) + System RAM: 20.706 GB ÷ (307.2 GB/s × 0.7) + 23.98 ms fixed per-token overhead (share-weighted) → 5.48 tok/s

GPU VRAM.bandwidth_gbs: 640 GB/s — component spec
GPU VRAM.efficiency_moe: 0.48 — Settings efficiency.gpu_vram — Discrete GPU: 10 dense + 3 MoE points; fit on published benchmarks, leave-one-out validated — docs/research/12b-estimator-validation.md
GPU VRAM.active_share: 0.4794 — placement
System RAM.bandwidth_gbs: 307.2 GB/s — component spec
System RAM.efficiency_moe: 0.7 — Settings efficiency.server_ddr5 — Server DDR5/MRDIMM single socket: 6 dense + 7 MoE points (small n); fit on published benchmarks, leave-one-out validated — docs/research/12b-estimator-validation.md. The 44.61 ms per-token overhead is a CPU-engine constant, not a property of DDR5: its fit is set by a 128-core Xeon 6980P running llama.cpp (55.3 ms/token on a 3B model) and SGLang (17.0 ms) with the CPU executing every layer (research 12 §6). A GPU spilling into this memory pays it only for the share of each token read from RAM (lib/perf/decode.ts refinement 3).
System RAM.active_share: 0.5206

Qwen3-ASR 1.7B (Speech)

Speed: estimate — Arithmetic that also depends on a tunable assumption, listed below.

tok_s = 1 / (bytes_per_token / (bandwidth × efficiency) + overhead)

GPU VRAM: 5.274 GB ÷ (640 GB/s × 0.86) + 1.58 ms fixed per-token overhead → 89.59 tok/s

GPU VRAM.bandwidth_gbs: 640 GB/s — component spec
GPU VRAM.efficiency: 0.86 — Settings efficiency.gpu_vram — Discrete GPU: 10 dense + 3 MoE points; fit on published benchmarks, leave-one-out validated — docs/research/12b-estimator-validation.md
GPU VRAM.active_share: 1 — placement
per_token_overhead_ms: 1.58 ms — Settings efficiency.gpu_vram.overheadMs — calibrated fit (D-013): Discrete GPU: 10 dense + 3 MoE points; fit on published benchmarks, leave-one-out validated — docs/research/12b-estimator-validation.md
bytes_per_token_gb: 5.2741 GB

Dense: every parameter is read per token.
Effective bits include embeddings and higher-precision tensors in mixed quants.
Assumes active weights are quantized like the file average.
KV cache is re-read on every token (DECISIONS D-008).
layer split across 2 GPUs: capacity adds, per-token bandwidth does not (layers run sequentially)

Your plan needs: estimate — Arithmetic that also depends on a tunable assumption, listed below.

Σ required_gb over every model in the stack

GLM-5.3-Flash 169.74 + Qwen3.8-Flash-Next 129.49 + Qwen3-ASR 1.7B 5.78 = 305 GB

GLM-5.3-Flash: 169.736978848 GB
Qwen3.8-Flash-Next: 129.48808638719998 GB
Qwen3-ASR 1.7B: 5.77887773952 GB

All models loaded at once (Keep all resident).
Not counted, because it is not measured yet: Qwen3.8-Flash-Next reads images through a separate projector file (mmproj); its size is not recorded yet, so the memory shown leaves it out. Qwen3-ASR 1.7B hears audio through a separate projector file (mmproj); its size is not recorded yet, so the memory shown leaves it out.

Memory for AI: derived — Arithmetic on observed inputs only. No tunable assumption enters it.

available_gb = Σ usable_gb over memory pools

GPU VRAM 62.08 + System RAM 243.2 = 305.28 GB

GPU VRAM.usable_gb: 62.08 GB — 64 GB physical
System RAM.usable_gb: 243.2 GB — 256 GB physical

Usable memory is below physical memory for at least one pool (OS/driver reservation; editable in Settings).

To spare: estimate — Arithmetic that also depends on a tunable assumption, listed below.

spare_gb = usable_gb − all-resident need

= 305.28 − 305 = 0.28 GB

usable_gb: 305.28 GB
need_gb: 305.00394297471996 GB

All models loaded at once (Keep all resident).
Not counted, because it is not measured yet: Qwen3.8-Flash-Next reads images through a separate projector file (mmproj); its size is not recorded yet, so the memory shown leaves it out. Qwen3-ASR 1.7B hears audio through a separate projector file (mmproj); its size is not recorded yet, so the memory shown leaves it out.
So the real room to spare is smaller than shown, by the size of what is not counted: the sizes of Qwen3.8-Flash-Next's and Qwen3-ASR 1.7B's projector files (mmproj) are not recorded.

Slowest model (what Fastest and Best value rank on): estimate — Arithmetic that also depends on a tunable assumption, listed below.

slowest = min(decode tok/s) over the Plan's text models

GLM-5.3-Flash 15.4, Qwen3.8-Flash-Next 5.5, Qwen3-ASR 1.7B 89.6 → Qwen3.8-Flash-Next 5.5 tok/s

GLM-5.3-Flash: 15.36277898004981 tok/s — lib/perf decode estimate
Qwen3.8-Flash-Next: 5.48418941023314 tok/s — lib/perf decode estimate
Qwen3-ASR 1.7B: 89.58730496849219 tok/s — lib/perf decode estimate

MoE: only active parameters are read per token (SPEC §3).
Effective bits include embeddings and higher-precision tensors in mixed quants.
Assumes active weights are quantized like the file average.
KV cache is re-read on every token (DECISIONS D-008).
Pools are read sequentially per token, so per-pool times add (harmonic combination).
layer split across 2 GPUs: capacity adds, per-token bandwidth does not (layers run sequentially)
Per-token overhead on a split placement is DERIVED, not measured: each pool's own single-pool figure is charged in proportion to the share of each token it serves, because the fixed cost is per operation and the split puts that share of the operations on each device.
Calibrated with that pool serving the whole model (D-013): GPU VRAM (1.58 ms), System RAM (44.61 ms).

Price per token a second: derived — Arithmetic on observed inputs only. No tunable assumption enters it.

cost_per_tok_s = landed_cost / decode_tok_s

= $7,462 / 5.48 tok/s = $1,360.56/tok/s

landed_cost: 7461.55 USD
decode_tok_s: 5.48418941023314 tok/s

Where each part is bought

ASRock Radeon AI PRO R9700 Creator 32GB (R9700 CT): Newegg (opens in a new tab), shipped, checked 29 d ago, access tier 2, route seed:research
Samsung M321R4GA3BB6-CQK 32GB DDR5-4800 RDIMM: SabrePC (opens in a new tab), shipped, checked 14 h ago, access tier 2, route sabrepc_ngstate
ASRock Rack GENOAD8QM3-2T/BCM: Newegg (opens in a new tab), shipped, checked 1 h ago, access tier 3, route newegg_realtime
Seasonic PRIME PX-2200: Newegg (opens in a new tab), shipped, checked 1 h ago, access tier 3, route newegg_realtime
AMD EPYC 9015: SabrePC (opens in a new tab), shipped, checked 14 h ago, access tier 2, route sabrepc_ngstate
Phanteks Enthoo Pro 2 Server Edition: Newegg (opens in a new tab), shipped, checked 1 h ago, access tier 3, route newegg_realtime
ARCTIC Freezer 4U-SP5: , shipped, checked , access tier 1, route arctic_jsonld

GLM-5.3-Flash (Code)

Speed: estimate — Arithmetic that also depends on a tunable assumption, listed below.

tok_s = 1 / (Σ_pool bytes_per_token × share_pool / (bandwidth_pool × efficiency_pool) + Σ_pool share_pool × overhead_pool)

GPU VRAM: 3.2 GB ÷ (640 GB/s × 0.48) + System RAM: 5.549 GB ÷ (307.2 GB/s × 0.7) + 28.87 ms fixed per-token overhead (share-weighted) → 15.36 tok/s

GPU VRAM.bandwidth_gbs: 640 GB/s — component spec
GPU VRAM.efficiency_moe: 0.48 — Settings efficiency.gpu_vram — Discrete GPU: 10 dense + 3 MoE points; fit on published benchmarks, leave-one-out validated — docs/research/12b-estimator-validation.md
GPU VRAM.active_share: 0.3657 — placement
System RAM.bandwidth_gbs: 307.2 GB/s — component spec
System RAM.efficiency_moe: 0.7 — Settings efficiency.server_ddr5 — Server DDR5/MRDIMM single socket: 6 dense + 7 MoE points (small n); fit on published benchmarks, leave-one-out validated — docs/research/12b-estimator-validation.md. The 44.61 ms per-token overhead is a CPU-engine constant, not a property of DDR5: its fit is set by a 128-core Xeon 6980P running llama.cpp (55.3 ms/token on a 3B model) and SGLang (17.0 ms) with the CPU executing every layer (research 12 §6). A GPU spilling into this memory pays it only for the share of each token read from RAM (lib/perf/decode.ts refinement 3).
System RAM.active_share: 0.6343 — placement
per_token_overhead_ms: 28.872107066297204 ms — derived: 0.366 × 1.58 ms (GPU VRAM) + 0.634 × 44.61 ms (System RAM) = 28.87 ms
bytes_per_token_gb: 8.7487 GB

MoE: only active parameters are read per token (SPEC §3).
Effective bits include embeddings and higher-precision tensors in mixed quants.
Assumes active weights are quantized like the file average.
KV cache is re-read on every token (DECISIONS D-008).
Pools are read sequentially per token, so per-pool times add (harmonic combination).
layer split across 2 GPUs: capacity adds, per-token bandwidth does not (layers run sequentially)
Per-token overhead on a split placement is DERIVED, not measured: each pool's own single-pool figure is charged in proportion to the share of each token it serves, because the fixed cost is per operation and the split puts that share of the operations on each device.
Calibrated with that pool serving the whole model (D-013): GPU VRAM (1.58 ms), System RAM (44.61 ms).
Batch size 1 (single user, interactive).

First reply: estimate — Arithmetic that also depends on a tunable assumption, listed below.

ttft_s = prompt_tokens / prefill_tok_s

= 32,768 / 1,996.44 = 16.4 s

prompt_tokens: 32768 tokens
prefill_tok_s: 1996.44 tok/s

No published prefill measurement for this hardware; scaled from peak compute (estimate).
Model q_dim unknown — attention FLOPs omitted, so long-prompt TTFT is understated.

Qwen3.8-Flash-Next (Images)

Speed: estimate — Arithmetic that also depends on a tunable assumption, listed below.

tok_s = 1 / (Σ_pool bytes_per_token × share_pool / (bandwidth_pool × efficiency_pool) + Σ_pool share_pool × overhead_pool)

GPU VRAM: 19.069 GB ÷ (640 GB/s × 0.48) + System RAM: 20.706 GB ÷ (307.2 GB/s × 0.7) + 23.98 ms fixed per-token overhead (share-weighted) → 5.48 tok/s

GPU VRAM.bandwidth_gbs: 640 GB/s — component spec
GPU VRAM.efficiency_moe: 0.48 — Settings efficiency.gpu_vram — Discrete GPU: 10 dense + 3 MoE points; fit on published benchmarks, leave-one-out validated — docs/research/12b-estimator-validation.md
GPU VRAM.active_share: 0.4794 — placement
System RAM.bandwidth_gbs: 307.2 GB/s — component spec
System RAM.efficiency_moe: 0.7 — Settings efficiency.server_ddr5 — Server DDR5/MRDIMM single socket: 6 dense + 7 MoE points (small n); fit on published benchmarks, leave-one-out validated — docs/research/12b-estimator-validation.md. The 44.61 ms per-token overhead is a CPU-engine constant, not a property of DDR5: its fit is set by a 128-core Xeon 6980P running llama.cpp (55.3 ms/token on a 3B model) and SGLang (17.0 ms) with the CPU executing every layer (research 12 §6). A GPU spilling into this memory pays it only for the share of each token read from RAM (lib/perf/decode.ts refinement 3).
System RAM.active_share: 0.5206 — placement
per_token_overhead_ms: 23.980284367226076 ms — derived: 0.479 × 1.58 ms (GPU VRAM) + 0.521 × 44.61 ms (System RAM) = 23.98 ms
bytes_per_token_gb: 39.7749 GB

MoE: only active parameters are read per token (SPEC §3).
Effective bits include embeddings and higher-precision tensors in mixed quants.
Assumes active weights are quantized like the file average.
KV cache is re-read on every token (DECISIONS D-008).
Pools are read sequentially per token, so per-pool times add (harmonic combination).
layer split across 2 GPUs: capacity adds, per-token bandwidth does not (layers run sequentially)
Per-token overhead on a split placement is DERIVED, not measured: each pool's own single-pool figure is charged in proportion to the share of each token it serves, because the fixed cost is per operation and the split puts that share of the operations on each device.
Calibrated with that pool serving the whole model (D-013): GPU VRAM (1.58 ms), System RAM (44.61 ms).
Batch size 1 (single user, interactive).

First reply: estimate — Arithmetic that also depends on a tunable assumption, listed below.

ttft_s = prompt_tokens / prefill_tok_s

= 32,768 / 577.57 = 56.7 s

prompt_tokens: 32768 tokens
prefill_tok_s: 577.57 tok/s

No published prefill measurement for this hardware; scaled from peak compute (estimate).
Model q_dim unknown — attention FLOPs omitted, so long-prompt TTFT is understated.

Run AI on your own computer.

Step 1:What do you want it to do?

Step 2:The models we'd use

Step 3:What it takes, and machines you can buy

Run AI on your own computer.

Step 1:What do you want it to do?

Step 2:The models we'd use

Code 19 models, ranked by SciCode

Images

Speech

Keep your conversations cached?

Step 3:What it takes, and machines you can buy

Your setup needs 306 GB of memory for AI, and 4.7 or more tokens a second.

Step 2:The models we'd use

Step 3:What it takes, and machines you can buy

Step 2:The models we'd use

Code 19 models, ranked by SciCode

Images

Speech

Keep your conversations cached?

Step 3:What it takes, and machines you can buy

Your setup needs 306 GB of memory for AI, and 4.7 or more tokens a second.

Already have a computer?

1× Radeon AI PRO R9700 32GB (32358-01-20G) · EPYC 9015 · 8× M321R8GA0BB0-CQK 64GB DDR5-4800 RDIMM

1× GeForce RTX 5080 16G SHADOW 3X OC · EPYC 9015 · 8× M321R8GA0BB0-CQK 64GB DDR5-4800 RDIMM

1× GeForce RTX 5080 WINDFORCE OC 16G (GV-N5080WF3OC-16GD) · EPYC 9015 · 8× M321R8GA0BB0-CQK 64GB DDR5-4800 RDIMM

1× Radeon AI PRO R9700 Creator 32GB (R9700 CT) · EPYC 9015 · 8× M321R8GA0BB0-CQK 64GB DDR5-4800 RDIMM

1× AI TOP Radeon AI PRO R9700 32GB (GV-R9700AI TOP-32GD) · EPYC 9015 · 8× M321R8GA0BB0-CQK 64GB DDR5-4800 RDIMM

2× Radeon AI PRO R9700 32GB (32358-01-20G) · EPYC 9015 · 8× M321R4GA3BB6-CQK 32GB DDR5-4800 RDIMM

2× Radeon AI PRO R9700 Creator 32GB (R9700 CT) · EPYC 9015 · 8× M321R4GA3BB6-CQK 32GB DDR5-4800 RDIMM

2× AI TOP Radeon AI PRO R9700 32GB (GV-R9700AI TOP-32GD) · EPYC 9015 · 8× M321R4GA3BB6-CQK 32GB DDR5-4800 RDIMM

Close, but not quite

Link machines together

1× Radeon AI PRO R9700 32GB (32358-01-20G) · EPYC 9015 · 8× M321R8GA0BB0-CQK 64GB DDR5-4800 RDIMM

1× GeForce RTX 5080 16G SHADOW 3X OC · EPYC 9015 · 8× M321R8GA0BB0-CQK 64GB DDR5-4800 RDIMM

1× GeForce RTX 5080 WINDFORCE OC 16G (GV-N5080WF3OC-16GD) · EPYC 9015 · 8× M321R8GA0BB0-CQK 64GB DDR5-4800 RDIMM

1× Radeon AI PRO R9700 Creator 32GB (R9700 CT) · EPYC 9015 · 8× M321R8GA0BB0-CQK 64GB DDR5-4800 RDIMM

1× AI TOP Radeon AI PRO R9700 32GB (GV-R9700AI TOP-32GD) · EPYC 9015 · 8× M321R8GA0BB0-CQK 64GB DDR5-4800 RDIMM

2× Radeon AI PRO R9700 32GB (32358-01-20G) · EPYC 9015 · 8× M321R4GA3BB6-CQK 32GB DDR5-4800 RDIMM

2× Radeon AI PRO R9700 Creator 32GB (R9700 CT) · EPYC 9015 · 8× M321R4GA3BB6-CQK 32GB DDR5-4800 RDIMM

2× AI TOP Radeon AI PRO R9700 32GB (GV-R9700AI TOP-32GD) · EPYC 9015 · 8× M321R4GA3BB6-CQK 32GB DDR5-4800 RDIMM

Close, but not quite