NVIDIA DGX Station GB300 (Grace Blackwell Ultra), 748GB

Writing speed: estimate — Arithmetic that also depends on a tunable assumption, listed below.

tok_s = 1 / (bytes_per_token / (bandwidth × efficiency) + overhead)

GPU HBM3e: 8.749 GB ÷ (7,100 GB/s × 0.48) + 1.58 ms fixed per-token overhead → 241.13 tok/s

GPU HBM3e.bandwidth_gbs

7100 GB/s — component spec

GPU HBM3e.efficiency_moe

0.48 — Settings efficiency.gpu_vram — Discrete GPU: 10 dense + 3 MoE points; fit on published benchmarks, leave-one-out validated — docs/research/12b-estimator-validation.md

GPU HBM3e.active_share

1 — placement

per_token_overhead_ms

1.58 ms — Settings efficiency.gpu_vram.overheadMs — calibrated fit (D-013): Discrete GPU: 10 dense + 3 MoE points; fit on published benchmarks, leave-one-out validated — docs/research/12b-estimator-validation.md

bytes_per_token_gb

8.7487 GB

• MoE: only active parameters are read per token (SPEC §3).
• Effective bits include embeddings and higher-precision tensors in mixed quants.
• Assumes active weights are quantized like the file average.
• KV cache is re-read on every token (DECISIONS D-008).
• Batch size 1 (single user, interactive).

First reply: estimate — Arithmetic that also depends on a tunable assumption, listed below.

ttft_s = prompt_tokens / prefill_tok_s

= 32,768 / 52,262.73 = 0.6 s

prompt_tokens

32768 tokens

prefill_tok_s

52262.73 tok/s

• No published prefill measurement for this hardware; scaled from peak compute (estimate).
• Model q_dim unknown — attention FLOPs omitted, so long-prompt TTFT is understated.

Writing speed: estimate — Arithmetic that also depends on a tunable assumption, listed below.

tok_s = 1 / (bytes_per_token / (bandwidth × efficiency) + overhead)

GPU HBM3e: 39.775 GB ÷ (7,100 GB/s × 0.48) + 1.58 ms fixed per-token overhead → 75.47 tok/s

GPU HBM3e.bandwidth_gbs

7100 GB/s — component spec

GPU HBM3e.efficiency_moe

0.48 — Settings efficiency.gpu_vram — Discrete GPU: 10 dense + 3 MoE points; fit on published benchmarks, leave-one-out validated — docs/research/12b-estimator-validation.md

GPU HBM3e.active_share

1 — placement

per_token_overhead_ms

1.58 ms — Settings efficiency.gpu_vram.overheadMs — calibrated fit (D-013): Discrete GPU: 10 dense + 3 MoE points; fit on published benchmarks, leave-one-out validated — docs/research/12b-estimator-validation.md

bytes_per_token_gb

39.7749 GB

• MoE: only active parameters are read per token (SPEC §3).
• Effective bits include embeddings and higher-precision tensors in mixed quants.
• Assumes active weights are quantized like the file average.
• KV cache is re-read on every token (DECISIONS D-008).
• Batch size 1 (single user, interactive).

First reply: estimate — Arithmetic that also depends on a tunable assumption, listed below.

ttft_s = prompt_tokens / prefill_tok_s

= 32,768 / 15,119.69 = 2.2 s

prompt_tokens

32768 tokens

prefill_tok_s

15119.69 tok/s

• No published prefill measurement for this hardware; scaled from peak compute (estimate).
• Model q_dim unknown — attention FLOPs omitted, so long-prompt TTFT is understated.

Writing speed: estimate — Arithmetic that also depends on a tunable assumption, listed below.

tok_s = 1 / (bytes_per_token / (bandwidth × efficiency) + overhead)

GPU HBM3e: 5.274 GB ÷ (7,100 GB/s × 0.86) + 1.58 ms fixed per-token overhead → 409.21 tok/s

GPU HBM3e.bandwidth_gbs

7100 GB/s — component spec

GPU HBM3e.efficiency

0.86 — Settings efficiency.gpu_vram — Discrete GPU: 10 dense + 3 MoE points; fit on published benchmarks, leave-one-out validated — docs/research/12b-estimator-validation.md

GPU HBM3e.active_share

1 — placement

per_token_overhead_ms

1.58 ms — Settings efficiency.gpu_vram.overheadMs — calibrated fit (D-013): Discrete GPU: 10 dense + 3 MoE points; fit on published benchmarks, leave-one-out validated — docs/research/12b-estimator-validation.md

bytes_per_token_gb

5.2741 GB

• Dense: every parameter is read per token.
• Effective bits include embeddings and higher-precision tensors in mixed quants.
• Assumes active weights are quantized like the file average.
• KV cache is re-read on every token (DECISIONS D-008).
• Batch size 1 (single user, interactive).

First reply: estimate — Arithmetic that also depends on a tunable assumption, listed below.

ttft_s = prompt_tokens / prefill_tok_s

= 30,000 / 508,550.93 = 0.1 s

prompt_tokens

30000 tokens

prefill_tok_s

508550.93 tok/s

• No published prefill measurement for this hardware; scaled from peak compute (estimate).
• Model q_dim unknown — attention FLOPs omitted, so long-prompt TTFT is understated.

Every model loaded: estimate — Arithmetic that also depends on a tunable assumption, listed below.

Σ required_gb over every model in the stack

GLM-5.3-Flash 169.74 + Qwen3.8-Flash-Next 129.49 + Qwen3-ASR 1.7B 5.78 = 305 GB

GLM-5.3-Flash

169.736978848 GB

Qwen3.8-Flash-Next

129.48808638719998 GB

Qwen3-ASR 1.7B

5.77887773952 GB

• All models loaded at once (Keep all resident).
• Not counted, because it is not measured yet: Qwen3.8-Flash-Next reads images through a separate projector file (mmproj); its size is not recorded yet, so the memory shown leaves it out. Qwen3-ASR 1.7B hears audio through a separate projector file (mmproj); its size is not recorded yet, so the memory shown leaves it out.

Memory for AI: derived — Arithmetic on observed inputs only. No tunable assumption enters it.

available_gb = Σ usable_gb over memory pools

GPU HBM3e 244.44 + CPU LPDDR5X 446.4 = 690.84 GB

GPU HBM3e.usable_gb

244.44 GB — 252 GB physical

CPU LPDDR5X.usable_gb

446.40000000000003 GB — 496 GB physical

• Usable memory is below physical memory for at least one pool (OS/driver reservation; editable in Settings).

To spare: estimate — Arithmetic that also depends on a tunable assumption, listed below.

spare_gb = usable_gb − all-resident need

= 690.84 − 305 = 385.84 GB

usable_gb

690.84 GB

need_gb

305.00394297471996 GB

• All models loaded at once (Keep all resident).
• Not counted, because it is not measured yet: Qwen3.8-Flash-Next reads images through a separate projector file (mmproj); its size is not recorded yet, so the memory shown leaves it out. Qwen3-ASR 1.7B hears audio through a separate projector file (mmproj); its size is not recorded yet, so the memory shown leaves it out.
• So the real room to spare is smaller than shown, by the size of what is not counted: the sizes of Qwen3.8-Flash-Next's and Qwen3-ASR 1.7B's projector files (mmproj) are not recorded.