# How solutions are sized

> UpStation catalog · Home › How solutions are sized · https://ai.upstation.io/sizing

Every price on this site is on the [homepage](https://ai.upstation.io/). Each product type and each product has its own page with the full details (spec, options, Intel/AMD equivalents, power and colo): [GPU servers](https://ai.upstation.io/gpu-servers) · [GPU hosts without GPUs](https://ai.upstation.io/gpu-hosts) · [Switches, cables and optics](https://ai.upstation.io/switches) · [Compute nodes](https://ai.upstation.io/compute-nodes) · [Storage nodes](https://ai.upstation.io/storage-nodes) · [Parts rate card](https://ai.upstation.io/rate-card) · [Power and colo](https://ai.upstation.io/power) · [How solutions are sized](https://ai.upstation.io/sizing) · [Reference builds and worked designs](https://ai.upstation.io/designs). AI assistants: the map is [llms.txt](https://ai.upstation.io/llms.txt); the data is [catalog.json](https://ai.upstation.io/catalog.json); every page is also Markdown (append .md, or send Accept: text/markdown).

Prices in USD as of 2026-09-09; final price confirmed on order. Continental US; Canada quoted case by case. Orders and quotes: [book a meeting](https://calendly.com/alecglover/intro-call). Every page: [llms.txt](https://ai.upstation.io/llms.txt) (map) · [homepage](https://ai.upstation.io/) (all tables) · [catalog.json](https://ai.upstation.io/catalog.json) (data).

UpStation sizes a deployment in this order: GPUs → nodes → networks → storage → racks and power → bill of materials. The rules below are planning conventions (the same data is in catalog.json under sizing); figures marked estimate are not vendor specifications. Two worked designs computed with these rules are listed under Reference builds.

### Before sizing, settle

- Workload: training, fine-tuning, inference, HPC (default training when 'AI at scale' is implied)
- GPU count or node count (8 GPUs per node), or model size and precision to derive it
- Parallelism: single GPU, multi-GPU in one node, multi-node (decides HGX vs PCIe and whether a GPU fabric exists)
- Growth: day-1 full (default) or +N GPUs over a horizon
- Storage: usable TB now and at horizon, dataset character (text, compressed image/video, uncompressed vision), checkpoint size and pause, protection level
- Site: per-rack kW limit, rack space, whether racks/PDUs/whips exist (customer, colo, or to be supplied)
- Colo term (1, 3 or 5 years) and whether InfiniBand is a hard requirement (not listed; 800G RoCE Ethernet at 1:1 usually satisfies the intent)

### GPUs to nodes

- Size a GPU cluster on accelerators (GPU count and aggregate GPU memory), never on host cores. Every listed GPU server has 8 GPUs: nodes = ceil(GPUs / 8). Nodes ship fully populated; partial population is not offered.
- Model memory: required GPU memory (GB) = parameters (billions) × bytes per parameter × 1.2 (weights plus overhead) + KV cache (grows with context length × batch). Bytes per parameter: FP16/BF16 2, FP8 1, INT4/AWQ 0.5. GPU count = ceil(required GB / GB per GPU), rounded up to a tensor-parallel width of 1, 2, 4 or 8 inside one node. Quantization is the biggest lever on GPU count: 70B at FP16 is about 168 GB (2× H200, or 1× B300); 70B at FP8 is about 84 GB (1× H200 or 1× RTX PRO 6000 for weights, plus KV cache).
- GPU memory per card: NVIDIA B300 (HGX NVL8) 288 GB, NVIDIA H200 (HGX) 141 GB, NVIDIA RTX PRO 6000 Server Edition 96 GB, NVIDIA RTX 5090 (passive server edition) 32 GB.
- HGX or PCIe: Does the model at its precision fit one GPU's memory, and can each GPU work alone? Yes: PCIe. No: HGX. Distributed training is never served by PCIe servers. B300 and H200 (NVLink, HGX): the GPUs must act as one (model does not fit one GPU, tensor or pipeline parallel, all-reduce across GPUs or nodes; distributed training at any node count). RTX PRO 6000 and RTX 5090 (PCIe): GPUs run independently (inference replicas that each fit one card, batch jobs, VDI, rendering); no GPU-to-GPU fabric and no back-end network by default.
- What an HGX cluster carries: An HGX cluster of 2 or more nodes carries a dedicated non-blocking GPU fabric (1 port per GPU), a storage tier sized to feed the GPUs, and up to 2× the power per node of a PCIe server; a PCIe inference deployment carries neither the fabric nor the parallel-file tier.
- Host memory: Platform standard (the listed base build, 1–2 TB on an 8-GPU node) is the default. Host RAM ≥ aggregate GPU memory (up to 1.5–2×) is a costed option taken only for CPU/NVMe offload, unified-memory oversubscription or host-staged checkpointing; price it with the rate card and snap to whole DIMM fills (16 or 32 DIMMs). CPUs: Report the listed CPUs as an assumption; they are not a sizing input. Local NVMe: Several TB of local NVMe per node for scratch and checkpoints (listed options); datasets live on shared storage.

### Networks

| Plane | Carries | Ports per node | Oversubscription |
|---|---|---|---|
| GPU fabric | Collectives only (all-reduce, GPUDirect RDMA); never storage | 1 port per GPU at the GPU generation's speed: H200 400G (listed option, 8× 400G), B300 800G (on board, 8× 800G). None on PCIe GPU servers unless the buyer asks | 1:1 non-blocking |
| Front-end (converged) | Client access, orchestration, in-band management, and storage unless a storage plane is dedicated | 2 ports per node, one to each switch of a pair (25G floor; 100G or 200G when the base build or option provides it) | 3:1 (storage-facing 4:3 or better) |
| Storage (dedicated) | Parallel file system or NAS data path at scale | 2 ports per storage node and per GPU node on the storage plane, only when that plane is dedicated (see 'Dedicated storage plane when'); otherwise storage rides the front-end plane | 1:1 |
| OOB | BMC and switch management | 1× 1G BMC port per node plus 1× 1G management port per switch, on a 48× 1G switch (1 per rack or per 48 ports) | n/a |

- One switch: One switch is enough when the plane's port demand fits one switch (GPU fabric: ≤ 32 or ≤ 64 GPU ports on the 32- or 64-port 800G/400G switches). Note it is a single point of failure; a leaf-spine pair or fat-tree removes it.
- Fat-tree: R-port switch at the host speed: R/2 ports down to hosts, R/2 up to spines. leaves = ceil(host ports ÷ (R/2)); uplinks = leaves × R/2; spines = max(2, ceil(uplinks ÷ R)); each spine port terminates one uplink. Rail-optimized: Default for GPU fabrics: 8 rails (one per GPU position); leaves per rail = ceil(nodes ÷ (R/2)); leaves = 8 × leaves per rail; hosts connect to leaves in a network rack (20 m links, optical — see the cable-medium rule under Links), leaves to spines at 20–50 m. Examples with 64-port switches: 8 nodes (64 GPU ports) → 1 switch or 2 leaves + 2 spines; 32 nodes (256 ports) → 8 leaves + 4 spines, 256 host links + 256 uplinks; 128 nodes (1,024 ports) → 32 leaves + 16 spines. Three tiers: When leaves > R a 2-tier fat-tree cannot reach every leaf: pods of R/2 leaves with their own spines and a super-spine tier (above about 1,000–2,000 GPUs on 64-port switches).
- Front-end leaf-spine: Access leaves (48× 25G + 8× 100G): 48 host ports; uplinks = ceil(48 × 25 ÷ 3 ÷ 100) = 4× 100G per leaf at 3:1; spines = max(2, ceil(uplinks ÷ spine ports)). A single MLAG pair (2 leaves) or two pairs (4 leaves, full-mesh 4-link square between pairs) needs no spine; 3 or more pairs go leaf-spine. Peer link: Each switch pair reserves 2 ports at its uplink speed for the peer link (in-rack DAC) plus a 1G heartbeat over OOB.
- Dedicated storage plane when: Sustained storage traffic approaches 40 GB/s per node or several hundred GB/s in aggregate (parallel file system at scale); otherwise storage nodes and GPU-node storage traffic share the front-end plane, with PFC/ECN enabled and storage-facing oversubscription 4:3 or better. A port-speed mismatch (100G storage node on a 200G switch) is a breakout, not a reason to split. The GPU fabric never carries storage: checkpoint flows head-of-line-block collectives and need different congestion tuning; storage must scale independently of GPU count.
- Links: 1 cable per link: host to leaf, leaf to spine, peer links. Transceivers: DAC/AOC 0; fibre point-to-point 2 per link (both ends); breakout 1 parent + N children. Default lengths: host to leaf 10–20 m (leaves sit in a network rack; 2 m only when a node genuinely shares the leaf's rack), leaf to spine 10–30 m; pick the medium from the cables table by reach. Breakout: 800G→2×400G, 400G→4×100G, 200G→2×100G, 100G→4×25G: one breakout cable connects N slower endpoints to one switch port; cages used = ceil(endpoints ÷ N).
- Cable medium for estimates: GPU node density forces inter-rack cabling: 1–3 HGX nodes fill a 40 kW rack, and a 3 m passive DAC reaches at most the adjacent rack (switch mid-rack of a 3-rack span). Even the best case — one 64-port switch fully serving 8 nodes (8 ports each) — would need about 3 nodes per rack inside that span, beyond normal datacenter power densities; a fat-tree spreads nodes further still. So estimate every GPU-fabric link (host to leaf and leaf to spine) as optical — 2× short-reach optics + multimode fibre, or AOC — never as passive DAC. Mixing copper in is entirely realistic: which links can drop to passive DAC (or 5 m AEC) depends on how racks, switches and elevations are actually installed — switch peer links, co-racked leaf↔spine, and nodes in or adjacent to the leaf's rack all qualify, and 25% of fabric cables landing as copper DAC is a reasonable minimum expectation. Take that as a savings at engineering time, not in the estimate.

### Storage

- Capture: Usable capacity needed (current + growth, TB), throughput (GB/s), protection level, media, protocol (file for AI), checkpoint size and pause, dataset size; raw capacity is a design output.
- Read demand (aggregate GB/s per GPU): LLM/text 0.15; compressed image/video 0.25–0.5; uncompressed vision/multimodal 0.5–0.8; cap 0.8. Aggregate cluster-sustained rates (published pod reference tiers). Never multiply a single-node peak (4–5 GB/s per GPU) by the cluster's GPU count; a 2,048-GPU pod needs about 1 TB/s, not 8–10 TB/s.
- Write demand: max(0.5 × read GB/s, checkpoint burst). Checkpoint burst GB/s = parameters (billions) × 16 GB ÷ acceptable pause (s); full mixed-precision optimizer state is about 16 bytes per parameter. Asynchronous checkpointing drops the sustained write need to 50–200 GB/s regardless of model size.
- Capacity: dataset + 3–5 retained checkpoints × (parameters (billions) × 16 GB ÷ 1,000).
- Plannable capacity, per-node RAID: plannable per node = raw × RAID efficiency × 0.85 (fill ceiling). A node is one failure domain: losing it takes its data offline unless the software also replicates across nodes. Distributed (Ceph): plannable = raw × (N − 1) ÷ N (self-heal reserve) × efficiency × 0.85; hosts ≥ k + m for erasure coding (≥ 3 for replica 3), plus 1 recommended; parity m = 2 by default; k ≤ 8. Scale-out NAS: usable = raw × efficiency (N + 2 across nodes: 0.6 at 5 nodes, 0.8 at 10, 0.89 at 16+) − 1 virtual hot-spare drive per node, × 0.97 file-system overhead, then × 0.85 fill. Fill ceiling 0.85 in every case.
- Node count: nodes = max(ceil(read GB/s ÷ per-node GB/s), ceil(capacity ÷ per-node plannable), protection floor). No extra headroom nodes; ceil() already rounds up. If plannable capacity exceeds 2× the need, the feed rate is a single-node peak or the tier is wrong: re-derive.

| Protection | Efficiency (usable ÷ raw before fill ceiling) |
|---|---|
| Replica 3 | 0.333 |
| EC 2+2 | 0.5 |
| EC 4+2 | 0.667 |
| EC 8+2 | 0.8 |
| EC 8+3 | 0.727 |
| EC 8+4 | 0.667 |
| RAID 6 10+2 | 0.833 |
| RAID 6 8+2 | 0.8 |
| RAID 6 6+2 | 0.75 |
| RAID 10 | 0.5 |

| Storage node | Raw TB | Ports | Line rate GB/s | Planning GB/s per node | RAID 6 10+2 plannable TB per node | Note |
|---|---|---|---|---|---|---|
| 2U all-NVMe storage node, dual socket, 24x 7.68TB NVMe (184TB raw) | 184 | 2× 100G (4 with the listed option) | 25 | 20 | 130.3 | line rate 2 × 100 Gb/s ÷ 8 = 25 GB/s; plan 20 GB/s per node (estimate, software dependent); 50 GB/s line rate with 4× 100G |
| 2U HDD storage node, single socket, 24x 24TB SAS (576TB raw) | 576 | 2× 100G | 25 | 5 | 407.8 | 24 nearline drives at about 250 MB/s each ≈ 6 GB/s streaming (estimate); plan 5 GB/s; add NVMe DB/WAL (1–4% of HDD capacity) via the rate card for Ceph |

- Ceph node rules: All-NVMe: 1 OSD per drive, 4–6 cores and 8 GB RAM per OSD, 2× 100G; HDD: 1 core and 5 GB RAM per OSD plus 16 GB base, 1 NVMe DB/WAL per 6 HDDs, 2× 25G or 100G; 3 monitors (5 at scale), 2 managers, ≥ 2 metadata servers; public and cluster networks, cluster ≈ public × replica size.
- Tiers: Aggregate read below about 1 TB/s: scale-out NAS or Ceph-class file on the listed storage nodes. 1–4 TB/s: still NAS-class by default; a parallel file system (WEKA, BeeGFS, Lustre, IBM Storage Scale) converged on the GPU nodes' NVMe or on ≥ 8 NVMe nodes when the buyer wants one. Above about 4 TB/s: a standalone parallel file system on all-NVMe nodes. Storage software is not in this catalog: the buyer supplies and supports it (Ceph, BeeGFS, WEKA, ZFS/NFS, ...); UpStation quotes the nodes.
- GPU nodes and storage: GPU nodes reach storage over the front-end plane (converged) or a dedicated storage plane; the GPU fabric never carries storage.

### Racks, power and management

- nodes per rack = min(floor(40 U ÷ RU per node), floor(rack kW ÷ provisioned kW per node)). Rack kW = the facility's stated limit, else 40 kW (a common air-cooled compute-rack envelope). Report whether power or space binds. Every listed server is air-cooled.
- Feeds: A + B feeds, each alone carrying the whole rack; per-rack circuits = smallest standard circuit whose rated kW ≥ rack provisioned kW, ×2, or several pairs when a rack exceeds one circuit.
- Placement: Leaves for GPU fabrics sit in a network rack (or mid-row); front-end pair and OOB switch in the rack they serve; spines in a central rack; storage nodes in their own racks.
- Management: Clusters of 4 or more GPU nodes add 3 management nodes (the listed 1U compute node: provisioning/PXE, cluster manager, scheduler control plane, monitoring; 2 for an active/passive pair at small scale, 1 for a lab) and one 48× 1G OOB switch per rack; the management nodes attach to the front-end plane.

| GPU server | RU | Provisioned kW | Per 40 kW rack | Rack kW | Binds on |
|---|---|---|---|---|---|
| 8x NVIDIA B300 (HGX B300 NVL8), Intel | 8 | 16.96 | 2 | 33.92 | power |
| 8x NVIDIA H200 (HGX H200), Intel | 8 | 11.34 | 3 | 34.02 | power |
| 8x NVIDIA RTX PRO 6000 Blackwell Server Edition, Intel | 4 | 8.47 | 4 | 33.88 | power |
| 8x NVIDIA RTX PRO 6000 Blackwell Server Edition, AMD | 4 | 8.37 | 4 | 33.48 | power |
| 8x NVIDIA RTX 5090 (passive server edition), AMD | 4 | 7.99 | 5 | 39.95 | power |
| 8-GPU host, Tier 1 (Supermicro), PCIe 5.0, Intel Xeon 6 (TDX) | 4 | 1.38 | 10 | 13.8 | space |
| 8-GPU host, Tier 1 (Supermicro), PCIe 5.0, Intel 5th Gen Xeon (TDX) | 5 | 2.19 | 8 | 17.52 | space |
| 8-GPU host, Tier 1 (Supermicro), PCIe 5.0, AMD | 5 | 1.25 | 8 | 10.0 | space |
| 8-GPU host, Tier 2 (Octoserver), PCIe 5.0, AMD | 4 | 1.31 | 10 | 13.1 | space |
| 8-GPU host, Tier 2 (Octoserver), PCIe 4.0 budget, AMD | 4 | 1.34 | 10 | 13.4 | space |

Generated 2026-09-22 from UpStation's catalog data; prices as of 2026-09-09; re-checked monthly. Book a meeting: https://calendly.com/alecglover/intro-call. Map of every page: https://ai.upstation.io/llms.txt