How solutions are sized

Prices in USD as of 2026-09-09; final price confirmed on order. Continental US; Canada quoted case by case. Orders and quotes: book a meeting. Every page: llms.txt (map) · homepage (all tables) · catalog.json (data).

UpStation sizes a deployment in this order: GPUs → nodes → networks → storage → racks and power → bill of materials. The rules below are planning conventions (the same data is in catalog.json under sizing); figures marked estimate are not vendor specifications. Two worked designs computed with these rules are listed under Reference builds.

Before sizing, settle

• Workload: training, fine-tuning, inference, HPC (default training when 'AI at scale' is implied)
• GPU count or node count (8 GPUs per node), or model size and precision to derive it
• Parallelism: single GPU, multi-GPU in one node, multi-node (decides HGX vs PCIe and whether a GPU fabric exists)
• Growth: day-1 full (default) or +N GPUs over a horizon
• Storage: usable TB now and at horizon, dataset character (text, compressed image/video, uncompressed vision), checkpoint size and pause, protection level
• Site: per-rack kW limit, rack space, whether racks/PDUs/whips exist (customer, colo, or to be supplied)
• Colo term (1, 3 or 5 years) and whether InfiniBand is a hard requirement (not listed; 800G RoCE Ethernet at 1:1 usually satisfies the intent)

GPUs to nodes

• Size a GPU cluster on accelerators (GPU count and aggregate GPU memory), never on host cores. Every listed GPU server has 8 GPUs: nodes = ceil(GPUs / 8). Nodes ship fully populated; partial population is not offered.
• Model memory: required GPU memory (GB) = parameters (billions) × bytes per parameter × 1.2 (weights plus overhead) + KV cache (grows with context length × batch). Bytes per parameter: FP16/BF16 2, FP8 1, INT4/AWQ 0.5. GPU count = ceil(required GB / GB per GPU), rounded up to a tensor-parallel width of 1, 2, 4 or 8 inside one node. Quantization is the biggest lever on GPU count: 70B at FP16 is about 168 GB (2× H200, or 1× B300); 70B at FP8 is about 84 GB (1× H200 or 1× RTX PRO 6000 for weights, plus KV cache).
• GPU memory per card: NVIDIA B300 (HGX NVL8) 288 GB, NVIDIA H200 (HGX) 141 GB, NVIDIA RTX PRO 6000 Server Edition 96 GB, NVIDIA RTX 5090 (passive server edition) 32 GB.
• HGX or PCIe: Does the model at its precision fit one GPU's memory, and can each GPU work alone? Yes: PCIe. No: HGX. Distributed training is never served by PCIe servers. B300 and H200 (NVLink, HGX): the GPUs must act as one (model does not fit one GPU, tensor or pipeline parallel, all-reduce across GPUs or nodes; distributed training at any node count). RTX PRO 6000 and RTX 5090 (PCIe): GPUs run independently (inference replicas that each fit one card, batch jobs, VDI, rendering); no GPU-to-GPU fabric and no back-end network by default.
• What an HGX cluster carries: An HGX cluster of 2 or more nodes carries a dedicated non-blocking GPU fabric (1 port per GPU), a storage tier sized to feed the GPUs, and up to 2× the power per node of a PCIe server; a PCIe inference deployment carries neither the fabric nor the parallel-file tier.
• Host memory: Platform standard (the listed base build, 1–2 TB on an 8-GPU node) is the default. Host RAM ≥ aggregate GPU memory (up to 1.5–2×) is a costed option taken only for CPU/NVMe offload, unified-memory oversubscription or host-staged checkpointing; price it with the rate card and snap to whole DIMM fills (16 or 32 DIMMs). CPUs: Report the listed CPUs as an assumption; they are not a sizing input. Local NVMe: Several TB of local NVMe per node for scratch and checkpoints (listed options); datasets live on shared storage.

Networks

• One switch: One switch is enough when the plane's port demand fits one switch (GPU fabric: ≤ 32 or ≤ 64 GPU ports on the 32- or 64-port 800G/400G switches). Note it is a single point of failure; a leaf-spine pair or fat-tree removes it.
• Fat-tree: R-port switch at the host speed: R/2 ports down to hosts, R/2 up to spines. leaves = ceil(host ports ÷ (R/2)); uplinks = leaves × R/2; spines = max(2, ceil(uplinks ÷ R)); each spine port terminates one uplink. Rail-optimized: Default for GPU fabrics: 8 rails (one per GPU position); leaves per rail = ceil(nodes ÷ (R/2)); leaves = 8 × leaves per rail; hosts connect to leaves in a network rack (20 m links, optical — see the cable-medium rule under Links), leaves to spines at 20–50 m. Examples with 64-port switches: 8 nodes (64 GPU ports) → 1 switch or 2 leaves + 2 spines; 32 nodes (256 ports) → 8 leaves + 4 spines, 256 host links + 256 uplinks; 128 nodes (1,024 ports) → 32 leaves + 16 spines. Three tiers: When leaves > R a 2-tier fat-tree cannot reach every leaf: pods of R/2 leaves with their own spines and a super-spine tier (above about 1,000–2,000 GPUs on 64-port switches).
• Front-end leaf-spine: Access leaves (48× 25G + 8× 100G): 48 host ports; uplinks = ceil(48 × 25 ÷ 3 ÷ 100) = 4× 100G per leaf at 3:1; spines = max(2, ceil(uplinks ÷ spine ports)). A single MLAG pair (2 leaves) or two pairs (4 leaves, full-mesh 4-link square between pairs) needs no spine; 3 or more pairs go leaf-spine. Peer link: Each switch pair reserves 2 ports at its uplink speed for the peer link (in-rack DAC) plus a 1G heartbeat over OOB.
• Dedicated storage plane when: Sustained storage traffic approaches 40 GB/s per node or several hundred GB/s in aggregate (parallel file system at scale); otherwise storage nodes and GPU-node storage traffic share the front-end plane, with PFC/ECN enabled and storage-facing oversubscription 4:3 or better. A port-speed mismatch (100G storage node on a 200G switch) is a breakout, not a reason to split. The GPU fabric never carries storage: checkpoint flows head-of-line-block collectives and need different congestion tuning; storage must scale independently of GPU count.
• Links: 1 cable per link: host to leaf, leaf to spine, peer links. Transceivers: DAC/AOC 0; fibre point-to-point 2 per link (both ends); breakout 1 parent + N children. Default lengths: host to leaf 10–20 m (leaves sit in a network rack; 2 m only when a node genuinely shares the leaf's rack), leaf to spine 10–30 m; pick the medium from the cables table by reach. Breakout: 800G→2×400G, 400G→4×100G, 200G→2×100G, 100G→4×25G: one breakout cable connects N slower endpoints to one switch port; cages used = ceil(endpoints ÷ N).
• Cable medium for estimates: GPU node density forces inter-rack cabling: 1–3 HGX nodes fill a 40 kW rack, and a 3 m passive DAC reaches at most the adjacent rack (switch mid-rack of a 3-rack span). Even the best case — one 64-port switch fully serving 8 nodes (8 ports each) — would need about 3 nodes per rack inside that span, beyond normal datacenter power densities; a fat-tree spreads nodes further still. So estimate every GPU-fabric link (host to leaf and leaf to spine) as optical — 2× short-reach optics + multimode fibre, or AOC — never as passive DAC. Mixing copper in is entirely realistic: which links can drop to passive DAC (or 5 m AEC) depends on how racks, switches and elevations are actually installed — switch peer links, co-racked leaf↔spine, and nodes in or adjacent to the leaf's rack all qualify, and 25% of fabric cables landing as copper DAC is a reasonable minimum expectation. Take that as a savings at engineering time, not in the estimate.

Storage

• Capture: Usable capacity needed (current + growth, TB), throughput (GB/s), protection level, media, protocol (file for AI), checkpoint size and pause, dataset size; raw capacity is a design output.
• Read demand (aggregate GB/s per GPU): LLM/text 0.15; compressed image/video 0.25–0.5; uncompressed vision/multimodal 0.5–0.8; cap 0.8. Aggregate cluster-sustained rates (published pod reference tiers). Never multiply a single-node peak (4–5 GB/s per GPU) by the cluster's GPU count; a 2,048-GPU pod needs about 1 TB/s, not 8–10 TB/s.
• Write demand: max(0.5 × read GB/s, checkpoint burst). Checkpoint burst GB/s = parameters (billions) × 16 GB ÷ acceptable pause (s); full mixed-precision optimizer state is about 16 bytes per parameter. Asynchronous checkpointing drops the sustained write need to 50–200 GB/s regardless of model size.
• Capacity: dataset + 3–5 retained checkpoints × (parameters (billions) × 16 GB ÷ 1,000).
• Plannable capacity, per-node RAID: plannable per node = raw × RAID efficiency × 0.85 (fill ceiling). A node is one failure domain: losing it takes its data offline unless the software also replicates across nodes. Distributed (Ceph): plannable = raw × (N − 1) ÷ N (self-heal reserve) × efficiency × 0.85; hosts ≥ k + m for erasure coding (≥ 3 for replica 3), plus 1 recommended; parity m = 2 by default; k ≤ 8. Scale-out NAS: usable = raw × efficiency (N + 2 across nodes: 0.6 at 5 nodes, 0.8 at 10, 0.89 at 16+) − 1 virtual hot-spare drive per node, × 0.97 file-system overhead, then × 0.85 fill. Fill ceiling 0.85 in every case.
• Node count: nodes = max(ceil(read GB/s ÷ per-node GB/s), ceil(capacity ÷ per-node plannable), protection floor). No extra headroom nodes; ceil() already rounds up. If plannable capacity exceeds 2× the need, the feed rate is a single-node peak or the tier is wrong: re-derive.

• Ceph node rules: All-NVMe: 1 OSD per drive, 4–6 cores and 8 GB RAM per OSD, 2× 100G; HDD: 1 core and 5 GB RAM per OSD plus 16 GB base, 1 NVMe DB/WAL per 6 HDDs, 2× 25G or 100G; 3 monitors (5 at scale), 2 managers, ≥ 2 metadata servers; public and cluster networks, cluster ≈ public × replica size.
• Tiers: Aggregate read below about 1 TB/s: scale-out NAS or Ceph-class file on the listed storage nodes. 1–4 TB/s: still NAS-class by default; a parallel file system (WEKA, BeeGFS, Lustre, IBM Storage Scale) converged on the GPU nodes' NVMe or on ≥ 8 NVMe nodes when the buyer wants one. Above about 4 TB/s: a standalone parallel file system on all-NVMe nodes. Storage software is not in this catalog: the buyer supplies and supports it (Ceph, BeeGFS, WEKA, ZFS/NFS, ...); UpStation quotes the nodes.
• GPU nodes and storage: GPU nodes reach storage over the front-end plane (converged) or a dedicated storage plane; the GPU fabric never carries storage.

Racks, power and management

• nodes per rack = min(floor(40 U ÷ RU per node), floor(rack kW ÷ provisioned kW per node)). Rack kW = the facility's stated limit, else 40 kW (a common air-cooled compute-rack envelope). Report whether power or space binds. Every listed server is air-cooled.
• Feeds: A + B feeds, each alone carrying the whole rack; per-rack circuits = smallest standard circuit whose rated kW ≥ rack provisioned kW, ×2, or several pairs when a rack exceeds one circuit.
• Placement: Leaves for GPU fabrics sit in a network rack (or mid-row); front-end pair and OOB switch in the rack they serve; spines in a central rack; storage nodes in their own racks.
• Management: Clusters of 4 or more GPU nodes add 3 management nodes (the listed 1U compute node: provisioning/PXE, cluster manager, scheduler control plane, monitoring; 2 for an active/passive pair at small scale, 1 for a lab) and one 48× 1G OOB switch per rack; the management nodes attach to the front-end plane.