UpStation
Design UPS-DSN-20261103-G9YQ
Prepared 2026-11-03 · prices as of 2026-11-03
Prepared for Example AI Lab · Head of ML Infrastructure
Budgetary design and estimate — not a quote

4× HGX B300 training cluster (32 GPUs) with NVMe storage and a services cluster, Montana colo

4× 8x NVIDIA B300 (HGX B300 NVL8) (32 GPUs, HGX class), 5× 2U all-NVMe storage node storage (208.3 TB plannable, replica_3), 3 cluster services nodes. Networks: gpu fabric 800G on 1 switch; front end 400G on 2 switches; oob 1G on 1 switch. 3 racks at ≤ 40 kW, 82.51 kW provisioned, in Montana colo. Every figure is computed from the ai.upstation.io catalog with the arithmetic shown below.

$3,307,252.84hardware (CapEx)
$21,453.00/mocolocation, 82.51 kW
$3,564,688.84year one
$4,079,560.843-year total
$4,594,432.845-year total

Delivery: 20 weeks, set by 8x NVIDIA B300 (HGX B300 NVL8), Intel, 3TB (Aivres) — Aivres KR9288X3. Colo: Montana at $260/kW/month × 82.51 kW, 36-month term.

Requirements

Workload
training
GPUs
32
Storage
200 TB, NVME
Rack limit
40 kW
Colo
Montana
Timeline
production in Q2 2027

How it was designed

  1. GPUs → nodes: ceil(32 ÷ 8) = 4 × 8x NVIDIA B300 (HGX B300 NVL8), Intel, 3TB (Aivres) (32 GPUs; nodes ship fully populated).
  2. Class: HGX — the GPUs must act as one (training, multi-node). Does the model at its precision fit one GPU's memory, and can each GPU work alone? Yes: PCIe. No: HGX. Distributed training is never served by PCIe servers.
  3. Storage (replica_3, Ceph across nodes, floor 3+1): plannable = 184 × N × (N−1)/N × 0.333 × 0.85; N = 5 gives 208.3 TB ≥ 204.5 TB and 5 × 20 = 100 GB/s ≥ 16.0 GB/s read.
  4. Storage demand: read = 32 × 0.5 = 16.0 GB/s; checkpoint burst = 18.7 GB/s; write = 18.7 GB/s; capacity = 204.5 TB.
  5. Cluster services (chosen with the buyer, optional): 3 × 1U CPU nodes for provisioning, scheduler and monitoring.
  6. GPU fabric: 32 × 800G ports (1 per GPU) ≤ 32 → one 32× 800G OSFP switch, 1:1 non-blocking (single point of failure; a leaf-spine removes it).
  7. Front-end: 2 ports per host, one to each switch of an MLAG pair of 32× 400G QSFP-DD switches; 7 ports per switch of 30 usable → 1 pair(s). Storage stays converged on this plane (well under 40 GB/s per node).
  8. OOB: 12 hosts + 3 switches = 15 × 1G ports → 1 × 48× 1G RJ45 + 4× 10G.
  9. GPU nodes per 40 kW rack = min(floor(40 U ÷ 10 U), floor(40 ÷ 16.96 kW)) = 2 (power binds).
  10. Compute 1: 22 U, 35.24 kW provisioned → 1 × 415V 3-ph 60A circuit pair (A+B), 82% of each feed; 8 power cords per feed.
  11. Compute 2: 20 U, 33.92 kW provisioned → 1 × 415V 3-ph 60A circuit pair (A+B), 79% of each feed; 6 power cords per feed.
  12. Services 1: 15 U, 13.35 kW provisioned → 1 × 208V 3-ph 50A circuit pair (A+B), 74% of each feed; 10 power cords per feed.

Network topology

flowchart TB
  GF["GPU fabric: 1× 32× 800G OSFP
800G 1:1, 32 links"] GN["4× 8x NVIDIA B300 (HGX B300 NVL8)"] -- "32× 800G" --> GF FE["Front-end MLAG pair: 2× 32× 400G QSFP-DD
400G, storage converged"] GN -- "2× 400G per node" --> FE SN["5× 2U all-NVMe storage node"] -- "2× 100G per node" --> FE MG["3× cluster services node"] -- "2 ports per node" --> FE FE -. "uplinks to your core (not priced)" .-> CORE(("Your network")) OOB["OOB: 1× 1G switch"] -. "BMC, 15 ports" .- GN

Rack elevation and power

Compute 122/40 U · 35.24 kW · 1× 415V 3-ph 60A A+B A 82% B 82% 1 6 11 16 21 26 31 36 gpu-01 8x NVIDIA B300 (HGX B3…16.96 kW gpu-02 8x NVIDIA B300 (HGX B3…16.96 kW gpu-sw-1 32× 800G OSFP switch…1.25 kW oob-1 48× 1G RJ45 + 4× 10G sw…0.07 kW Compute 220/40 U · 33.92 kW · 1× 415V 3-ph 60A A+B A 79% B 79% 1 6 11 16 21 26 31 36 gpu-03 8x NVIDIA B300 (HGX B3…16.96 kW gpu-04 8x NVIDIA B300 (HGX B3…16.96 kW Services 115/40 U · 13.35 kW · 1× 208V 3-ph 50A A+B A 74% B 74% 1 6 11 16 21 26 31 36 fe-1a 32× 400G QSFP-DD switch…0.75 kW fe-1b 32× 400G QSFP-DD switch…0.75 kW stor-01 2U all-NVMe storage n…1.65 kW stor-02 2U all-NVMe storage n…1.65 kW stor-03 2U all-NVMe storage n…1.65 kW stor-04 2U all-NVMe storage n…1.65 kW stor-05 2U all-NVMe storage n…1.65 kW svc-01 1U compute node1.2 kW svc-02 1U compute node1.2 kW svc-03 1U compute node1.2 kW gpu node storage node services node gpu fabric front end oob
RackSpaceProvisionedCircuitsDevices
Compute 122/40 U35.24 kW1× 415V 3-ph 60A (A+B)2× 8x NVIDIA B300 (HGX B300 NVL8), 1× 32× 800G OSFP switch — GPU fabric, 1× 48× 1G RJ45 + 4× 10G switch — OOB
Compute 220/40 U33.92 kW1× 415V 3-ph 60A (A+B)2× 8x NVIDIA B300 (HGX B300 NVL8)
Services 115/40 U13.35 kW1× 208V 3-ph 50A (A+B)2× 32× 400G QSFP-DD switch — front-end pair, MLAG, 5× 2U all-NVMe storage node, 3× 1U compute node

Bill of materials

ItemQtyUnit (USD)ExtendedLead timeSource
GPU servers
8x NVIDIA B300 (HGX B300 NVL8), Intel, 3TB (Aivres) — Aivres KR9288X34537,196.402,148,785.60Configure-to-order on Aivres B300 allocation (October 2026 s/p/UPS-GPU-B300-8-T2-AIV
Storage
2U all-NVMe storage node, dual socket, 24x 7.68TB NVMe (184TB raw) — SYS-222C-TN5177,958.91889,794.55Barebone in stock (2026-11-03); configure-to-order. Est. 3-5/p/UPS-SN-INT-NVME
Cluster services
1U compute node, dual socket, 64 cores, 512GB, diskless NVMe (virtualisation host) — SYS-122C-TN334,523.31103,569.93Barebone in stock (2026-11-03); configure-to-order. Est. 3-5/p/UPS-CN-INT-1U
GPU fabric
32× 800G OSFP switch (GPU fabric) — FS N8650-32OD, PicOS137,500.0037,500.0026 in stock (global warehouse, 2026-11-03). ~1 week/p/UPS-NET-800G-32
GPU fabric cabling
800G link: 2× SR8 + 2× MTP-12 OM4 10 m — node to leaf, 1 per GPU port (estimated optical)322,058.0065,856.00/switches
Front-end network
32× 400G QSFP-DD switch (front-end pair, MLAG) — FS N8610-32D, PicOS220,000.0040,000.0033 in stock (global warehouse, 2026-11-03). ~1 week/p/UPS-NET-400G-32
+2× 400G north-south (budgeted at the Supermicro equivalent option; Aivres fits its own 400G NICs and confirms them on its quote) (listed option, one per GPU node)42,693.9410,775.76/p/UPS-GPU-B300-8-T2-AIV
Front-end network cabling
400G AOC 10 m — GPU nodes to the pair, 2 per node81,055.008,440.00/switches
400G→4×100G breakout 2 m — storage nodes to the pair (4 nodes per cable)4199.00796.00/switches
400G→4×100G breakout 2 m — cluster services nodes to the pair (4 nodes per cable)2199.00398.00/switches
400G 2 m passive DAC — peer link2174.00348.00/switches
Out-of-band
48× 1G RJ45 + 4× 10G switch (OOB) — FS S3410-48TS, PicOS1989.00989.00593 in stock (global warehouse, 2026-11-03). ~1 week/p/UPS-NET-1G-48
Hardware total$3,307,252.84

Assumptions

Not included