AI Factory Architecture in an Existing Facility: Fitting GPU Racks, Network Fabric and the Control Plane

AI factory architecture in an existing building starts with its deliverable critical load. Counting the GPUs that fit in a rack based on a vendor's datasheet is not enough; the building can only support so many racks. Modern AI racks can draw 100–150 kW or more, compared with 20–30 kW for legacy air-cooled racks.

The following six-step sizing sequence answers five questions:

1. GPU count: How many GPUs fit in the building? Divide deliverable critical load by rack power draw, then multiply by the number of GPUs per rack.
2. Cooling: Can the existing cooling system be reused, supplemented or retrofitted?
3. Network fabrics: What are the budget requirements for scale-up and scale-out connectivity?
4. Storage: How much NVMe storage is needed for hot data, and how much object storage for archives?
5. Control plane: What are the GPU allocation, scheduling and failure-response requirements?

The sequence assumes deliverable critical load, floor loading and cooling capacity are known, or that a readiness audit is underway to establish them.

Step 1: Let Deliverable Critical Load Cap the GPU Count

Deliverable critical load is the power the facility can supply to critical IT equipment under its existing electrical and facility constraints. Not all of this capacity goes directly to GPU racks; some must support other IT infrastructure.

For example, an NVIDIA GB200 NVL72 rack contains 36 NVIDIA Grace CPUs and 72 NVIDIA Blackwell GPUs in a liquid-cooled cabinet, drawing approximately 120 kW.

Completion check: Document the deliverable critical load, subtract power reserved for other IT infrastructure, divide the remainder by rack power draw and record the maximum rack and GPU counts.

Step 2: Fit the Training Pods Within the Existing Facility

A GPU cabinet is the scale-up unit: a liquid-cooled rack containing GPUs, NVLink/NVSwitch interconnects and coolant connections.

A GPU pod is the scale-out unit: multiple cabinets connected to move data between them. For example, an NVIDIA DGX GB200 SuperPOD reference architecture contains eight compute racks and 576 GPUs. Each GB200 compute tray uses four 400 Gb/s NVIDIA ConnectX-7 interfaces for scale-out networking over InfiniBand.

In an existing facility, pod layout must account for available floor space, rack positions, service clearances, cooling connections and cable pathways. These constraints determine whether cabinets can be grouped together or must be distributed across the hall.

A SuperPOD is a defined reference architecture, not simply a group of eight racks. Deployments with different layouts or rack counts must be evaluated against their networking and system requirements.

Training and inference have different infrastructure requirements:

• Training: AHEAD's production AI factory guide emphasizes dense accelerated compute, high-throughput storage and low-latency networking.
• Inference: Responsiveness, efficiency and cost-efficient scaling become priorities. Our guide to inference infrastructure architecture covers these requirements separately.

Completion check: Draw the pod layout within the existing hall, showing rack positions, GPUs per cabinet, required clearances, scale-out network and interfaces per compute tray.

Step 3: Decide Whether to Reuse, Supplement or Retrofit Cooling

Compare the facility's deliverable cooling capacity per rack with the heat load from Step 1. A 120 kW rack produces approximately four to six times the heat of a legacy 20–30 kW rack.

Qualified structural and mechanical engineers must confirm floor loading, cooling distribution and coolant distribution unit (CDU) requirements before deployment.

The comparison produces one of three verdicts:

1. Reuse: The existing cooling system supports the new rack loads without material changes.
2. Supplement: The existing system supports part of the load, with additional cooling capacity required.
3. Retrofit: The existing system cannot support the new rack density and requires substantial modifications, typically introducing direct liquid cooling.

For retrofits, confirm that floor loading and space for CDUs, manifolds and associated plumbing are sufficient. These requirements should be established through a GPU readiness audit.

Completion check: Document the cooling verdict for each hall, including floor-loading limits and CDU clearance requirements.

Step 4: Determine Whether the Existing Facility Can Support Both Network Fabrics

An AI factory deployment requires two distinct interconnect fabrics:

1. Scale-up fabric: Connects GPUs within a cabinet using technologies such as NVLink and NVSwitch.
2. Scale-out fabric: Connects GPUs across cabinets using technologies such as InfiniBand or RoCEv2.

NVIDIA's Enterprise AI Factory Design Guide describes low-latency, high-bandwidth networks, including 100–400 GbE RDMA-capable fabrics and dedicated GPU interconnects.

Higher-bandwidth reference architectures may require different connectivity. For example, NVIDIA's GB200 architecture uses 400 Gb/s scale-out interfaces on each ConnectX-7 NIC.

In an existing facility, the scale-up fabric is generally integrated into the GPU cabinet. The scale-out fabric must fit within the building's available network infrastructure.

Check whether existing fiber and cable pathways, rack-to-rack distances, switch locations and network equipment space can support the required connectivity. Legacy network infrastructure may need to be supplemented or replaced to support the new fabric.

Completion check: Prepare separate fabric budgets specifying interconnect technology, port count and copper or optical connectivity. Confirm that the existing pathways and network spaces can accommodate the scale-out design, including any required modifications.

Step 5: Determine Whether Existing Storage Can Support the AI Workloads

NVIDIA describes a tiered storage approach in its Enterprise AI Factory Design Guide, allocating the highest-performance storage to the most demanding workloads.

Examples include:

1. NVMe storage: High-throughput, high-IOPS storage for hot training datasets and models.
2. Object storage: Lower-cost storage for cold archives and historical datasets.

The guide also describes parallel, scale-out storage for streaming datasets, checkpoints and embeddings to accelerators without bottlenecks.

For an existing facility, evaluate whether installed storage systems can meet the AI workload's capacity, throughput and IOPS requirements. Existing object storage may be reusable, while training workloads could require additional NVMe or parallel storage to prevent accelerators from waiting for data. Also verify that the existing storage network can support the required throughput.

Agentic workloads introduce additional capacity requirements. NVIDIA identifies these workloads as inherently bursty, requiring sufficient compute resources to accommodate fluctuations in demand. Throughput metrics such as tokens per second or queries per second help determine the compute capacity and headroom required.

Storage must separately accommodate dataset capacity, throughput, IOPS and checkpointing behavior.

Completion check: Document which existing storage systems can be reused, supplemented or replaced. Specify NVMe capacity and throughput for hot datasets and checkpoints, object-store capacity for colder data, and workload-throughput assumptions for overall infrastructure sizing.

Step 6: Define the Control Plane Requirements for the Existing Facility

The control plane manages GPU allocation, workload scheduling and infrastructure monitoring across the deployed cabinets, fabrics and storage systems.

In an existing facility, the control plane must operate within the available GPU capacity and physical infrastructure constraints established in the previous steps. For example, if the deployment spans multiple halls or pods, the scheduler needs to account for GPU availability and network topology when placing workloads.

The key requirement for the vendor is how the scheduler assigns workloads across cabinets, monitors infrastructure health and responds to failures, including a failed cabinet or fabric link. The control plane should also provide visibility into GPU utilization so operators can manage the capacity available within the facility.

Completion check: Document the required GPU allocation, scheduling, monitoring and failure-response behaviors, including how they account for the existing facility's pod layout and available infrastructure.

Worked Example: 2,000 kW of Deliverable Critical Load

Assume an existing facility has 2,000 kW of deliverable critical load and will deploy NVL72-class racks drawing approximately 120 kW each. For this calculation, assume the 2,000 kW is available for GPU racks after reserving capacity for other IT equipment.

• Maximum rack count: 2,000 kW ÷ 120 kW = 16 full racks.
• Reserved capacity: Hold back the capacity of one rack, leaving 15 deployable racks.
• GPU count: 15 racks × 72 GPUs = 1,080 GPUs.
• Pod configuration: 15 racks could be organized into two groups of eight and seven racks, subject to the selected system and network architecture.

The existing hall must have sufficient floor space, rack positions, cooling connections and network pathways to accommodate the proposed layout.

The scale-out design must support communication between racks within and across these groups. This illustrative grouping does not establish that the deployment constitutes two NVIDIA DGX GB200 SuperPODs; SuperPOD compliance depends on the defined reference architecture and its networking and system requirements.

The cooling decision depends on the plant capacity established during the readiness audit. If the hall was designed for 20–30 kW racks, substantial cooling modifications will be necessary to accommodate this deployment.

The existing network and storage infrastructure must also be evaluated against the requirements of the 1,080-GPU cluster to determine what can be reused and what requires expansion or replacement.

What This Sequence Assumes Was Already Audited

Three inputs from the readiness audit are required:

1. Deliverable critical load: The maximum power available to critical IT equipment under the facility's electrical and operating constraints.
2. Floor loading: The structural capacity to support liquid-cooled racks and associated equipment.
3. Existing cooling capacity: The cooling available per rack position, determining the approach in Step 3.

This sizing sequence does not cover the detailed electrical path from utility feed through switchgear and UPS to the racks, or electrical substation sizing.

End-to-end check: Verify that the rack count from Step 1 fits the physical pod layout in Step 2, that the existing or upgraded cooling in Step 3 supports those racks, and that the network fabrics, storage and control plane in Steps 4–6 can support the deployment within the facility's constraints.

FAQ

What specifically changes in a retrofit versus a new build?

A retrofit inherits the existing building's power ceiling, floor-loading limits, cooling infrastructure and network pathways. A new build can design these systems around the target AI workload. The sizing process is similar, but a retrofit must work within established physical and electrical constraints.