NVIDIA unveils 576‑GPU Rubin Ultra for AI

NVIDIA has revealed a 576‑GPU rack‑scale solution called Rubin Ultra, using the new CPO and NVLink 7 technologies to fuse the chips into a single super‑GPU. The system is built on the Kyber liquid‑cooled rack and will deliver 15 EFLOPs of FP4 inference and 5 EFLOPs of FP8 training, a 14‑fold increase over its predecessor. It also packs up to 1 TB of HBM4E memory across four reticles, giving the rack 365 TB of HBM and 4.6 PB/s bandwidth, while the NVLink interconnect offers 1.5 PB/s and the ConnectX‑9 Spectrum chip adds 115.2 TB/s of data movement.

CPO and NVLink link all GPUs

Each Kyber rack contains 576 GPUs that act as one unit, thanks to the CPO optical switches that connect the NVLink links. NVIDIA says the modular OCP‑based design lets installers plug the rack in a few minutes, and the reference design can bring a deployment from months to about two weeks. The company announced that CPO racks are now in volume production, ending earlier delays.

Red Hat has partnered with NVIDIA to secure autonomous AI agents that run on the Vera Rubin infrastructure. The collaboration centers on the Red Hat AI Factory, which combines OpenShell sandbox runtimes, NVIDIA’s Sentry security framework on BlueField DPUs, and Argus monitoring that inspects host memory to detect malicious activity. The setup also isolates compromised workloads in milliseconds and reduces host CPU load by moving networking and security functions to the data‑processing unit.

Together, the Rubin Ultra’s high performance per watt and the Red Hat security stack aim to support 10‑trillion‑parameter AI models in enterprise environments. The joint reference design also adds Model‑as‑a‑Service controls, vLLM inference engines, and support for Nvidia Nemotron models, allowing companies to govern model access, track usage, and keep performance stable across hybrid clouds.