NVIDIA、アジェンティックAI時代を牽引するRubin GPUアーキテクチャを発表

What began as discrete AI model training and human-facing chat interfaces has evolved into always-on AI factories dedicated to producing intelligence at scale. These factories are now tasked with powering agentic workflows that reason, plan, use tools, verify intermediate results, and execute complex multistep tasks across vast contexts.

Agentic workloads are not defined by a single prompt and response, but by sustained inference across many reasoning steps. They demand low per-step latency, high decode throughput, efficient long-context attention, large KV cache capacity, and the ability to scale models across tightly coupled GPU domains. The data center must be reimagined as a single unit of compute, a vision realized with theNVIDIA Vera Rubin platform.

At the core of the platform is the NVIDIA Rubin GPU, designed to deliver up to 10x more agentic throughput per unit of energy than NVIDIA Blackwell (Figure 1). Enhanced Tensor Cores with expanded precision flexibility, a new HBM4 memory subsystem, and the third-generation Transformer Engine—delivering up to 50 petaflops of NVFP4 performance—work together to accelerate agentic workloads efficiently.

This post examines how the NVIDIA Rubin GPU and its co-designed scale-up system address the end-to-end bottlenecks of agentic inference, from data movement and compute efficiency to long-context execution and rack-scale deployment.

How does Rubin GPU architecture support agentic workloads?

The Rubin GPU (Figure 2) is constructed from reticle limited compute dies to achieve high density and efficiency. These two dies are unified on a single package through a high-speed inter-die link called the NVIDIA High-Bandwidth Interface (NV-HBI).

The architecture begins with the challenge of keeping an enormous amount of compute productive as agentic workloads shift between reasoning, generation, retrieval, and tool use. Its 336 billion transistors, 224 streaming multiprocessors (SMs), and 896 Tensor Cores provide the raw compute density, while the third-generation Transformer Engine adapts precision across numerical formats. That flexibility enables the Rubin GPU to deliver up to 50 petaflops of NVFP4 inference performance while preserving accuracy.

But performance depends on more than Tensor Core throughput. The Rubin GPU organizes compute resources into Graphics Processor Clusters (GPCs) with a large centralized L2 cache. The GigaThread Engine coordinates work, MIG Control partitions the GPU for multiple workloads, and NV-DEC accelerates decoding. Together, these capabilities help the Rubin GPU turn compute density into sustained utilization across the diverse and dynamic workloads that define large-scale agentic systems.

That utilization also depends on how quickly data can reach compute cores. Rubin integrates up to 288 GB of HBM4 memory, driven by dedicated HBM controllers and 12-Hi stacks, to deliver up to 22 TB/s of peak bandwidth. The enhanced Tensor Memory Accelerator (TMA) manages high-efficiency movement across complex data layouts, while NVIDIA NVLink 6 provides 3,600 GB/s scale-up bandwidth to the NVLink Switch for all-to-all GPU-GPU communication, NVLink-C2C delivers 1,800 GB/s for coherent CPU-GPU communication, and x16 PCIe Gen 6 provides up to 256 GB/s of host connectivity.

Finally, large-scale agentic deployments must protect data as it moves through this execution domain. Confidential Computing with TEE-I/O is designed to secure data at rest, in transit, and in use across the AI factory. Together, these compute, memory, connectivity, and security capabilities form the GPU-level foundation for agentic workloads that must sustain efficient execution across increasingly large models and scale-up domains.

How does the Rubin GPU accelerate critical inference paths?

Peak compute alone is not enough to accelerate agentic inference. Real-world performance also depends on how efficiently the GPU moves data, executes matrix operations, processes long-context attention, and transitions between dependent kernels. This section explores the Rubin GPU features designed to reduce overhead in these critical execution paths.

Accelerating rack-scale MoE weight and token movement

Mixture-of-experts (MoE) models dynamically route tokens across many expert networks. As the number of experts grows, efficiently locating and moving expert weights becomes increasingly important to inference performance.

The Rubin GPU enhances the Tensor Memory Accelerator to reduce data-movement overhead for expert-heavy models. Its improved descriptor handling allows software to work more efficiently with tensors that share common layouts but reside at different locations in memory.

Rubin improves this with inline descriptor update support for TMA (Figure 3). Instead of modifying the descriptor in memory, the kernel can keep one unified descriptor for tensors that share the same layout and override fields such as the memory pointer and stride directly in the TMA instruction at runtime.

This helps MoE models scale more efficiently as expert counts increase. By reducing metadata-management and data-movement overhead, Rubin enables more GPU time to be devoted to useful inference computation, supporting higher throughput for agentic workloads that rely on large MoE models.

Doubling the efficiency of rack-scale matrix operations

Rubin doubles the Tensor Core throughput per clock by doubling the amount of data it can process along the dimension. This optimization not only helps throughput bound kernels but also memory and latency bound kernels.

As model execution is split across many GPUs, each GPU often receives a smaller slice of the output work while the reduction dimension remains large.

The benefit of a larger dimension is fewer loop iterations. In Figure 4, a GEMM that requires four iterations on Blackwell can be completed with two iterations on Rubin. Fewer iterations reduce loop overhead, improve Tensor Core utilization, and help both context and decode GEMMs run more efficiently at high tensor-parallel scale.

Addressing key challenges in long-context processing for agentic AI

Long-context and agentic AI workloads put increasing pressure on attention. As context windows grow, the model must compare more tokens, normalize larger attention-score matrices, and apply those scores to the value data used to produce the next layer’s output. This makes attention one of the most important performance paths for improving tokens per second per user.

Rubin accelerates attention by combining activation sparsity with adaptive compression and improved softmax throughput. One simple, safe, and effective way to use Rubin’s new sparsity features in attention is as follows. The attention pipeline begins with a dense computation to generate the intermediate attention scores. Rubin can then load that intermediate data from Tensor Memory into a structured 2:4 sparse compressed form, generating both nonzero values and the metadata needed to use them efficiently while reducing the scores’ write cost and storage requirements. This allows the later attention stages to operate on less data while preserving the dense output format expected by the rest of the model.

This compressed intermediate representation reduces work in two key places: softmax and the second attention GEMM. Softmax can operate on the nonzero attention values, and the following multiplication with the dense matrix can use sparse MMA with the nonzeros from Softmax and the metadata from the original compression step. The result is less compute and data movement in the most expensive part of long-context attention improving tokens/watt, without requiring the surrounding model pipeline to change its interface.

Rubin also improves the softmax throughput. As Tensor Core throughput increases, softmax can become a bottleneck because it depends on exponential math and reductions across attention rows. Rubin increases exponential throughput, including 2x FP32 and 4x BF16/FP16 throughput versus the Blackwell baseline, helping softmax keep pace with faster matrix operations.

• *Together, these features make the Rubin attention acceleration more than a single-kernel improvement. Activation sparsity reduces the amount of intermediate attention work, while faster exponentials reduce the softmax bottlenecks.

Improving kernel execution efficiency

As inference scales across larger models and more GPUs, raw Tensor Core throughput is only part of the performance story. The GPU also needs to move efficiently from one kernel to the next. In inference, this is especially important because activations often sit on the critical path: one kernel produces activation data, writes it to memory, and the next kernel consumes that data to continue generating the next token.

Traditional producer-consumer execution can create bubbles in the GPU timeline. A producer kernel may complete work for some tiles or thread blocks early, but the consumer may not begin useful work until a broader dependency is resolved. The Blackwell programmatic dependent launch improves this by allowing earlier consumer-kernel progress, but dependent work can still wait for required activation data to become available.

Rubin enables more fine-grained coordination between dependent kernels. This allows consumer work to begin earlier as required input data becomes available, rather than waiting for a larger set of producer work to complete.

The result is a more tightly packed GPU timeline, reduced idle gaps, and improved overlap between dependent kernels. This is particularly valuable for agentic inference, where activations move sequentially through the model and kernel-to-kernel latency directly affects tokens per second per user.

How do Rubin memory and communication sustain high-throughput inference?

As models, context windows, and GPU domains grow, data movement becomes as important as computation. Rubin is designed to improve the flow of weights, activations, KV cache data, and communication traffic within the GPU and across scale-up systems. The following sections look at the memory and communication innovations that help sustain high-throughput inference.

Accelerated scale-up communications

As inference scales from a single GPU to full rack-level systems, communication becomes part of the critical performance path. When communication is fused directly inside a GPU kernel, the kernel does not stop and hand control back to the CPU; it directly writes data or performs reductions over NVLink to another GPU while computation is still in flight.

Traditional GPU-to-GPU communication requires coordination and synchronization work in addition to moving payload data. These steps can add latency and consume interconnect bandwidth, particularly when communication occurs frequently within distributed inference workloads.

Rubin introduces counted writes for device-initiated NVLink communication. This capability streamlines synchronization for GPU-to-GPU data transfers by allowing the receiving GPU to track transfer completion more efficiently.

NVLink fused communication with counted writes provides a lower-latency mechanism for coordinating GPU-to-GPU data movement, helping keep computation moving instead of waiting on synchronization operations.

Co-designing a memory subsystem for maximum power and compute efficiency

The decode, or generation, phase of inference is fundamentally memory subsystem bound. It is not about peak bandwidth specs but rather how efficiently each kernel can utilize the entire memory subsystem. Modern reasoning and agentic workloads amplify this constraint by spending more end-to-end runtime in decode: long contexts, large KV caches, and interactive token generation make achieved memory bandwidth a critical performance lever.