One Blackwell GPU, Two Memory Neighborhoods

A program sees one GPU and allocates one buffer. Inside a large accelerator, however, not every compute unit is equally close to every memory controller. Blackwell’s two-die organization makes distance a performance question: how often does data cross the internal fabric, and is the cost large enough to justify changing how work is partitioned?

One address space can hide different costs

NVIDIA describes Blackwell GPUs as two physical dies with separate L2/cache and DRAM-controller resources. When an SM accesses memory associated with the other die, data travels across the L2 fabric. NVIDIA’s “locality domain” describes this internal locality, but the company explicitly says it does not imply full conventional NUMA capabilities. CUDA still presents unified memory; coherent caching helps prevent repeated accesses from refetching data across the fabric. (NVIDIA locality-domain article)

That distinction prevents overreaction. A non-local access is not automatically slow, and NVIDIA says NUMA-unaware code can still achieve peak DRAM bandwidth. The effect depends on access pattern, cache reuse, power, occupancy and whether work must cross between compute neighborhoods. A stable partition may keep data local; a shared working set repeatedly touched by both sides may not.

The right model is “unified address space, non-uniform internal path.” It is not a claim that developers should manually place every allocation.

MIG can expose locality, with a communication trade

NVIDIA’s February 2026 engineering post explores mapping Multi-Instance GPU partitions one-to-one with Blackwell locality domains. In its Wilson-Dslash stencil example, the authors report up to 2.25 times speedup at a 400-watt power limit, attributing the result to avoiding L2-fabric power overhead. The same post says that at higher power limits, added MPI communication latency over PCIe or NVLink can outweigh localization for larger workloads. NVIDIA labels the technique exploratory and says later capabilities may supersede it. (NVIDIA engineering post)

That trade matters. MIG offers isolated GPU instances; splitting one workload across instances can also create an inter-process coordination problem. If a computation is separable and each partition can reuse its own data, localization may reduce internal traffic. If the algorithm exchanges data at every timestep, synchronization and transfer may erase the gain.

This is not the same job as cluster scheduling. A scheduler decides which workloads receive GPU resources; locality tuning asks where data and compute land inside the device and what communication that placement introduces. Our GPU scheduling explainer covers fleet-level allocation.

Partitioning can also change contention and available compute, so a faster MIG result does not isolate locality by itself. Keep the same overall power cap, problem size, software and competing-workload conditions; report the resource shape and any changed compute capacity. For a spatial solver, vary subdomain size and halo width. For a streaming pipeline, measure whether each partition reuses data before eviction. If a win disappears with working-set size or concurrency, document it as a regime-specific tuning option rather than a stable property.

One practical failure mode is a decomposition that makes each GPU partition appear faster in isolation but increases total completion time because the partitions exchange more boundary data. Another is an apparently local layout that loses once cache reuse is warm, since coherent caching may already be serving repeated remote reads efficiently. Compare cold and steady-state behavior and include the first-touch and synchronization costs. The point is to determine whether cross-domain traffic is material for this exact kernel, not to assume the physical seam is automatically the slow path.

Measure before partitioning

Start with the unpartitioned baseline and a workload whose data ownership is understood. Record GPU model and power limit, driver and CUDA versions, MIG layout, working-set size, communication volume, throughput and latency. Compare at the same power envelope and problem size. For a stencil, vary decomposition and halo exchange so the communication cost is visible.

Look for evidence that locality is the bottleneck: memory stalls or cross-domain activity, and an improvement that survives end-to-end timing rather than appearing only in a kernel microbenchmark. Use Nsight and GPU telemetry to locate the issue. “Memory-bound” alone does not show that traffic crossed the die boundary; cache misses can come from capacity or access order.

Then test at production scale and include boundary exchange, synchronization and management of multiple instances. NVIDIA’s own example shows why a power-constrained regime can differ from a higher-power one. Do not generalize a Wilson-Dslash percentage to transformer inference.

When engineers should care

This layer matters when an accelerator is large enough for internal paths to differ and a workload has repeated, structured access. Scientific stencils and domain decompositions are plausible candidates. A transformer kernel may be memory-bound, but that does not prove splitting it across locality domains helps; its layout and communication pattern need separate evidence.

The safer starting point is to preserve the unified model and profile first. Change partitioning only when measurements show a repeatable bottleneck and the new communication pattern is understood. A coherent GPU can still have internal data distances whose costs emerge under defined conditions. That narrows the hypothesis and saves teams from applying a blanket “NUMA-aware” rewrite to code that may already be efficient.