Intel Xeon 6980P SNC3 vs. HEX Clustering Mode Performance

With the Intel Xeon 6900P "Granite Rapids" processors that launched last week there are SNC3 and HEX clustering modes for these new processors. The default Sub-NUMA Clustering 3 (SNC3) mode for the three compute dies while the HEX mode is like SNC1 mode formerly for all three compute dies acting as one NUMA node. Using the flagship 128-core Intel Xeon 6980P processors, I ran some benchmarks looking at the real-world performance difference for SNC3 vs. HEX clustering modes on Granite Rapids.

The Intel Xeon 6900P processors support SNC3 and HEX clustering modes with SNC3 being the default so each compute die is exposed as its own NUMA domain. SNC3 will restrict the L3 cache and four memory channels to each of the three compute dies to avoid latency overhead where as with HEX mode it's all as one big domain per socket. But with being exposed as a single NUMA domain for the entire Xeon 6900P series processor can induce L3 cache and memory latency penalties.

Ultimately it comes down to the workloads that have NUMA optimizations or not in determining what clustering mode is most optimal. Being curious about the real-world performance impact and for reference purposes, I ran some benchmarks of the Xeon 6980P dual socket server in both SNC3 and HEX modes. Changing the SNC mode was the lone change between the rest runs. Disabling SNC from within the Xeon Granite Rapids BIOS is the means of switching over to HEX mode.

The dual Xeon 6980P processors were running Ubuntu 24.04 LTS with the Linux 6.8 kernel during this round of benchmarking for comparing SNC3 vs. HEX mode across a few dozen different benchmarks.