Intel Xeon Platinum 8490H "Sapphire Rapids" Performance Benchmarks

Given Intel's significant focus this generation on AI workloads and the increasing industry interest there, let's start this benchmarking with some numbers using Intel's own OpenVINO software that has enjoyed optimizations for Sapphire Rapids to show its potential:

Going from AMD EPYC Milan(X) to Genoa I was floored back in November with the OpenVINO performance now that EPYC has AVX-512 with Zen 4... But with the OpenVINO results on Sapphire Rapids I was overwhelmed with excitement at the performance. Granted, OpenVINO is open-source Intel-developed software and has been optimized for Intel processors. Thanks to AMX, the Xeon Platinum 8490H performance was phenomenal both generationally over Ice Lake and compared to AMD EPYC. The OpenVINO performance on Sapphire Rapids was outright dominating.

While the $17k price tag came with some sticker shock compared to AMD EPYC 9004 series pricing and prior Xeon Scalable "Ice Lake" pricing, at least in AI workloads with AMX and other workloads able to take advantage of new Sapphire Rapids features, the price can actually be justified.

Not only was the throughput of the Xeon Platinum 8490H slaughtering everything else, but the latency was much lower as well.

Both the EPYC 9654 and Xeon Platinum 8490H were consuming around 325 Watts on average but with the 8490H peaking to 377 Watts compared to 360 Watts with the Genoa flagship.

Across a range of models with OpenVINO the performance uplift from Ice Lake was phenomenal and shooting past Genoa for this important open-source AI software.

The "worst case" with the OpenVINO benchmarks were for some models finding the Xeon Platinum 8490H delivering only similar performance in 2P configurations to the EPYC 9554~9654. But some of these models also didn't simply scale as well either with at times the 9554 having a slim lead over the 9654.