Intel Xeon Max Performance Delivers A Powerful Combination With AMX + HBM2e

Written by Michael Larabel in Processors on 7 July 2023 at 03:00 PM EDT. Page 2 of 3. 50 Comments.
OpenVINO benchmark with settings of Model: Face Detection FP16, Device: CPU. Xeon Max 9480 2P, HBM Only was the fastest.

First up was the OpenVINO benchmark with the face detection (FP16) model. AMX makes a massive difference to the performance right away. When limiting to oneDNN AVX-512 FP16 usage and no AMX, the Xeon Max 9480 processors were much slower than EPYC while with AMX alone was enough to put the Xeon Max 9480 2P ahead of the EPYC 9554/9654 2P configurations. However, moving to HBM caching and then HBM-only mode really allowed Xeon Max to deliver a dramatic lead over EPYC Genoa in this test. The Xeon Max 9480 2P in HBM-only mode was 1.73x the speed of the EPYC 9654 2P, which itself was 1.23x the speed of the lower core count EPYC 9554 2P albeit closer in core size to the 56-core Xeon Max 9480.

OpenVINO benchmark with settings of Model: Face Detection FP16, Device: CPU. Xeon Max 9480 2P, HBM Only was the fastest.

The latency also showed a profound difference with HBM2e and AMX.

OpenVINO benchmark with settings of Model: Face Detection FP16, Device: CPU. Xeon Max 9480 2P, HBM Only was the fastest.

Engaging AMX and going to HBM-only drove up the combined two socket power consumption by just about 13 Watts both on average and the peak recording. With this OpenVINO run the Xeon Max 9480 2P CPU power results were lower than the AMD EPYC 9554/9654 2P.

OpenVINO benchmark with settings of Model: Face Detection FP16-INT8, Device: CPU. Xeon Max 9480 2P, HBM Only was the fastest.
OpenVINO benchmark with settings of Model: Face Detection FP16-INT8, Device: CPU. EPYC 9554 2P was the fastest.

When using the face detection model with FP16-INT8 mode, the Xeon Max performance showed the huge benefit still from AMX and HBM2e though here the EPYC 9554/9654 processors were exhibiting lower latency.

OpenVINO benchmark with settings of Model: Person Detection FP16, Device: CPU. EPYC 9654 2P was the fastest.

With the OpenVINO person detection (FP16) test, the EPYC 9654 2P managed to slightly outperform the optimal Xeon Max configuration but was made a competitive race thanks to AMX and HBM2e.

OpenVINO benchmark with settings of Model: Person Detection FP16, Device: CPU. Xeon Max 9480 2P, HBM Only was the fastest.

The Xeon Max 9480 2P with AMX and HBM only did deliver lower latency than the EPYC Genoa processors tested.

OpenVINO benchmark with settings of Model: Person Detection FP16, Device: CPU. Xeon Max 9480 2P, HBM Only was the fastest.

The Xeon Max 9480 power results also remained lower than the EPYC 9554/9654 processors for OpenVINO.


Related Articles
Ampere Computing 2024 Roadmap Update: 256 Core 3nm CPU In 2025
Benchmarking The First RISC-V Cloud Server: Scaleway EM-RV1 Performance
AMD Announces Ryzen PRO 8840 & PRO 8000G Series CPUs
Intel Announces Gaudi 3 AI Accelerator, Intel Xeon 6 Brand
AMD Zen 4 vs. Zen 4C Performance, Zen 4C Core Scaling With Ryzen 5 8500G
Intel Core i3 14100 / i5 14500 vs. AMD Ryzen 5 8500G / 8600G In 500+ Benchmarks