AMD Preparing Linux For Smart Data Cache Injection With "Upcoming" CPUs
Sent out today was a new patch series in preparing the Linux kernel's resource control "resctrl" functionality for L3 Smart Data Cache Injection Allocation Enforcement (SDCIAE). That new patch series explains of SDCI and SDCIAE:
"Upcoming AMD hardware implements Smart Data Cache Injection (SDCI). Smart Data Cache Injection (SDCI) is a mechanism that enables direct insertion of data from I/O devices into the L3 cache. By directly caching data from I/O devices rather than first storing the I/O data in DRAM, SDCI reduces demands on DRAM bandwidth and reduces latency to the processor consuming the I/O data. The SDCIAE (SDCI Allocation Enforcement) PQE feature allows system software to limit the portion of the L3 cache used for SDCI."
The AMD Smart Data Cache Injection support depends in turn upon PCI Express TLP Processing Hints (TPH) that were covered on Phoronix a few months ago when AMD engineers posted patches there. PCI Express TLP Processing Hints are hints that can be injected for improving latency and lowering traffic congestion when there are several possible cache locations in the system to indicate the optimal location of a Transaction Layer Packet (TLP).
The patches indicate "upcoming" and "new" AMD hardware will support Smart Data Cache Injection. But SDCI was announced as a feature originally back for AMD EPYC Genoa(X) / Bergamo. Since then we haven't heard much about SDCI and are only seeing these Linux kernel patches now. The timing is also a bit peculiar ahead of the AMD EPYC Zen 5 "Turin" launch. So given the current messaging, it's not clear if this SDCI support is for existing EPYC Bergamo/Genoa(X) server processors or only for upcoming platforms if there ends up being a significant difference in the SDCI implementation or not.
SDCI has appeared in AMD programmer documentation since last year while the Linux kernel patches are only surfacing now and building off the recent PCIe TPH work. AMD SDCI is able to let DMA data be pre-fetched into the cache of target CCXs rather than going first into DRAM, in order to help lower latency, increase performance, and conserve memory bandwidth. In digging deeper into the AMD Linux efforts around SDCI, at least the initial drivers (SDCI users) in mind for this functionality are around Linux network drivers.
31 Comments
