Cache Coherent Device Memory For HMM

Jerome Glisse at Red Hat continues working on Heterogeneous Memory Management for the mainline Linux kernel and hopefully will be merged soon. He's now extended HMM with cache-coherent device memory support.

Heterogeneous Memory Management is focused on allowing a process address space to be mirrored and allowing system memory to be transparently used by any device process. HMM for Linux has been sought after by NVIDIA and other companies as there are big implications for OpenCL, Vulkan, and related areas.

Jerome's latest work is on HMM-CDM to allow for cache coherent device memory support. He explained on the mailing list:

Cache coherent device memory apply to architecture with system bus like CAPI or CCIX. Device connected to such system bus can expose their memory to the system and allow cache coherent access to it from the CPU.

Even if for all intent and purposes device memory behave like regular memory, we still want to manage it in isolation from regular memory. Several reasons for that, first and foremost this memory is less reliable than regular memory if the device hangs because of invalid commands we can loose access to device memory. Second CPU access to this memory is expected to be slower than to regular memory. Third having random memory into device means that some of the bus bandwith wouldn't be available to the device but would be use by CPU access.

Those interested in more details on the HMM-CDM work can be found via this patch series. This latest code is just less than 400 lines on top of the existing HMM code-base.