Alibaba Eyes Linux CPU Scheduler Changes To Better Handle QEMU With SMT/HT Threads

There is a phenomenon where running a multi-threaded workload inside a virtual machine (VM) with Simultaneously Multi-Threading (SMT / Intel Hyper Threading) that a sibling thread could find itself busy while the CPU core is idle. A new Linux CFS patch series aims to make the scheduler better adapt to the QEMU topology.

An Alibaba engineer sent out a patch series today to the CFS load balancing code to adapt for the QEMU CPU topology to try to ensure a sibling thread isn't busy before the main thread.

"Multithreading workloads in VM with Qemu may encounter an unexpected phenomenon: one hyperthread of a physical core is busy while its sibling is idle.
...
The main reason is that hyperthread index is consecutive in qemu native x86 CPU model which is different from the physical topology. As the current kernel scheduler implementation, hyperthread with an even ID number will be picked up in a much higher probability during load-balancing and load-deploying.

This RFC targets to solve the problem by adjusting CFS loadbalance policy:

1. Explore CPU topology and adjust CFS loadbalance policy when we found machine with qemu native CPU topology.

2. Export a procfs to control the traverse length when select idle cpu."

That patch series was sent out a short time ago and already it's facing rejection by Intel engineer Peter Zijlstra in the approach and SMT enumeration assumptions. So we'll see what more comes of this effort for the kernel or what may ultimately be punted to QEMU as its problem.