Linux 6.4 Delivers A Big Performance Boost For VDUSE

Merged last week for the Linux 6.4 kernel were all of the VirtIO and Virtual Data Path Acceleration (VDPA) changes. Interesting from that pull request is delivering a big performance bump for VDUSE.

For those that don't recall, VDUSE was merged to the kernel back in 2021 with Linux 5.15. VDUSE is around implementing VDPA devices in user-space. The vDPA framework is the VirtIO Data Path Acceleration code within the kernel for devices complying with the VirtIO specification but relying on a vendor-specific control path. The vDPA framework is increasingly used for abstracting hardware and concealing some elements of the hardware complexities to software while having a unified user-space API as a VirtIO device. VDUSE allows for implementing vDPA software-emulated devices in user-space.

Bytedance engineers in the past few months have been working on improving the performance for VDUSE. With a set of patches to mitigate the virtqueue lock contention within the virtio-blk driver, Bytedance is seeing around a 50% performance improvement: from 600k to 900k IOPS in their tests. The patches from Bytedance also add a sysfs interface to allow changing the affinity of IRQ callbacks for each VDUSE virtqueue to help with performance tuning and also a sysfs interface to set the bounce buffer size for virtio-vdpa for tuning with high throughput workloads. The 50% boost alone is nice while these new knobs for further performance tuning is icing on the cake.

The VirtIO/VDPA changes for Linux 6.4 also bring a reduction in the interrupt rate for VirtIO, scalability improvements for vhost-scsi, and other improvements. More details on these changes via this pull.

Those wishing to learn more about VDUSE in general can do so via the kernel.org documentation.