Keith Packard On Needed DRM Changes For VR HMDs

[phoronix]

Phoronix: Keith Packard On Needed DRM Changes For VR HMDs

Earlier this month it was revealed that Keith Packard had begun working with Valve to address DRM changes for Linux VR. He's now commented a bit on the technical approach he's pursuing for better dealing with virtual reality head-mounted displays (HMDs) with the open-source Linux drivers...

http://www.phoronix.com/scan.php?page=news_item&px=DRM-VR-HMD-Changes-Needed

[BwackNinja]

Pardon my ignorance, because I haven't used it myself, but isn't Zaphod Mode meant exactly to handle this scenario?

You don't want your desktop on your VR HMD. You want it to be entirely independent (because you wouldn't move windows onto it, so no Xinerama). Start an X Server on the output that goes to your VR HMD alone and start applications with a second DISPLAY variable set, or have an autodetect mechanism.

You even have separate inputs for your VR experience, so those can also be associated only with the secondary X Server and need not be enabled on your regular desktop.

All you'd really need is Zaphod autoconfiguration support to make that a seamless experience.

I don't know how well that would work with a multi-monitor setup combined with a VR HMD though.

[Terrablit]

Pardon my ignorance, because I haven't used it myself, but isn't Zaphod Mode meant exactly to handle this scenario?

You don't want your desktop on your VR HMD. You want it to be entirely independent (because you wouldn't move windows onto it, so no Xinerama). Start an X Server on the output that goes to your VR HMD alone...

They don't want a windowing system on the second one. Check this quote from the linked post on Keith's blog.

We could just hack up the window system(s) and try to let applications reserve the HMD monitors and somehow removing them from the normal display area so that other applications don't randomly pop up in the middle of the screen. That would probably work, and would take advantage of much of the existing window system infrastructure for setting video modes and performing page flips. However, we've got a pretty spiffy standard API in the kernel for both of those, and getting the window system entirely out of the way seems like something worth trying.

They go into details of how it could maybe work with the existing infrastructure, but there's enough kernel API support there that they think they could make a better solution. From what I'm seeing, the problem space that they're looking at is that of a system that already has display outputs and a windowing system, where you hook up the VR headset solely for VR applications (e.g. games). In that particular scenario, there's not a ton of benefit to routing it through the windowing system to later on bypass it, unredirect, or all the various techniques in existence for getting X out of the way for high-performance full-screen applications. We spend a lot of effort on bypassing X server for rendering and video. And performance is a priority for the interested parties.

Additionally, Zaphod mode is also part of the X Server, which means we can't depend solely on that, given that we're looking forward to newer, non-X solutions like Wayland. At some point in time, stuffing all the square pegs into round holes becomes less productive.

[jaxxed]

The proposed DRM leasing could be non-X systems could re-use.

[pixo]

I wonder if this could be used for multiseat and if 3D acceleration would work.
Would be great for HTPC/Steam seat using the main GPU.

[BwackNinja]

...Additionally, Zaphod mode is also part of the X Server, which means we can't depend solely on that, given that we're looking forward to newer, non-X solutions like Wayland. At some point in time, stuffing all the square pegs into round holes becomes less productive.

Ok, then it seems Zaphod Mode would work for the current applications (assuming you get working hw-accelerated DRI on all screens), but moving forward that's too limiting of a solution because of the caveats when you run on X and the fact that X is no longer the grand manager of all displays.

After re-reading the post on Keith's blog a few times, I get the feeling that we've run into a situation that calls into question whether a 'card' is the right abstraction for a windowing system (or anything else running on KMS) to operate on. The leasing model is a built on telling the main DRM Master some white lies (output X isn't connected, when it really is) which is somewhat hackish but functionally equivalent to a change in abstractions, deals with how these resources would be used and moved around, and doesn't break existing projects now that we're not just talking about X.

I can't say I like all aspects of this approach, but it looks like the most sane way to do this.

[coder]

Without trying to weigh in on the best short-term solution to this problem, it does strike me as interesting to consider the long game. Eventually, it might be the case that VR users want the ability to pop up desktop windows or some other 3D app manifestation in their VR space, or maybe the equivalent of something like systray icons.

Since we don't really know what sort of VR workspace will work best, we can't say that any given part of X would have a suitable mapping to it. So, rather than argue against the proposed changes, I hope they're something on which an AR/VR-oriented equivalent of X could eventually be built.