Keith Packard Exploring "Semi-Automatic Compositing" For The X.Org Server

[phoronix]

Phoronix: Keith Packard Exploring "Semi-Automatic Compositing" For The X.Org Server

Keith Packard's latest work for Valve on improving the Linux display stack is on what he's exploring around "semi-automatic compositing" but at this point it's still a risky bet with the new protocol yet to be written...

http://www.phoronix.com/scan.php?page=news_item&px=Xorg-Semi-Auto-Compositing

[schmidtbag]

Shouldn't this have been written like... 20 years ago? Seems like this would help reduce CPU and GPU load, which old hardware obviously could've taken advantage of.

[Linuxhippy]

I always found it a crazy idea to implement the compositor as seperate process, compared to e.g. a plugin / shared library loaded into Xorg. Resizing windows has always been rather slow with X11, especially on (e.g. embedded) single-core-machines - the number of context switches involved is huge: X-Server, Window-Manager, Application and Compositor all have to run and talk to each other, sometimes multiple times.

The funny or sad thing is, some of the worst architectural misconceptions in Xorg (Trapezoids for geometry in Xrender, cmopositor as seperate process, bandwidth-saving-X11-protocol) were designed by exactly the some guys still working in the core design and architecture of XOrg/X11.

[bregma]

Shouldn't this have been written like... 20 years ago? Seems like this would help reduce CPU and GPU load, which old hardware obviously could've taken advantage of.

Dunno if it reduces CPU and GPU load so much (there are fewer copies and context switches, sure, but that's not always the bottleneck) so much as frame latency and predictability. The queue would go from "submit your draw, and it may or may not appear on the screen at some point some time maybe, hopefully after your previous draw and before your next" to "submit your draw, and it will be on the screen within 3 frames." As you can imagine, that would be a desirable improvement regardless.

[GunpowaderGuy]

hadn't x.org entered a permanent feature freeze ? , if not it really ought to

[linuxgeex]

Dunno if it reduces CPU and GPU load so much (there are fewer copies and context switches, sure, but that's not always the bottleneck) so much as frame latency and predictability. The queue would go from "submit your draw, and it may or may not appear on the screen at some point some time maybe, hopefully after your previous draw and before your next" to "submit your draw, and it will be on the screen within 3 frames." As you can imagine, that would be a desirable improvement regardless.

It's more about reducing input lag than anything else. Which is achieved by reducing the number of buffering steps and related overhead, and by taking proper advantage of GPU compositing (from blitter to shaders) and hardware display buffer swaps, using a proper triple buffered display pipeline. Triple buffered doesn't mean it takes 3 frames before it is presented by the display hardware either, but sadly a lot of LCD panels introduce >1 frame of additional input lag as well (they read the frame, composite it with their own HUD and display calibration/preferences, calculate the FRC temporal dithering relative to the error of the last frame, then show it.)

Properly done triple buffering has 3 hardware swappable display regions: the one being seen, a second one that must be completed before VBLANK arrives, and a third one that is normally actively being rendered to, feeding the second slot, and allows getting a bit ahead of the display timing so that short performance spikes don't have to impact framerate, with a caveat of increased input lag. That implies half a frame delay for the one being seen, up to a full frame for the one queued to be seen, and latency inversely proportional to the work time for the third frame if it is allowed to get that far ahead. This isn't the input lag you experience in the end. It will always be greater than this. 1.5 frames is basically the bare minimum if you use an LCD panel, have a perfect software stack, and there are no performance issues. Anywhere from 3.5 if the third frame buffer is completed instantly and no late events arrive before it is migrated. Down to 2.5 if the third frame is barely completed on time. Down to 1.5 if the frame is barely completed on time to be submitted as the second frame just as VBLANK arrives (0.5 average of the hardware to show it and 1 frame for LCD input lag)

The worst artifacts appear when buffer sync is managed poorly and frame rate decimation is in play due to performance limits. Then you get obvious things like out of order fields showing interlaced content (swaps happening asyncronously to frame updates), and hugely amplified framerate drops when decimation is happening both in input and output stages. Ie if a video player is overloaded and drops to showing every second frame, and the compositor drops to showing every second frame, you are at 1/4 display framerate even though twice as many frames are being rendered... actually quite a bit worse than 1/4 if the decimation isn't synchronous between the two, which it seldom is.

[Guest]

Isn't something like this already available on Wayland? I remember Collabora worked on something like this.

[gwgwg]

There's some irony in VR driving the need for applications to be aware of exactly when pixmaps are presented on the screen, while the analogous Color Management requirements that applications be able to be aware of which display their pixels land on is disregarded as a "wrong approach" by the Wayland crowd (even though they can offer no alternative themselves, and fail to appreciate the impact of what alternatives might be possible on applications.)

[smitty3268]

Sorry KeithP, but this is stupid. It will be 4 years before this can be finished, implemented, stabilized, and deployed to the masses. By then, X will be dead in terms of desktop users which is who's going to want to be using VR anyway.

[michael-vb]

hadn't x.org entered a permanent feature freeze ? , if not it really ought to

As long as someone still wants to work on it (and customers like Valve still see value in paying for that), why? Nearly everyone involved seems to agree that the current X server has major problems; the majority of the developers decided to solve that by creating Wayland, which starts afresh in a lot of areas. It is ninety percent done, and the magic final ten percent will probably take a long time; in the mean time there are a lot of regressions to be fixed when switching over from X, and the unavoidable problems which no one thought of are coming out as it sees the light of real use. In the mean time Keith is quietly fixing the problems in the X server, and making a pretty good job of it. That is the sort of thing which gets people less exited than starting all over again, but it can get to the same place in the same time at a fraction of the cost. I expect that it will, but that people will have switched anyway.

That said, I accept the majority vote, even if my vote would have gone the other way, and I am also using Wayland these days. I pay the price, mainly of regular crashes until the issues have been solved, but appreciate the countless hours of work and expertise which have gone into both servers, and in the end know that most of the issues will be worked out over time.