X.Org Server Development Process Is Questioned

[Ericg]

No, if they just stripped away unused features so the X.org Server would not be a full implementation of the X11 protocol, then that would not be X12, it would be a partial implementation of X11 or a a implementation of a subset of the X11 protocol, or X11 light.

X12 is an another protocol and a discussed successor to the X11 protocol.
http://www.x.org/wiki/Development/X12

Also, Wayland is something different. Also, since Wayland can use XWayland to run X.org Server on top of Wayland, then we could still cut fat of X.org Server while using it together with Wayland.

The wayland comments weren't meant to be something taken super literally, he was talking about a cut down, lightweight, minus-legacy version of X11, i was implying that by the time they specced it out (Cuz remember, the three input subsystems are all inter-dependent on eachother, so lets go make a fourth that is those 3 done right to keep things lightweight >_> *only half sarcasm*) figured out what exactly could be cut / replaced / conjoined, wrote anything that needed to be wrote, coded anything that needed to be coded, tested it, and released it... Wayland would be even closer to adoption, if not being used already.

I do agree though that cutting the fat from X11 for the sake of XWayland would be a good idea to keep the backend lightweight

EDIT: the "X12 is Wayland" comment I made was more about the spirtiual succesor, yes I know X12 was being written and worked up, but I dont think any of the main Xorg devs are actively working on it anymore... least none that I've heard, Daniel could probably confirm this: Most of X.org's devs have or will probably throw their weight in with Wayland rather than write up an X12, correct?

[uid313]

The wayland comments weren't meant to be something taken super literally, he was talking about a cut down, lightweight, minus-legacy version of X11, i was implying that by the time they specced it out (Cuz remember, the three input subsystems are all inter-dependent on eachother, so lets go make a fourth that is those 3 done right to keep things lightweight >_> *only half sarcasm*) figured out what exactly could be cut / replaced / conjoined, wrote anything that needed to be wrote, coded anything that needed to be coded, tested it, and released it... Wayland would be even closer to adoption, if not being used already.

Wow, I had no idea that it was so bad, that they were all depended on each other, what a conjoined mess.
I was thinking maybe we could remove X logical font description (XLFD), and other legacy stuff that isn't used these days.

I do agree though that cutting the fat from X11 for the sake of XWayland would be a good idea to keep the backend lightweight

Yeah, I wonder what things we could ditch.[/quote]

EDIT: the "X12 is Wayland" comment I made was more about the spirtiual succesor, yes I know X12 was being written and worked up, but I dont think any of the main Xorg devs are actively working on it anymore... least none that I've heard, Daniel could probably confirm this: Most of X.org's devs have or will probably throw their weight in with Wayland rather than write up an X12, correct?

As I understand it, X12 is the name of an protocol that has not been specified (yet, and probably never will be). So far there have only been discussions of how X12 would look and work if it were ever created.
Discussions about what should be thrown away, what should be fixed, what developer would like to have in a different way, and what new stuff they would like to see in it.

[Ericg]

Wow, I had no idea that it was so bad, that they were all depended on each other, what a conjoined mess.
I was thinking maybe we could remove X logical font description (XLFD), and other legacy stuff that isn't used these days.

Watch the "The Real Story Behind X and Wayland" lecture that Daniel Stone (daniels here on the forums if you go back a page in this thread :P) did for linux.conf.au , he, David Airlie, and Keith Packard all did talks at linux.conf.au about X, Wayland and whats wrong with X as a whole. Daniel made a great, half-joking, comment about input under x that was "There's currently about three people on the planet that really understand how input under X works...and I really wish I wasn't one of them."

Linux.conf.au videos:
http://mirror.linux.org.au/linux.conf.au/2013/webm/

If you wanna watch them, you're looking for:
"Future_directions_for_the_X_Window_System" by Keith Packard
"Teaching_the_X_server_new_tricks" by David Airlie
and "The_real_story_behind_Wayland_and_X" by Daniel Stone

As I understand it, X12 is the name of an protocol that has not been specified (yet, and probably never will be). So far there have only been discussions of how X12 would look and work if it were ever created.
Discussions about what should be thrown away, what should be fixed, what developer would like to have in a different way, and what new stuff they would like to see in it.

Its a protocol that was in the works about the same time krh decides "Lets write Wayland!" Michael did an article or two about X12 around the sametime Wayland picked up.

Something that really bothers me about X12, from the very wiki page you linked, is that they are making Network Transparency mandatory... We've spent the last 5 years without true network transparency (if you use shared memory or DRI2 in your application, your application isnt truely network transparent anymore). X11 is network-capable, it is not network-transparent unless you decide to go without any and advancements made in the last 5-7years. Which some do, their choice.

But I really dont see how X12 could do network transparency without basically recreating VNC. You can't do network transparency in the 'traditional X way' without breaking a lot of and regressing in a lot of ways.

In case you are wondering, uid, Wayland does have networking built in, in theory. Its basically a better VNC, meaning, server creates the image, server compresses the image, and server sends it over the wire.

'Traditional X way' is, I believe, sending the rendering commands over the wire and then the local server does the rendering. Which is faster rendering in THEORY, but the problem is...Core X is synchronous. Every command has a break where the client sends back an "Okay, received and done." which the server waits to see, then sends the next one. Waits, sends the next one, over and over.

As much as people yell "X11 is network transparent and is X's biggest feature!" ...it really isn't. Because of the synchronous nature, network transparency is actually X11's biggest failure and THE worst-case-scenario. Every command sent to the local server, and every confirmation back to remote server is affected by latency and network lag. So instead of 1-way lag, where its just the server sending commands. You have double lag because its the latency of the packet that has the rendering commands, and then the latency of the reply back for success/failure.

[daniels]

EDIT: the "X12 is Wayland" comment I made was more about the spirtiual succesor, yes I know X12 was being written and worked up, but I dont think any of the main Xorg devs are actively working on it anymore... least none that I've heard, Daniel could probably confirm this: Most of X.org's devs have or will probably throw their weight in with Wayland rather than write up an X12, correct?

I can't really speak for the others, but I haven't heard any particular criticisms of its technical fundamentals. Just the usual ones like being paid by an OS vendor who still relies entirely upon X11, so not really being able to just go work on Wayland.

As for X12 - it was never really being 'designed' per se. It was a list we periodically added to of things we'd change if we were, one day, creating a new version of X. There was never a serious attempt to make it exist.

[Ericg]

I can't really speak for the others, but I haven't heard any particular criticisms of its technical fundamentals. Just the usual ones like being paid by an OS vendor who still relies entirely upon X11, so not really being able to just go work on Wayland.

As for X12 - it was never really being 'designed' per se. It was a list we periodically added to of things we'd change if we were, one day, creating a new version of X. There was never a serious attempt to make it exist.

Thank you Daniel, seriously (Both for your work and your postings) I wish you and David spent a little more time in the forums because I'm not sure which is worse, here or LWN when it comes to misinformation in the comments... And thank you for the clarification, i try to give the most correct info I can from all the various dev's blogs / mailings but even I get mixed up sometimes haha

[squirrl]

Thank you Daniel, seriously (Both for your work and your postings) I wish you and David spent a little more time in the forums because I'm not sure which is worse, here or LWN when it comes to misinformation in the comments... And thank you for the clarification, i try to give the most correct info I can from all the various dev's blogs / mailings but even I get mixed up sometimes haha

Did you hear about the mass migration back to Xfree86?

All the distributions have voted and will be returning regardless whether 3D works or not.

It was all a mistake. UTAH GLX The fall of a great 2D regim.


oh well... wouldn't surprise me but j/k!

[daniels]

XAA and DRI is removed now?

XAA has been removed, DRI has been almost completely removed. It's 3717 lines of code, nicely isolated from everything else. But you're right, it'd be a good deprecation to make.

Is EXA removed yet? Can it be removed? Will it be removed?

I doubt EXA will ever be removed. The primary focus of EXA isn't acceleration, it's on buffer management. Buffers can be in one of three places: system memory, graphics memory accessible by the CPU, and straight graphics memory. Each of these have their own tradeoffs, and where they need to be depends on the usage pattern (i.e. the balance between CPU and GPU accesses). So EXA has a hugely complex model to try to manage this, and is still used by the AMD and Nouveau drivers, amongst others.

UXA was developed by Intel, because they don't have the system memory vs. graphics memory split, and now with Sandy Bridge and up they don't even have to worry about caching. So it made sense for them to split from EXA, but they're about the only open driver which can do this.

Now DRI3 is on the way, so when that happens, will DRI2 be removed?

I don't know the answer to that one.

What else can be removed?
Somethings cant be fixed/changed without breaking everything.
But maybe we can remove lots of things that nobody uses these days.
Sure, X.org Server wont be 100% compliant with X11 anymore, but at least we will have got rid of lots of old legacy cruft, and made the code base smaller and more easier to maintain.

Honestly? Really not a lot. DRI 1 is a really good suggestion which I didn't think of when we cleaned it up. But the server codebase is about half the size it was when we released 7.0 back in 2005; since then we've added AIGLX and subsequently DRI2, XI2.0 (multi-pointer) and 2.2 (multitouch) as well as input hotplug, RandR 1.2, and a hell of a lot more. The headline figure for lines of code in the X server is a bit over 500,000 which seems like a lot, but when you consider the ~20,000 lines of core rendering, the core font code, the various deprecated-but-still-used input extensions, etc, there's not a great deal left to cut out. And, if you're removing those bits, why still bother being X? There will always still be ancient proprietary Motif apps to which no-one has the source code that need to be supported by RHEL. Always.

And even then, you'd be stuck with an unmaintainable codebase, crippled in spec and implementation by bending over backwards for apps you don't even support anymore. I don't think the X server is ever going to be long-term tractable without a total rewrite, but it's just not feasible to rewrite the server whilst still being compatible with X11 apps. The behaviours they rely on are far too subtle to get right without years of really frustrating tweaking.

There is one big improvement still to be made which would result in a pretty big net reduction of code and a huge step towards the server actually being tractable: removing all the DDXes (Xnest, Xephyr, Xquartz, etc) other than Xorg. Of course, this requires xf86-video-nested to be complete in order to replace Xephyr, a Xorg driver for OS X, and so on. But doing that would cut out huge impedence/translation layers and a lot of special cases which are never reached by Xorg, and go a long way to making the server a tractable project to work on, as well as more accessible to newcomers.

Thank you Daniel, seriously (Both for your work and your postings) I wish you and David spent a little more time in the forums because I'm not sure which is worse, here or LWN when it comes to misinformation in the comments... And thank you for the clarification, i try to give the most correct info I can from all the various dev's blogs / mailings but even I get mixed up sometimes haha

Ha thanks, glad some of it gets through to someone!


I need to clarify something as well - misread your comment earlier. There are still a bunch of X developers who still work on it - Keith Packard, Dave Airlie, Peter Hutterer, Alan Coopersmith and Aaron Plattner come to mind. So it's not really fair to say that it's abandoned as a project. Also, some of the people who left left naturally; we pushed as much of the development as possible out of X into DRM/KMS, Mesa, and others. So for example, Eric Anholt used to count as an active X developer, but as more code went into Mesa, he moved over there. But it's safe to say that the people : project size ratio was really harsh even when we had participation at its strongest, and it's now notably worse.

[garegin]

Daniel. I asked some people before. What are the causes of x server restarts and will they be prevalent with wayland too. Some said that the cause is the toolkit and the wm.

[Linuxhippy]

So instead of 1-way lag, where its just the server sending commands. You have double lag because its the latency of the packet that has the rendering commands, and then the latency of the reply back for success/failure.

But usually commands are transported asynchronously to the server, right? So you sacrifice a bit of debug-ability for higher performance.
What I find to be the biggest problems, are applications using XLib as it were a synchronous API, which it actually isn't.

Regards

[Ericg]

Hey Daniel, a few questions about your X/Wayland talk that i've been meaning to ask.

1) What'd you mean when you said "I dont have a slide saying its not introspectable... but X isn't introspectable." Were you referring to the horrendous error codes? function names? something completly different?

2) You briefly hit on the error code numbers at one point, should I take that to mean that X returns pure integer errors, no helpful hints associated with them? And in that same context...(disclaimer: I havent look at the code base, way above my level of coding at the moment) if they do return pure integers, why aren't the error codes enumerated? At least then you could return strings, such as "DEVICE_NOT_FOUND" for the example of someone unplugging the mouse. enums got added to C89 and were a base feature of C++ so its not a matter of language.

3) Personal opinion on when Wayland/Weston is "Usable"? You said in the talk that you'd start using it when gnome shell and the touchpad were brought up to date (which you were meant to do ) krh was running it at XDC last year and it looked fairly good.

I ask because Wayland has hit [stable] in Arch, weston is in [testing], qt5 has hit [stable] gtk with the wayland backend is in [testing] or [stable] (not sure which, i'm a KDE guy not a Gnome guy lol) and I think mesa with all the required compile-time flags is in testing, though it may have been moved to stable with the last mesa update. So you're about to get a distro full of potential bug testers that has latest-stable of everything Wayland needs (in theory) across the board, that can download all the needed packages fairly easily (yay pacman)