Gallium3D OpenGL 4.1 State Tracker Redux

[phoronix]

Phoronix: Gallium3D OpenGL 4.1 State Tracker Redux

There was a Gallium3D OpenGL 4.1 State Tracker proposed for this year's Google Summer of Code to benefit X.Org / Mesa. As this state tracker was going to be written from scratch and without any dependence on Mesa itself, the consensus among the core developers was that the work was simply too ambitious for a lone student developer to complete over the course of a summer. A new proposal has now been drafted by Denis Steckelmacher, the Belgian student developer interested in open-source OpenGL 4.1 support...

http://www.phoronix.com/vr.php?view=OTIzMw

[HokTar]

Micheal, you sound like working on a hw driver is a bad thing.

Marek has a point there but I can't fully agree, features do matter.
For example I'm a bit tired of the edges I get with even compiz not to mention anything more serious. I'd really appreciate some MSAA in r600g!
Another usual problem is video decoding but we all know that by now.

Denis could still continue Zack's work on the opencl state tracker, clover. Having a fast and reliable architecture for GPGPU stuff would be a nice addition for the HPC capabilities and desktops would make good use of it soon, I think.

Just saying...

[elanthis]

People should work on what they want to work on. It's Open Source, not a company. He only gets paid if he completes the work, so if he doesn't pull through, it's his loss and nobody else's.


That opinion aside...

Personally, of all things at this point, I'd rather just see a whole new damn graphics API built over Gallium 3D. Fuck OpenGL and DirectX both.

Give me the cleanliness of DirectX 10/11's API with the portability of an OS-neutral C API, using a cleaned up version of Cg's shader language and semantics, and a usable utility library like DirectX's for shader/texture loading and the like.

Please.

/* create and populate an attribute buffer (example using convenience set data command) */
ngl3Buffer* buffer = ngl3BufferCreate(ctx, sizeof(Vertex) * nVertices, NGL_BUFFER_STREAM, NGL_BUFFER_WRITE);
ngl3BufferSetData(buffer, aVertices, sizeof(Vertex) * nVertices);

/* create and populate an index buffer (example using explicit lock, copy, unlock commands) */
ngl3Buffer* indices = ngl3BufferCreate(ctx, sizeof(uint16_t) * nIndices, NGL_BUFFER_STREAM, NGL_BUFFER_WRITE);
map = ngl3BufferLock(indices, NGL_BUFFER_WRITE);
memcpy(map, aIndices, sizeof(uint16_t) * nIndices);
ngl2BufferRelease(indices, map);

/* create and popular a constant buffer */
ngl3Buffer* constants = ngl3BufferCreate(ctx, sizeof(float) * 16, NGL_BUFFER_ONEFRAME, NGL_BUFFER_WRITE);
ngl3BufferSetData(buffer, gProjectionMatrix, sizeof(float) * 16);

/* define an attribute layout */
ngl3Layout* layout = ngl3LayoutCreate(ctx, 3);
ngl3LayoutAddSemantic(layout, NGL_FLOAT3, "POS0", buffer);
ngl3LayoutAddSemantic(layout, NGL_FLOAT3, "COLOR0", buffer);
ngl3LayoutAddSemantic(layout, NGL_FLOAT2, "TEXCOORD0", buffer);

/* define a constant buffer layout */
ngl3Layout* clayout = ngl3LayoutCreate(ctx, 1);
ngl3LayoutAddSlot(clayout, NGL_FLOAT4x4, 0, constants);

/* define a render target */
ngl3Target* target = ngl3TargetCreate(ctx);
ngl3TargetSetDefaultFramebuffer(target);
ngl3TargetSetDefaultDepthbuffer(target);

/* load shaders (use low-level API -- you should have a standard higher-level effects/technique API as well!) */
ngl3Program* program = ngl3ProgramCreate();
ngl3ProgramLoadFile(program, "foo.vs", NGL_VERTEX_3_0);
ngl3ProgramLoadFile(program, "foo.gs", NGL_GEOMETRY_3_0);
ngl3ProgramLoadFile(program, "foo.fs", NGL_FRAGMENT_3_0);

/* render primitives */
ngl3Render* cmd = ngl3RenderCreate(ctx);
ngl3RenderSetTarget(cmd, target);
ngl3RenderSetConstants(cmd, constants);
ngl3RenderSetAttributes(cmd, layout);
ngl3RenderSetIndices(cmd, indices, nIndices);
ngl3RenderSetProgram(cmd, program);
ngl3RenderSetType(cmd, NGL_TRIANGLES);
ngl3RenderExecute(cmd);

The objects can be reused as much as possible. They're typed, so none of that OpenGL numeric identifier what-was-this-object-again bullshit. Objects that make sense to be write-only are write-only (like layouts). Those small objects that you may create a lot of are just backed by a simple pool allocator.

No global state machine. You have to create objects using an explicit context. You can have as many contexts as you want, create different contexts for different threads, use one for rendering and another some other off-screen purposes, whatever you want.

A render state command object instead of a shitload of redundant draw commands like OpenGL. Anyone actually writing an OpenGL renderer just wraps the stupid OpenGL draw commands behind a render state object anyhow, because sometimes you want to enable/disable various bits of render state and nobody wants to have to do a huge if (foo) DrawFoo() else (bar) DrawFoo() else ... every time they want to draw something. As a bonus, render command objects can be added to a render list which can be stored or passed off to another context later for actual rendering, which is what you need for multi-threaded rendering. (You create and populate buffers in threads that are doing your scene composition and culling, those create render command lists, pass those back to the main thread, and the main thread then executes the render lists.)

Everything uses a typed accessor to make the API simpler. Direct3D uses parameter structs, but you have to be careful to clear them and use them properly, while using proper objects with accessors removes the margin for error. This is not a significant source of overhead, as you are only calling a few dozen functions per pass at most; the function call overhead won't even show up on a profiler on any significant application (e.g., anything besides micro-benchmarks).

All of these functions are actual well-defined extern functions in a header file, not function pointers loaded up by some magic crappy backend. Why? IntelliSense. Code completion engines in all popular code editors get confused by OpenGL. (Note that having a C++ API is even better in this case, as then the code completion lets a user see which methods exist for any particular object just by typing object->[ctrl-space]).

The 3 in the API calls is just an example of how there should be versioning. Don't do things like OpenGL. The API that makes sense on DirectX 9 class hardware does not make sense for DirectX 11 class hardware, nor for GLES class hardware.

Oh, I didn't add an example of textures, but just to clarify on that: be sure that you separate texture and sampler objects (both OpenGL and DirectX got this wrong, then both half-ass fixed it), and that you be sure to include OpenGL's support for deferred mipmap generation.

Include standardized convenience APIs wherever it makes sense. High-level effects stuff makes sense. Loading data from memory, streams, and files makes sense (potentially using your own stream API just because there's so many different stream APIs in low-level libraries that may need wrapping, like FILE* vs iostream and so on; and because MSVC's debug mode means you can't mix release-mode DLL's and debug-mode DLL's if they use stream-based APIs).

Also note that the API is designed to be easily bindable by C++ or scripting languages, unlike OpenGL which is near impossible to bind (efficiently) due to the crazy-ass global state machine. A simple header-only C++ wrapper should be defined as part of the API standard so people on C++-friendly platforms don't have to gouge out their eyes.

I have multiple professional game developers working on real engines begging for such an API (or just begging for non-DirectX platforms to disappear... but we don't want that). I'm willing to bet that, if implemented, documented, and covered by a test suite, such an API could get enough support to even get the big vendors to implement it in their drivers. (And in the short term, on Windows at least, such an API could just wrap DirectX -- wrapping OpenGL is near impossible though due to the global state machine semantics and the lack of multi-thread capabilities.)

using ngl3;

Buffer* buffer = ctx->CreateBuffer(...);

// etc.

Damn I want this API to exist.

It's close to what my OpenGL wrapper looks like, but that is limited by all the limitations of OpenGL (including being stuck with GLSL or wrapping up the non-Open Cg framework).

[mdias]

...
Personally, of all things at this point, I'd rather just see a whole new damn graphics API built over Gallium 3D. Fuck OpenGL and DirectX both.
...

+1

Implementing this on gallium3D would remove the need to wrap it to d3d on windows.

[przemoli]

to resovle S3TC patent problem!

Buy out S3!

[elanthis]

Floating point textures are more interesting than S3TC (yes, both are important).

Good luck buying out Microsoft, though.

[89c51]

would a new API be able to circumvent the patent problems of OpenGL and how big of a task would be for someone to create and maintain something like that???

Also isn't Floating point textures a pure software patent valid only to the US?? Someone -i think dave airlie- mentioned that S3TC is hw/sw related so its kind of different.

[BlackStar]

would a new API be able to circumvent the patent problems of OpenGL and how big of a task would be for someone to create and maintain something like that???

A new API wouldn't solve anything, because the patents are in the algorithms themselves.

As for the amount of work, it's doable if you layer it on top of OpenGL or D3D (that's what 3d engines do). It's impractical if you wish to communicate with the hardware directly (without going through GL/D3D).

[TechMage89]

I can't imagine it would be too terribly difficult to write a new API on top of G3D. The question is, given the lack of proprietary support, could you get anyone to use it?

[mdias]

I can't imagine it would be too terribly difficult to write a new API on top of G3D. The question is, given the lack of proprietary support, could you get anyone to use it?

Most probably yes... The only missing bit is good drivers and Gallium3D not being widespread yet...

It wouldn't solve the patent issues, that's a different problem.