In the past few days there's been
some progress in benchmarking newer Steam Linux games thanks to support from Valve developers and separate from that was now
support for automated benchmarking of Metro Redux (Metro Last Light Redux and Metro 2033 Redux). The Metro Redux test ability led to yesterday's
22-way AMD/NVIDIA GPU comparison for this OpenGL 4 Linux game. The latest excitement? Getting automated tests up and running for Epic Games' Unreal Engine 4!
The Steam Linux game benchmarking advancement that was talked about in the earlier article is help from Valve in being able to gain rights from game publishers to redistribute OpenGL game traces to use as benchmarks for newer AAA Linux games that otherwise don't support automated benchmarking.
That article though went over some of the downsides in benchmarking Linux games with OpenGL traces. The Metro Redux testing advancement though was something that coincidentally also got finished up this past week.
The Metro Redux game titles launched on Linux support in early December but
its benchmark support was partially broken as currently shipped by 4A Games. In the meantime, I've been working with
Lauri Kasanen on getting (great) improvements to the handy libframetime library for providing
frame-time information for modern Linux games that don't supply that data. With his latest Git code, I've found out that not only is it working well for the Metro Redux games to supply the performance data but it's also working for the various Unreal Engine 4 demos.
For months there's been
the UE4 Linux demos Wiki page of various "tech demos" for Epic Games' latest game engine. These demos have also provided performance information, but it hasn't been easily supporting automation -- no dumping of FPS data to the console or a file, not the ability to automatically quit the demos upon completion or a certain period of time, etc. However, with libframetime now working well, I've adapted some of these UE4 Linux engine demos to doing automation that's good enough for basic benchmarking via the Phoronix Test Suite.
Libframetime is working with these demos for dumping per-frame performance information, these demos are basic "fly by" demos that go uninterrupted unless you're using the keyboard or mouse, and then the Phoronix Test Suite test profiles are just killing the process after a predetermined amount of time. If using
the new Phoronix Test Suite 5.6 code, this works nicely for some basic
Unreal Engine 4 Linux benchmarking.
These Linux demos are also not distributed through Steam so
it avoids the issues of Steam that make proper benchmarking more difficult with new updates being seamlessly pushed down without the user control that can affect the performance, not having to deal with a mess of environment variables to workaround the Steam runtime, can be cached automatically via the built-in Phoronix Test Suite download caching mechanisms, the necessity of owning the license to a game to just benchmark it, etc. It's also more accurate than OpenGL game traces since it's doing more than just replaying the GL commands.
This is nice for basic UE4 Linux benchmarking and better than running tests on just OpenGL traces, but
it's not perfect. At this time the UE4 demos don't seem to support running with any command-line arguments for specifying windowed vs. full-screen testing, resolutions, or any visual setting adjustments (
Update: the resolution / full-screen options are
now supported). There's also possible interference from using the keyboard or mouse while running these tests. But for those running the tests side-by-side in an isolated and controlled environment, this should work out nicely at least until more proper Unreal Engine 4 benchmarking possibilities are available.
The UE4 demo PTS test profiles end up being
quite simple and as an example you can see this raw test profile to see how it's implemented. Of the various UE4 Linux demos, the ones I found interesting to include at this point are
Atlantis, Elemental, Matinee, and Mountains.
Assuming you're on Linux x86_64 with OpenGL 3.2 or newer and Phoronix Test Suite 5.6 Git, you can run
phoronix-test-suite benchmark ue4-atlantis ue4-elemental ue4-matinee ue4-mountains to carry out all of these Unreal Engine 4 Linux OpenGL performance tests in a fully-automated and reproducible fashion. Alternatively, I also made a
Unreal Engine 4 test suite that contains these four test profiles plus any other UE4 tests that might be added in the future.
For some reference data,
there's initial UE4 results on OpenBenchmarking.org for a NVIDIA GeForce GTX 980 on Ubuntu Linux while other benchmarks are coming soon.
Needless to say, in the days ahead you can expect
a large Linux graphics card comparison with these UE4 test cases on the latest graphics drivers... Certainly for the proprietary AMD and NVIDIA Linux drivers while I need to see how these demos are performing for the open-source Mesa/Gallium3D drivers.
Appreciate all this work invested in benchmarking modern Linux games? Consider returning the favor via
subscribing to Phoronix Premium, viewing
Phoronix without AdBlock,
making a PayPal tip, and/or liking us on
Facebook,
Twitter, and
Google+.
Lastly, many thanks go out to Lauri Kasanen for his work on improving
libframetime to make the underlying performance monitoring possible for these less than friendly benchmarks. I'm also hopeful that Epic Games will make improvements to further enhance UE4 benchmarking given
they've previously publicly expressed Phoronix Test Suite benchmarking as one of the roads to be explored for improving UE4.
