OpenGL Frame Latency / Jitter Testing On Linux

Beyond there finally being Team Fortress 2 benchmarks on Linux, at Phoronix is now also support for OpenGL frame latency benchmarks! It's another much sought after feature and request for graphics hardware and driver testing.

Topping off the day besides the TF2 benchmarking support now in the Phoronix Test Suite by simply running phoronix-test-suite benchmark tf2 on Steam, a number of test profiles now support reporting the OpenGL frame latency during benchmarking.

For numerous months there have been requests by Phoronix readers to deliver frame latency graphs to complement the average FPS reporting on our Linux gaming benchmark results. With Phoronix Test Suite 4.8 to further reinforce the FPS numbers there is also min and max FPS reporting, but with the latest Phoronix Test Suite Git code and the OpenBenchmarking.org test profiles there is now frame latency graphs.

For supported games/engines, a line graph shows the amount of time (in milliseconds) that it takes to render each frame of the given benchmark. Other Windows sites have done this in the past, and there's been requests by Phoronix readers to have such graphs, but not until recently has there been commercial interest from Phoronix Test Suite stakeholders to have the functionality. It's also taken some restructuring within pts-core to make it possible for one test profile/run to generate multiple data results / graphs. It's always just a matter of running e.g. phoronix-test-suite benchmark openarena unvanquished!

Lots of good free Linux benchmarks are coming...

Being a feature now of the Phoronix Test Suite, the support is unified and very easy to exploit by test profiles in a seamless manner -- where it's supported by the upstream game engine, right now with those idTech3-derived titles that have the com_speeds engine option. The test profiles simply need to set the com_speeds 1 option and then to the Phoronix Test Suite result-parser XML file simply add com-speeds-frame-latency-totals as the Identifier to the new ExtraData result parser XML tag. By making these two simple changes, the Phoronix Test Suite will now automatically collect and record the frame latency (the total frame latency -- com_speed's all value) when running the test profile (here's a look at one of the supported test profile contents using this new feature).

On the client side, the OpenGL frame latency is then automatically recorded if using Phoronix Test Suite 4.8 or newer (right now it's in Git, but PTS 4.8-Sokndal Milestone 3 will be released in the next few days). So beyond just having the OpenGL frame-rate reported, you will also have the frame latency graphs automatically rendered:


At the moment, this feature is enabled in the test profiles for Urban Terror, Unvanquished, OpenArena, and Reaction. Support for other test profiles may be added later. Thanks to the Phoronix Test Suite design, this works in a completely uniform and standardized way and also works for the phoronix-test-suite on all platforms: just not Linux, but OS X, BSD, Solaris, and Windows.

I'm still tweaking the frame latency graphs and making other changes ahead of the official Phoronix Test Suite 4.8 release this quarter. Any feedback is appreciated and welcome -- either by emailing me, @MichaelLarabel on Twitter, or using the Phoronix Forums. Also still being toyed with is whether to always render the frame latencies by default for supported test profiles, the graph visuals, and whether to average the frame latencies across multiple test runs. As PTS users know, most Phoronix Test Suite runs are done at least three times (or increased dynamically if the standard deviation exceeds a threshold) and then averaged. At the moment, the frame latency data is used from the last run, but pending feedback and further testing the individual frame latencies could be averaged across all runs.

For some results from this early OpenGL frame latency testing, see:

- Some AMD A10-6800K Ubuntu results on the Catalyst driver.

- Intel Core i7 4770K tests on Mesa and more, like a resolution comparison.

- Some Apple OS X 10.9 "Mavericks" OpenGL frame latency results.

- An AMD APU vs. Intel Haswell Linux comparison!

- NVIDIA Linux data

Aside from providing feedback on new features, if you are an organization relying upon our open-source automated benchmarking software, there are commercial services offered including enterprise support and custom engineering services. If you're just an end-user or Phoronix reader that finds the new activity helpful, please consider going premium or making a PayPal tip.