Wishlist Items For OpenBenchmarking.org

[phoronix]

Phoronix: Wishlist Items For OpenBenchmarking.org

Now that the new Phoronix site is rolled out and tweaking on that almost complete, one of my next work items on the list are some improvements to OpenBenchmarking.org...

http://www.phoronix.com/scan.php?page=news_item&px=OpenBenchmarking-Wishlist-1

[M1kkko]

Re-structure the openbenchmarking.org site to make it more useful to end users and also easier to contribute to.

Here's some possible use cases I can imagine:

• Find out how well my GPU can run game x
• Find out which GPU is currently the fastest on the market
• Find out which CPU has the best performance for certain workloads
  • Which one is the most energy-efficient (ie. has the best performance per watt)?
• Compare different drivers' performance (ie. proprietary, open source) for gpu x on benchmark y.
• Compare different operating systems' / window managers effect on gaming performance for gpu x and benchmark y.
• Contribute missing information to the database so that others can make use of it (e.g. let's say that game x has not yet been tested on hardware y on operating system z, but I own hardware y and game x, I'd like to contribute that information so that others can easily make use of it)

Either I'm more stupid than the average Joe or these use cases are currently not very well fulfilled.

In short, all benchmark data on the site should be indexed by all kinds of factors of benchmarking, e.g. by processor, graphics card, operating system, kernel version, window manager and so on, so I could easily find the information I'm looking for (eg. "all gaming benchmarks with hardware x") and also contribute missing information back to the site.

I would even consider setting up a linux system for automated benchmarking if it were easier for others to find and make use of that information.

If, by some miracle, this functionality already exists and I've managed to miss it, please forgive me.

[Michael]

Re-structure the openbenchmarking.org site to make it more useful to end users and also easier to contribute to.

Here's some possible use cases I can imagine:

• Find out how well my GPU can run game x
• Find out which GPU is currently the fastest on the market
• Find out which CPU has the best performance for certain workloads
  • Which one is the most energy-efficient (ie. has the best performance per watt)?
• Compare different drivers' performance (ie. proprietary, open source) for gpu x on benchmark y.
• Compare different operating systems' / window managers effect on gaming performance for gpu x and benchmark y.
• Contribute missing information to the database so that others can make use of it (e.g. let's say that game x has not yet been tested on hardware y on operating system z, but I own hardware y and game x, I'd like to contribute that information so that others can easily make use of it)

Either I'm more stupid than the average Joe or these use cases are currently not very well fulfilled.

In short, all benchmark data on the site should be indexed by all kinds of factors of benchmarking, e.g. by processor, graphics card, operating system, kernel version, window manager and so on, so I could easily find the information I'm looking for (eg. "all gaming benchmarks with hardware x") and also contribute missing information back to the site.

I would even consider setting up a linux system for automated benchmarking if it were easier for others to find and make use of that information.

If, by some miracle, this functionality already exists and I've managed to miss it, please forgive me.

Thanks for your feedback. Most of the functionality is more or less easily possible, the main shortcoming is just exposing it effectively via the UI given that I'm not an UI/UX expert. The main reason why items like these aren't exposed in rudimentary form is due to current system limitations with the current OpenBenchmarking.org databases growing so large, but hopefully now being on faster hardware and working out some performance optimizations, it's features like these I hope to enable soon on OB. Thanks again for lengthy and useful feedback.

[pete910]

To be able to actually sign up would be good, Also for the "upload results" to work. Though that might be because I cant join.

Sounds a bit like a rip off but something similar like anandtech, list gpu/cpu scores to compare directly.

[Stunts]

Michael, here is something I think would be pretty damn cool, but I'm not sure is possible in the current form:
Ask for a specific GPU, driver and benchmark - say "Radeon HD6870, Catalyst X.XX, OpenArena" and get a single graph with all the systems that are comparable.
Ie, instead of getting 10 bars, I would get a single bar with the average FPS and the respective error bars. In this bar would be systems with different CPUs, Memory, etc, but this would be useful to compare GPUs across "average systems". I would be able to compare this bar of the chart with "nVidia GTX650, nvidia 345.xx, OpenArena".
Do you think this is remotely possible, or even makes any sense?

Thanks for considering it.

[Michael]

Michael, here is something I think would be pretty damn cool, but I'm not sure is possible in the current form:
Ask for a specific GPU, driver and benchmark - say "Radeon HD6870, Catalyst X.XX, OpenArena" and get a single graph with all the systems that are comparable.
Ie, instead of getting 10 bars, I would get a single bar with the average FPS and the respective error bars. In this bar would be systems with different CPUs, Memory, etc, but this would be useful to compare GPUs across "average systems". I would be able to compare this bar of the chart with "nVidia GTX650, nvidia 345.xx, OpenArena".
Do you think this is remotely possible, or even makes any sense?

Thanks for considering it.

It's actually all rather trivially possible... Just a matter of how to expose it well visually while being efficient. From a command-line could more or less do it now but just a matter of maximizing efficiency at scale.

[Michael]

To be able to actually sign up would be good, Also for the "upload results" to work. Though that might be because I cant join.

Result uploading has no known bugs against it, please advise if that's the case.

[Stunts]

It's actually all rather trivially possible... Just a matter of how to expose it well visually while being efficient. From a command-line could more or less do it now but just a matter of maximizing efficiency at scale.

That's good to hear! But I suppose that something like that can be quite taxing on the server if not done in the most effective way. Regardless, thanks for the quick reply. I'll keep my eye out for that feature.

[pete910]

Any insight to the upload problem Michael that you had seen in the 15way 1080 gpu comments?

Hmmm so locally you have the results all working and good but when you go to upload them, it gives you the existing URL? What happens if you try:

phoronix-test-suite upload-results 1507265-BE-1080PLINU24

I have a feeling it's connected to the sign up problem that I have. Cert issue maybe?

[Michael]

Any insight to the upload problem Michael that you had seen in the 15way 1080 gpu comments?

I have a feeling it's connected to the sign up problem that I have. Cert issue maybe?

Can you email that result's composite.xml file to me at michael at phoronix .com