Phoronix Test Suite 1.0 Release Blockers

[Michael]

In this thread propose features and bug fixes you would like to see addressed in Phoronix Test Suite by 1.0 "Trondheim" stage...

[deanjo]

I would consider moving the project to the opeensuse build service where it could be built for many distro's, many architectures and you would then not have to worry about bandwidth for the larger stuff.

[Michael]

Originally Posted by deanjo

I would consider moving the project to the opeensuse build service where it could be built for many distro's, many architectures and you would then not have to worry about bandwidth for the larger stuff.

It's PHP, it's a script... Or are you referring to some other aspect of it? How would the OpenSuSE Build Service address the bandwidth needs for larger stuff?

[deanjo]

Originally Posted by Michael

It's PHP, it's a script... Or are you referring to some other aspect of it? How would the OpenSuSE Build Service address the bandwidth needs for larger stuff?

Most of the projects that you use already have repo's there compiled for various distro's and arch's. Using the build service would allow you to make installable packages for all the major distro's so all they have to do is add the repo to their favorite package manager and download.

[Michael]

The problem with using the distro's package for the test itself is that there is no control over any compile-time optimizations or any other changes made by that package maintainer that could skew the results compared to other distro packages or the vanilla source. However, if you look at the latest development work for trondheim-pts you'll see the "External Dependencies" feature, where the distribution's packaging system is now being used in some cases for obtaining GLUT for instance as well as some other dependencies for the benchmark but not the benchmark itself.

[deanjo]

Originally Posted by Michael

The problem with using the distro's package for the test itself is that there is no control over any compile-time optimizations or any other changes made by that package maintainer that could skew the results compared to other distro packages or the vanilla source. However, if you look at the latest development work for trondheim-pts you'll see the "External Dependencies" feature, where the distribution's packaging system is now being used in some cases for obtaining GLUT for instance as well as some other dependencies for the benchmark but not the benchmark itself.

There is nothing stopping you from popping the source from your favorite project and setting up the optimization flags you want on the build service. You can set up a complete repo of all the packages you use just for use with the test suite. You do not have to depend on other people's tweaks and mods at all. Making a live distro is a snap as well if you want with kiwi. I would really recommend you look into the power of the service that they are offering.

[Michael]

Originally Posted by deanjo

There is nothing stopping you from popping the source from your favorite project and setting up the optimization flags you want on the build service. You can set up a complete repo of all the packages you use just for use with the test suite. You do not have to depend on other people's tweaks and mods at all. Making a live distro is a snap as well if you want with kiwi. I would really recommend you look into the power of the service that they are offering.

It does sound more advanced than I had thought it was... I'll look into it.

[hwertz]

Originally Posted by Michael

In this thread propose features and bug fixes you would like to see addressed in Phoronix Test Suite by 1.0 "Trondheim" stage...

I'd put a note about NUM_CPU_JOBS in the README, or rig something up to set it from /proc/cpuinfo or the like. Per this thread on mplayer compilation test crashing the default NUM_CPU_JOBS of just plain leaving it blank uses too much RAM for some systems. If the test is meant to simulate juggling around dozens of active processes, this would change the test too much though.

[Michael]

Originally Posted by hwertz

I'd put a note about NUM_CPU_JOBS in the README, or rig something up to set it from /proc/cpuinfo or the like. Per this thread on mplayer compilation test crashing the default NUM_CPU_JOBS of just plain leaving it blank uses too much RAM for some systems. If the test is meant to simulate juggling around dozens of active processes, this would change the test too much though.

This is fixed in git. It was just a simple regression when I accidentally renamed the NUM_CPU_JOBS to SYS_CPU_JOBS but didn't think about the profiles NUM_CPU_JOBS contains the number of cores (as parsed by /proc/cpuinfo) plus one. I'll likely push out PTS 0.3.1 today due to this crashing bug.

[Michael]

Bump for 1.0 feature requests / blocker bugs.