How to create a new test?

[janmartin3]

Hi all,

I like to create a test for software not covered so far.
Unfortunately when clicking Create a Suite at http://openbenchmarking.org/create_suite
I am getting only:
Coming Soon
This feature is currently disabled.

Is there a way to do without?
Or does anyone know how soon 'soon' is?

Thanks,
Jan

P.S.:
I searched this forum and could not find anything on creating new tests.
Where to look?

[Michael]

Hi all,

I like to create a test for software not covered so far.
Unfortunately when clicking Create a Suite at http://openbenchmarking.org/create_suite
I am getting only:
Coming Soon
This feature is currently disabled.

Is there a way to do without?
Or does anyone know how soon 'soon' is?

Thanks,
Jan

P.S.:
I searched this forum and could not find anything on creating new tests.
Where to look?

Creating a test profile is what you're after... I don't have a concise guide covering the entire process at the moment due to lack of time, but most people are able to figure it out by looking at an example and either modifying it or just learning from it. So an easy example would be to look at a test profile like c-ray or smallpt or openarena.

1. first fetch the test profile locally.... that can be done by a variety of ways, it's all done automatically by PTS. Easy way is just like running: phoronix-test-suite info smallpt .... which will download the data from OpenBenchmarking.org if it's not already present locally.
2. look inside ~/.phoronix-test-suite/test-profiles/pts/smallpt*
3. from there you should be able to figure it out... pretty easy, collection of bash scripts and XML.

Then any further questions can be asked in the forums.

[janmartin3]

Just found out that
phoronix-test-suite build-suite
creates a framework for a new test suite.

Also in
.phoronix-test-suite/installed-tests
there are a few tests to look at for inspiration.

Is there a more detailed HowTo available anywhere?

Thanks,
Jan

[Michael]

Just found out that
phoronix-test-suite build-suite
creates a framework for a new test suite.

Also in
.phoronix-test-suite/installed-tests
there are a few tests to look at for inspiration.

Is there a more detailed HowTo available anywhere?

Thanks,
Jan

That's to build a suite, which is a collection of tests to be executed. If you want to support a new piece of software, that's just a test profile, which my previous post covers.

Within the installed-tests/ directory is where tests are installed. If you want to create a test profile look within the test-profiles/ directory.

Only documentation would be what's on here or the linked PDF - http://phoronix-test-suite.com/?k=documentation - from that page.

[myxal]

Within the installed-tests/ directory is where tests are installed. If you want to create a test profile look within the test-profiles/ directory.

Only documentation would be what's on here or the linked PDF - http://phoronix-test-suite.com/?k=documentation - from that page.

Sorry to resurrect this old thread, but I'm running into the same problem and, unfortunately, not much has changed regarding documentation for asset creation, AFAICT.

Anyway, I created the xml files and bash scripts inside a directory - now what? I tried copying the dir into ~/.phoronix-test-suite/test-profiles/local, then pts/, with no results (the test didi not show up in list-available-test or in list-installed-tests). Is there some special syntax to debug-install a test from a local storage, rather than OB.org repository?

Note - I'm on OS X, so no GUI here.

[Michael]

Sorry to resurrect this old thread, but I'm running into the same problem and, unfortunately, not much has changed regarding documentation for asset creation, AFAICT.

Anyway, I created the xml files and bash scripts inside a directory - now what? I tried copying the dir into ~/.phoronix-test-suite/test-profiles/local, then pts/, with no results (the test didi not show up in list-available-test or in list-installed-tests). Is there some special syntax to debug-install a test from a local storage, rather than OB.org repository?

Note - I'm on OS X, so no GUI here.

If you put the test profile in local/ it should work fine without OB. Just run 'phoronix-test-suite benchmark local/<whatever the sub-folder you called it>'

[myxal]

If you put the test profile in local/ it should work fine without OB. Just run 'phoronix-test-suite benchmark local/<whatever the sub-folder you called it>'

Thanks, now I see what was wrong - I included the version number in the directory name (scan-basic-0.1.0), but PTS apparently doesn't expect this.
Useful note: just name the profile directory normally.

Also, the test is never listed under "list-available-tests", only under "list-installed-tests" after installation - is this expected?

[Michael]

Also, the test is never listed under "list-available-tests", only under "list-installed-tests" after installation - is this expected?

With phoronix test suite 4.4/Git, the local tests will now be shown under list-available-tests. Thanks.