Page 1 of 3 123 LastLast
Results 1 to 10 of 24

Thread: We Can Automatically Find Functional Regressions Too

  1. #1
    Join Date
    Jan 2007
    Posts
    13,421

    Default We Can Automatically Find Functional Regressions Too

    Phoronix: We Can Automatically Find Functional Regressions Too

    Last October we reported on new capabilities within the Phoronix Test Suite that allowed performance regressions to be located within any code-base by leveraging our open-source testing framework with the git-bisect command to automatically traverse a tree in a binary manner until the Phoronix Test Suite spots the regression-causing commit. As an example we tracked down a regression in the mainline Linux kernel with the EXT4 file-system without any manual intervention. Now not only can we automatically track down any performance regressions, but the Phoronix Test Suite can also help you spot any functional regressions...

    http://www.phoronix.com/vr.php?view=ODM1Mw

  2. #2
    Join Date
    Jun 2009
    Posts
    10

    Default

    Great stuff. I think the automatic testing work you're doing for linux is badly needed and unappreciated.

    However, (and this is very off-topic), do you think you could tone down the advertising on your site? I just had to install adblock because I physically couldn't read this article with the flashing something2media ad to the side. The roll-over links are pretty annoying too. Static banners are fine.

    Also, 90% of the real links in the article just go to a generic search (engadget do it a lot too), which is pretty useless. I assume you're doing it to increase your pagerank, but I'm pretty sure google doesn't count intra-site links.

    Keep up the good work.

  3. #3
    Join Date
    Jan 2009
    Location
    Columbus, OH, USA
    Posts
    323

    Default

    Oh hey, you reported the bug! Thanks!

  4. #4
    Join Date
    Feb 2008
    Location
    Linuxland
    Posts
    4,733

    Default

    Good stuff on the reporting.

    Though, git bisect was already scriptable; how does using PTS for this change things?

  5. #5

    Default

    Quote Originally Posted by curaga View Post
    Though, git bisect was already scriptable; how does using PTS for this change things?
    PTS makes it very easy to just plug-in the test conditions and then to deploy a performance or functional regression test in a turn-key manner that's fully automated and reproducible.

  6. #6
    Join Date
    Jan 2009
    Location
    Vienna, Austria; Germany; hello world :)
    Posts
    607

    Default

    Quote Originally Posted by Timmmm View Post
    Great stuff. I think the automatic testing work you're doing for linux is badly needed and unappreciated.

    However, (and this is very off-topic), do you think you could tone down the advertising on your site? I just had to install adblock because I physically couldn't read this article with the flashing something2media ad to the side. The roll-over links are pretty annoying too. Static banners are fine.

    Also, 90% of the real links in the article just go to a generic search (engadget do it a lot too), which is pretty useless. I assume you're doing it to increase your pagerank, but I'm pretty sure google doesn't count intra-site links.

    Keep up the good work.
    please think of the (potential) epileptics among your readers (I'm not)


    and thanks for adding a very useful feature & reporting it

  7. #7
    Join Date
    Sep 2008
    Posts
    989

    Default

    Finding functional regressions is even more useful than finding performance regressions. Functional regressions are through-and-through defects, whereas performance is extremely situational.

    OTOH, coming up with a true/false script for determining whether a defect is present can prove challenging. Unless you provide some helpers along those lines, it's basically another open programming environment prompting the developer to come up with an automatic way to determine if defects are present. Making that determination is often left up to test cases written in the native programming language of the project.

  8. #8
    Join Date
    Oct 2009
    Posts
    15

    Default

    But where is the bug? (follow the bugreport).
    I guess some scripting to check if the latest libs are used would be useful too.

    Waiting for PTS to catch some real bugs. Wonder what the revised title of this article would be...

  9. #9
    Join Date
    Oct 2007
    Location
    Sweden
    Posts
    174

    Default

    You dealt with it professionally. Good work.

  10. #10
    Join Date
    Dec 2007
    Location
    Edinburgh, Scotland
    Posts
    574

    Default

    Finally a decent article where you did something useful with the PTS!

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •