Page 1 of 4 123 ... LastLast
Results 1 to 10 of 47

Thread: Linux 2.6.24 Through Linux 2.6.33 Benchmarks

Hybrid View

  1. #1
    Join Date
    Jan 2007
    Posts
    14,759

    Default Linux 2.6.24 Through Linux 2.6.33 Benchmarks

    Phoronix: Linux 2.6.24 Through Linux 2.6.33 Benchmarks

    At Phoronix we have been benchmarking the Linux kernel on a daily basis using Phoromatic Tracker, a sub-component of Phoromatic and the Phoronix Test Suite. We launched our first system in the Linux kernel testing farm just prior to the Linux 2.6.33 kernel development cycle and found a number of notable regressions during the past three months. Now with the Linux 2.6.34 kernel development cycle getting into swing, we have added an additional two systems to our daily kernel benchmarking farm. One of the systems is an Atom Z520 system but what makes it more interesting is that the system is using a Btrfs file-system and then the second new system added to the kernel tracker is a 64-bit setup. However, to provide a historical look at the Linux kernel performance, we have ran some fresh benchmarks going back to the Linux 2.6.24 kernel and ending with the recently released Linux 2.6.33 kernel.

    http://www.phoronix.com/vr.php?view=14633

  2. #2

    Default

    I can't believe such big regressions like apache where left untouched and unresolved during so much release. I don't understand either why the PostgreSQL performance regression passed through the rc phase... Either you just prove to the worl the kernel development model is flawed, either their is a flaw in the tests you ran. In both case, some analysis and explanation would have been nice along the results.

  3. #3

    Default

    Quote Originally Posted by Xheyther View Post
    I can't believe such big regressions like apache where left untouched and unresolved during so much release. I don't understand either why the PostgreSQL performance regression passed through the rc phase... Either you just prove to the worl the kernel development model is flawed, either their is a flaw in the tests you ran. In both case, some analysis and explanation would have been nice along the results.
    Usually those tests are meaningless. If there's intentional change in the kernel it's described here like regression. In example some file system has different mode set as default in newer kernel and this can make big impact on some benchmarks - PostgreSQL etc. Apache benchmark is meaningless, because it's not done properly.

  4. #4

    Default

    Quote Originally Posted by kraftman View Post
    Apache benchmark is meaningless, because it's not done properly.
    http://lkml.indiana.edu/hypermail/li...9.1/02631.html

    Be careful not to run ab on the same machine as you run apache, otherwise
    the numerous apache processes can limit ab's throughput. This is the same
    reason as why I educate people so that they don't run a single-process
    proxy in front of a multi-process/multi-thread web server. Apparently
    it's not obvious to everyone.

  5. #5
    Join Date
    Nov 2009
    Posts
    328

    Default

    I like this article a lot!! testing from 24-33, When I first read the Phoromatic Tracker i was wondering if that could be done. Phoromatic Tracker is a superb tool! well done.

    I always thought that there will be regression on performance from 24 to 33, because kernel is implementing more APIs, graphics code, graphics focus... but this article shows me that i was worng.

  6. #6
    Join Date
    Feb 2008
    Location
    127.0.0.1
    Posts
    89

    Default

    I was very excited about PTS at the beginning.
    Now it produces meaningless numbers. your conclusions based on the PTS results could be wrong.
    Like with HD test. It's results leading to very wrong assumption. 20$ CPU can not play HD movie with high bit rate.
    I can't find testing file systems with different settings producing something useful.
    You can use some tests to evaluate some components , but i can't see the point to post this 24-33 tests. they are at least misleading.

  7. #7

    Default

    Quote Originally Posted by Jimbo View Post
    I always thought that there will be regression on performance from 24 to 33, because kernel is implementing more APIs, graphics code, graphics focus... but this article shows me that i was worng.
    This article mainly shows that tests are often meaningless.

  8. #8
    Join Date
    Jun 2006
    Posts
    311

    Default

    Skip to the next response to that thread..

    I turned on apache, and played with ab a bit, and yup, ab is a hog, so
    any fairness hurts it a badly. Ergo, running ab on the same box as
    apache suffers with CFS when NEW_FAIR_SLEEPERS are turned on. Issuing
    ab bandwidth to match it's 1:N pig nature brings throughput right back.


    http://lkml.indiana.edu/hypermail/li...9.1/02861.html

    Remember that you can't test anything, and testing in the obvious path will usually result in flat lines - since they represent the 95% path.

    As indicated above, what has been identified is that in some scenarios CFS completely tanks. The ab is just a tool to make this visible.

    As per usual, if there is any benchmark which you believes provides a suitable equivalent scenario but is more "correct", please tell us.

  9. #9

    Default

    Quote Originally Posted by mtippett View Post
    Skip to the next response to that thread..

    I turned on apache, and played with ab a bit, and yup, ab is a hog, so
    any fairness hurts it a badly. Ergo, running ab on the same box as
    apache suffers with CFS when NEW_FAIR_SLEEPERS are turned on. Issuing
    ab bandwidth to match it's 1:N pig nature brings throughput right back.


    http://lkml.indiana.edu/hypermail/li...9.1/02861.html

    Remember that you can't test anything, and testing in the obvious path will usually result in flat lines - since they represent the 95% path.

    As indicated above, what has been identified is that in some scenarios CFS completely tanks. The ab is just a tool to make this visible.
    In this scenario with fair sleepers enabled, yes. However, this scenario is one out of the reality until someone runs apache on a single machine which is not recommended.* I think it's something natural scheduler is tuned to perform well in real situations. So, what's the point of this benchmark?

    As per usual, if there is any benchmark which you believes provides a suitable equivalent scenario but is more "correct", please tell us.
    Maybe replace "correct" by more meaningful. The problem is I'm not sure what could be equivalent scenario to this benchmark and if there's no such scenario this benchmark means: we've got different results in Apache benchmark running on the same machine which isn't recommended. Btw. about what scenario you were talking about? Such *?

  10. #10
    Join Date
    Jun 2006
    Posts
    311

    Default

    Quote Originally Posted by kraftman View Post
    Usually those tests are meaningless. If there's intentional change in the kernel it's described here like regression. In example some file system has different mode set as default in newer kernel and this can make big impact on some benchmarks - PostgreSQL etc. Apache benchmark is meaningless, because it's not done properly.
    A Regression is a unexpected change in behavior. If the kernel developers make a change in one area, and they are not expecting the behavior change in other areas those areas have regressed.

    Remember that file system tuning (turning features on and off) is a specialist skill. Most people get very concerned with making changes that they may believe are uninformed when they may be risking their data. The maintainers of the filesystems and the distros that package them are the ones that control the behavior.

    I'd like you to expand on your "not done properly" if you could.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •