Page 2 of 2 FirstFirst 12
Results 11 to 13 of 13

Thread: The Past 12 Linux Kernels Benchmarked

  1. #11
    Join Date
    Aug 2006
    Posts
    90

    Default

    I hate to sound ungrateful, I mean so far no one has done these sorts of benchmarks, and they're useful. However, its possible to greatly improve how useful these numbers could be with a little extra care. First, what number of tests did you run? Was it 1, 10, 100? If it is stated in the article I missed it, but the average of 100 tests carries a lot more weight than just 1. Ideally, an error bar, standard deviantion, or confidence interval would also be included. At least some measure of how sure you are that the result is indicative of the measurement. This is especially important in cases where the benchmarks are very close, since it lets us run a correlation test to make sure they are actually statistically different.

    Also, on a more pedantic level is the number of significant figures in the graphs. It should reflect the precision of the score. The general rule of thumb is everything but the last digit should be fixed on repeat of measurement (or withing the standard deviation). A score of 28.1293 for example, means that every time you get 28.129x. Ssam's data for example would only warant a score of 63 MB/s, even though the actual average of the three scores is 63.47 MB/s since both the units digit varied between measurements and the standard deviation is roughly ~.75 MB/s.

    Kcalc (the standard KDE calculator app) and a wide variety of Linux graphics packages can compute this stuff automatically.

  2. #12

    Default

    Quote Originally Posted by Tillin9 View Post
    I hate to sound ungrateful, I mean so far no one has done these sorts of benchmarks, and they're useful. However, its possible to greatly improve how useful these numbers could be with a little extra care. First, what number of tests did you run? Was it 1, 10, 100?
    It's the average of four runs for each test.

  3. #13
    Join Date
    Jun 2008
    Posts
    197

    Default

    At least we know Linux is not getting slower, like windows.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •