Page 1 of 11 123 ... LastLast
Results 1 to 10 of 105

Thread: What Shall We Benchmark Next? Let Us Know!

Hybrid View

  1. #1
    Join Date
    Jan 2007
    Posts
    15,646

    Default What Shall We Benchmark Next? Let Us Know!

    Phoronix: What Shall We Benchmark Next? Let Us Know!

    With Phoromatic we make it easy to build a test farm for benchmarking and automated regression management and to prove it we began monitoring the Linux kernel performance on a daily basis where we track the performance of the latest Linux kernel code on a daily basis using multiple systems. This has been going on for months and then yesterday we announced that Phoromatic reached a 1.0 status and that we have launched Phoromatic Ubuntu Tracker, a way to monitor the performance of Ubuntu Linux as a whole on a daily basis by benchmarking the most recent development packages. A day has passed so now we are already thinking of what next to add to our test farm for continuous performance tracking...

    http://www.phoronix.com/vr.php?view=ODA3NA

  2. #2
    Join Date
    Feb 2009
    Posts
    4

    Default

    Why not do some testing of popular applications on Wine on a daily basis? Wine's git HEAD is rather stable (Alexandre does unit tests + compile test before he commits anything) so that should work fairly well.

  3. #3
    Join Date
    Jan 2010
    Posts
    10

    Default Wine3d!

    Wine for sure, I use it every day for Warcraft 3, Starcraft one and two. And I think I'm not the only one

    Both opengl and directx may be very interesting on different cards/drivers.

  4. #4
    Join Date
    Jul 2009
    Posts
    261

    Default

    i guess anything that might help in finding regressions in any way. but it ll be hard to find anything that evolves as fast as the kernel, so that daily snapshots would make sense.
    what im missing personally is to see other hardware drivers being benchmarked, like WLAN cards, or chipsets. i would like to see if there are differences in reliability and performance, and if theres any development.
    maybe different programs: wicd vs. network manager
    that would be rather ideas for testing once in a while.
    But a transfer rate/data loss/signal strengh tests for instance would be interesting.
    i have the feeling its being neglected a bit.

  5. #5
    Join Date
    Mar 2010
    Posts
    4

    Default

    Few weeks ago, I asked if it was possible to bench KDE4 vs KDE3 as it was said in the early devs of KDE4 that it would consume less memory than KDE3.And then no news about it.
    I'm just curious
    My post: http://www.phoronix.com/forums/showthread.php?t=22401

  6. #6
    Join Date
    May 2009
    Posts
    10

    Default FreeBSD and Networking

    Hi Phoronix,

    Why don't you track the performance of another kernel, such as FreeBSD? It will be interesting to compare the development approach taken by the different camps. Does FreeBSD experience performance regressions (as Linux recently did with ext4fs)? Does FreeBSD get faster or slower with every release?

    I wouldn't expect UFS2 to change much as it is tried and tested (although the introduction of Soft Updates + Journaling [SUJ] might change that). ZFS, on the other hand, may experience significant fluctuations as fixes and improvements are implemented. The kernel is also slowly moving away from the GIANT lock (similar to the python GIL) so that may result in performance increases (or decreases).

    Another area of benchmarking that I do not recall Phoronix doing is on the network stack. How does Linux, FreeBSD and other OS's compare in TCP/IP(6) throughput, load balancing, firewalls, etc. How resilient are the kernels to DOS attacks and other maliciousness.

    With clang maturing and FreeBSD taking a close look at adopting it that might be another area for consideration. I would expect the compilation time, using clang, to increase as more features (especially C++) are implemented yet the runtime (of the compiled binaries) should decrease as more optimizations are implemented. This could also be compared to GCC 4.x progression.

    Regards

  7. #7
    Join Date
    Sep 2009
    Posts
    33

    Default

    I'm amazed that you have got a tool to do this. way to go!

    I''d like to see how Wine and gfx is improving, but that would be a bi-weekly event if it's remains released each fortnight.

    Is this request viable?

  8. #8
    Join Date
    Jun 2008
    Location
    New Zealand
    Posts
    68

    Default

    A wine benchmark would be cool, if you could base it on a native game, which also has an openGL and a Direct3D renderer for the Windows version.
    (ie Unreal 2004)

    So you could then test with lower spec cards from AMD and NVidia, and whatever graphics Intel have, and see what it looks like with the native performance, and the wine performance in the drivers with Direct3D and openGL.

  9. #9

    Default

    Quote Originally Posted by poofyyoda View Post
    So you could then test with lower spec cards from AMD and NVidia, and whatever graphics Intel have, and see what it looks like with the native performance, and the wine performance in the drivers with Direct3D and openGL.
    Unlikely to do all of that for a tracker unless there is significant upstream interest in the tracker and also an increase in use of the affiliate shopping links / Premium subscribers / anything to justify the electricity for a number of systems dedicated to that task.

  10. #10
    Join Date
    Mar 2010
    Posts
    60

    Default

    The question here really is to find something with an open development model, regular updates and relevance to people and companies.


    Once thing that comes to mind would be to benchmark android.

    I don't know if xorg-eddgers ppa is being benchmarked along with the newest kernel.

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •