And I disagree with that. It still stands that a number that can't be repeated is something they could have pulled out their asses.
Let’s take Anandtech’s canned Crysis GPU benchmark results for example. They show a 3870 X2 averaging over 31fps at 1680 resolution with High Quality settings and about 28fps at 1920 resolution at High Quality settings in Crysis. We trust Anandtech to relate benchmark data to us that is “correct.” But I also believe the benchmarking tools it has used in no way relate to real world video card performance you are going to experience at home playing the game. We have actually used the ATI HD 3870 X2 to play the game Crysis and our gaming experience in no way mirrors those graphical settings at those framerates. What if we play Crysis with the resolution and quality settings represented in their review?
To put it plainly, it was a painful gaming experience.
Anandtech’s results in no way suggest to the reader what the video card might actually perform like in Crysis or what resolutions or quality settings might be configured.
Do you want video card reviews that suggest “relative performance of a graphics card” based on timedemo benchmarks when some cards benchmark better than others, or do you want an evaluation of those video cards' in-game performance in the latest and greatest computer games that you are going to be playing with it?
HardOCP is very firm in its commitment to give our readers video card evaluations that will allow them to make good purchasing decisions based on real world expectations of the product. We have no interest in showing you “relative performance” based on a “benchmark.” We sit down and spend hours and hours playing the games on each video card and then share our thoughts and analysis. The simple fact of the matter is that HardOCP’s video card evaluation experiences cannot be replicated by clicking a single mouse button and putting a number on a graph.
Timedemo benchmarking of video cards is broken. We have proven this on the preceding pages with today’s most graphically intensive gaming title. Many will argue that timedemo benchmarking is the only scientific approach to video card performance analysis that can be trusted. Why you would want to trust a performance metric that is in no way shape or form going to relate to your gaming experience is beyond me. There is also no doubt that there are some games out there that benchmark perfectly in relation to their real world gameplay. We just don’t know what they are, and quite frankly we don’t care. Today's Crysis benchmarks that in no way reflect real world gameplay are enough validation for us to keep on doing it “our way.” If you want someone’s idea of overall "relative performance" of a graphics card based on timedemo benchmarks, HardOCP.com is not for you. We are going to make sure that our video card evaluations give you a solid idea of the actual gaming performance you will experience at home when playing the game.