Welcome to Viper's Lair - Benchmarking

As I stated before, we buy video cards to play games, so therefore we should focus on this, instead of the synthetic results that don't really mean much in the way of real life performance. We would also want to be flexible to a certain extent, while holding on to some benchmarks for comparison's sake. Quake III might be a choice for comparisons, but there are many other programs that came out after that, that are just as good. A couple of examples of this are Jedi Knight II (same engine as QIII), Serious Sam SE, or even Max Payne. We as reviewers want to be flexible because the gaming world around is changing at a fast pace, with new games being released with new features, there is a need to show how video cards perform in these environments which can't be done with current games (think Doom3).

This brings us to the next point. Until the release of UT2003, most Direct X benchmarks used in reviews consisted of 3D Mark 2001. There are other options for DirectX benchmarks, one just has to look around. The guys at devised a way to benchmark Max Payne which turned out to be a very good benchmark. An interesting benchmark I found out about at was for a DirectX 8 game called Ballistics, which is a very nice 'racing' game that can be used to test video cards in a non fps game. The last one I'll mention is that of AquaMark, it is harder to come by, but is an interesting real/synthetic benchmark.

Another thing I have noticed sorely lacking from many reviews (even some of mine and others who write for this site) is a lack of quality tests, and more specifically 2D quality tests. Since you are looking at a 2D display for the entire time you use your computer (unless you have some new 3D display), it therefore deserves, rather requires, testing to see its quality in everyday use. The best way I've seen is to have a blind test with multiple people (preferred, to represent regular computer users), using one video card as a base for comparison to the others. Good tests are very importent as well, including text and color images, to test the saturation and quality of the images produced by the video card (not the monitor as it is a control in this case). If more people did this, it would help many people in their purchases (mine included) as I don't want to buy a video card with pathetic 2D quality (and yes many are sold even today).

One last thing we will look at with video card benchmarking is detail. We need to provide you the reader with as much unbiased information as possible to counter any bias we might have. How can we do this? An interesting way of doing this is by providing a graph of the second by second frame rate, while also providing the minimum/average/maximum frame rates. This has been made possible by the program Fraps, which doesn't use up much in the way of resources (~1% of the frame rate) and allows for the frame rate to be recorded every second. Lets look at an example of this in action using the results from Serious Sam SE at 1600*1200 from my Parhelia review. First we'll look at just a bar graph, and then look at the graph found in that review.

Nothing unusual here, it looks as if the Parhelia is just a little bit faster than the Kyro II, apart from that we can't infer much else. What about if we chart the results?

Video Card	Min FPS	Avg. FPS	Max FPS
Kyro II:	0	30.98	40
Parhelia:	18	33.68	56

With the graph as well as the minimum/average/maximum frame rates we can see more about the performance of the card. We see that the Parhelia is more CPU limited than the Kyro II, with the Kyro II dropping to 0 fps twice, not only the once that we can see in the table of results. We can see the graph shows us the most information and allows us to find more about the performance of a card at a given resolution. Now let us look at benchmarking for CPU's/motherboards.