Anand,
Thanks for responding to my post. I do come to your site a lot, probably daily, and on many occasions I have purchased stuff based on your recommendation not just for myself, but for my company and when I recommend some equipment to friends, I base many of my recommendations on data and opinions that you and your team provides. When I do buy or recommend these computers, I try to determine what the users will be using these computers for (and budget of course).
The uses are typically business client PC, heavy Excel, queries to for Access or other client databases, VBA code inside office apps, servers of all kinds (web, database, web, proxy), running server and client scripts (VB, Java), Photoshop, AutoCAD, programming tools, and of course 3D games, ripping CDs (variable bit rate, highest quality takes a while), I often Zip or WinRar large files for sending over slow connections.
The selection of extremely detail SPEC scores with 2 to 3 word explanations of what the individual test probably does gives me no data as to how that test applies to any of the real world situations above. In absence of going over the source code, I am at a loss of what to make of the tests. I like to think of myself that I am no dummy, but after staring at the results for a while I just threw up my hands and said to myself that I have no idea what?s going on. The ?analysis?, (which was something that I voted that I want more of in the latest poll) shed very little light on what was really going on. And even when it did, the different optimization settings often produced results contradicting the analysis.
Which brings me to the main point about the SPEC benchmark. It is heavily dependant on the compiler and compiler settings. A good compiler, the right compiler optimization improved the results sometimes by 50% in favor of one compiler / CPU or another. (In the DDR review, AMD actually seems to have benefited more than Intel)
Did the CPU get 50% better? Hardly. Will I ever run an application that used the particular compiler with those particular settings? Highly unlikely. The quality of the compiler that may never be used in real life may completely throw off the numbers that you report. So now we are not really testing CPUs anymore, but compilers.
Now in a comparison of AMD CPU and an Intel CPU, we have Intel that spends millions on their compiler group with the sole purpose of making their CPUs look better in the SPEC benchmarks (for which we have to pay in higher prices for Intel CPUs) and AMD doesn?t. (Third Party Compaq doesn?t have the same vested interest as much as Intel does).
Let?s take an example of your 183.quake.
The Product A scored 319 and product B scored 263 under the heaviest optimization. What does it tell me? I am afraid nothing. I have not read the code; the fact that it has something to do with an earthquake in Northbridge (rather than Southbridge?) is again no information. Now the analysis says that with little optimization product B beat the pants of product A, with more optimization the result was the opposite, plus some mumbo jumbo about the compilers.
What information am I supposed to take from this? Which real world application does this hint will perform well? Is it WinZip? Photoshop?
How about doing the detail about individual applications that make up the Sysmark or Winstone. How about running WinZip or WinRar, that everybody uses and knows what they do using the shrink-wrapped code and see which CPU/mobo/memory will compress my file faster? I don?t think the roundabout way of deducing it from the Northbridge earthquake simulation, quality of the compiler, optimizations used is better than running the common applications directly in their shrinked wrapped form.
I appreciate your explanation about the editorial policy, and I think it is a good one. As far as ?Naturally speaking?, I was meant to be more general, not directed at Anandtech in particular. All the web site used it (coincidently), and many didn?t have the integrity to say what you said about the optimizations, and the fact that the application was not just a regular app, but the best case scenario for Piii processor, not likely to be seen anywhere else.
But I am sorry that I have to say that what you say about SPEC did not convince me at all, and my questions about the curiosity of the timing and their relevance to real world remain un-answered.
I don?t want to take any more of your time answering my post, it is better spend evaluating hardware and bringing all your great reviews. I just hope the whole SPEC thing, if you are planning to get back to it (and spend all the hours doing it) will be placed in proper perspective (as a pissing contest IMO) and more emphasis will be placed on things we do with the computers to make money, communicate and have fun.