It may seem a bit obsessive to pick on a review of hardware that most people on this board consider a waste of time, but you must understand that many consumers have not the slightest interest in fragging a Quake3 monster. The vast majority of computers purchased today are used primarily for standard office/content creation tasks, and this is where integrated video chipsets show their value. If AnandTech wants to attract a broader audience and be the top source for those who make the purchasing decisions for such machines (computer resellers and consultants, network administrators, IT managers), they need to put just as much effort into their reviews of so-called "value" products as of "performance" products.
But AnandTech's recent Integrated Video Chipset Roundup leaves too many questions unasked.
1) Is it wise to use a batch of reference boards in such a comparison?
We've seen time and again how performance can vary a great deal from an engineeing sample to a final product. While AnandTech's reviews are usually most thorough, one consistent criticism has been their relative lack of attention to the true status of a product: Is it an engineering sample? How close is it to the final product? Are units shipping at this moment? How do those differ from the unit being reviewed? What room is their for improvement? Any hardware enthusiast looking for a solid judgement on a product will tell you that the one thing they can't stomach is uncertainty. There's nothing worse than reading a nice review and wondering if the benchmarks will be meaningless in two weeks due to new BIOS/driver/PCB revisions. This is especially true in the case of brand new chipsets -- manufactures work with chipset designers to implement dozens of little BIOS tweaks that collectively can have a significant impact on performance.
As we speak, KM133 motherboards are available from Microstar and Chaintech, and SiS 730S boards are shipping from PC Chips. Why not get your hands on these products and evaluate what consumers will actually be using, as opposed to engineering samples and reference boards that will never see the light of day?
2) Why does the KT133 perform 2 to 5% better than the KM133 on standard Office / Content Creation benchmarks when using an external graphics accelerator? Memory bandwidth issues should be non-existant here, leading one to believe that the deficiency is due to a pre-release, unoptimized board. But we can't say for sure. That's the problem.
3) Why is the Duron just barely edging out the Celeron using integrated video on Office / Conent Creation benchmarks?
Supposedly the ProSavage and SiS 300 graphics cores are not as fast in 2D as their Intel counterpart. But does 2D performance even matter today? Has anyone in the past five years ever been doing some computer work and thought to themselves, "Boy I wish that display speed was faster"? Performance in 2D screen drawing has been better than human vision ever since the advent of the PCI bus. I can't possibly imagine a situation where 2D speed is an issue these days.
Here's a possible explanation: the Intel chipset's more efficient memory controller enables it to better cope with the bandwidth burden of an integrated UMA archtiecture, resulting in less of a performance decrease for other memory-intensive processes running concurrently with the integrated video. Is this correct? I don't know. But it should have been explored, because we all know that the Duron should be wiping the floor with the Celeron, not just politely edging it out.
Perhaps current office/content creation benchmarks are too reliant on 2D performance. After all, the real delay in most business environments is hard drive access due to virtual memory when running multiple applications. Maybe we need some benchmarks that produce scores more realistic with everyday Windows usage. SysMark 2000, for instance, makes a big deal of the time it takes to apply a filter onto an enormous graphic in Photoshop (of the size that no one but professional DTP workers will encounter; typical web graphics are puny, filtered in milliseconds), to convert a digital media file with Windows Media Encoder 4.0, or to apply a special effect in Elastic Reality. Is that the kind of stuff your average SOHO worker is doing on a daily basis? Of course not. We need benchmarks that stress heavy business multitasking, which is where even today's systems can look slow. Just look at a typical business machine and you'll find about a dozen useless apps running in the system tray, a pointless virus scanner inspecting every single file the OS touches, and a backup tool constantly archiving data, while the user is doing relatively mundane things like accounting with Quicken or web browsing with IE or correspondence with Word. Disc access and memory performance are the keys here; 2D speed is as fast as it'll ever need to be.
4) Why is the Celeron consistently beating the Duron in gaming benchmarks with integrated video?
Even when the integrated graphics core remains the same (ie. KM133 vs PM133 or 730S vs 630S) the Celeron 800 scores higher than the much more powerful Duron 800 on 3D gaming benchmarks. What is to blame here? I have no idea, and the review didn't even raise the issue.
5) Why not examine the DVD performance issue with the KM/PM133?
Obviously something is wrong with the benchmark or the drivers or both. It's a bit misleading because CPU utlization while watching a DVD shouldn't be a big deal anyway -- you're not trying to play Quake at the same time. But there should have been some research into the problem, because the Savage4 has historically featured excellent DVD performance, and its Hardware Motion Compensation and iDCT features should at least allow it to beat the ALi Alladin TNT2 which lacks them.
Modus
But AnandTech's recent Integrated Video Chipset Roundup leaves too many questions unasked.
1) Is it wise to use a batch of reference boards in such a comparison?
We've seen time and again how performance can vary a great deal from an engineeing sample to a final product. While AnandTech's reviews are usually most thorough, one consistent criticism has been their relative lack of attention to the true status of a product: Is it an engineering sample? How close is it to the final product? Are units shipping at this moment? How do those differ from the unit being reviewed? What room is their for improvement? Any hardware enthusiast looking for a solid judgement on a product will tell you that the one thing they can't stomach is uncertainty. There's nothing worse than reading a nice review and wondering if the benchmarks will be meaningless in two weeks due to new BIOS/driver/PCB revisions. This is especially true in the case of brand new chipsets -- manufactures work with chipset designers to implement dozens of little BIOS tweaks that collectively can have a significant impact on performance.
As we speak, KM133 motherboards are available from Microstar and Chaintech, and SiS 730S boards are shipping from PC Chips. Why not get your hands on these products and evaluate what consumers will actually be using, as opposed to engineering samples and reference boards that will never see the light of day?
2) Why does the KT133 perform 2 to 5% better than the KM133 on standard Office / Content Creation benchmarks when using an external graphics accelerator? Memory bandwidth issues should be non-existant here, leading one to believe that the deficiency is due to a pre-release, unoptimized board. But we can't say for sure. That's the problem.
3) Why is the Duron just barely edging out the Celeron using integrated video on Office / Conent Creation benchmarks?
Supposedly the ProSavage and SiS 300 graphics cores are not as fast in 2D as their Intel counterpart. But does 2D performance even matter today? Has anyone in the past five years ever been doing some computer work and thought to themselves, "Boy I wish that display speed was faster"? Performance in 2D screen drawing has been better than human vision ever since the advent of the PCI bus. I can't possibly imagine a situation where 2D speed is an issue these days.
Here's a possible explanation: the Intel chipset's more efficient memory controller enables it to better cope with the bandwidth burden of an integrated UMA archtiecture, resulting in less of a performance decrease for other memory-intensive processes running concurrently with the integrated video. Is this correct? I don't know. But it should have been explored, because we all know that the Duron should be wiping the floor with the Celeron, not just politely edging it out.
Perhaps current office/content creation benchmarks are too reliant on 2D performance. After all, the real delay in most business environments is hard drive access due to virtual memory when running multiple applications. Maybe we need some benchmarks that produce scores more realistic with everyday Windows usage. SysMark 2000, for instance, makes a big deal of the time it takes to apply a filter onto an enormous graphic in Photoshop (of the size that no one but professional DTP workers will encounter; typical web graphics are puny, filtered in milliseconds), to convert a digital media file with Windows Media Encoder 4.0, or to apply a special effect in Elastic Reality. Is that the kind of stuff your average SOHO worker is doing on a daily basis? Of course not. We need benchmarks that stress heavy business multitasking, which is where even today's systems can look slow. Just look at a typical business machine and you'll find about a dozen useless apps running in the system tray, a pointless virus scanner inspecting every single file the OS touches, and a backup tool constantly archiving data, while the user is doing relatively mundane things like accounting with Quicken or web browsing with IE or correspondence with Word. Disc access and memory performance are the keys here; 2D speed is as fast as it'll ever need to be.
4) Why is the Celeron consistently beating the Duron in gaming benchmarks with integrated video?
Even when the integrated graphics core remains the same (ie. KM133 vs PM133 or 730S vs 630S) the Celeron 800 scores higher than the much more powerful Duron 800 on 3D gaming benchmarks. What is to blame here? I have no idea, and the review didn't even raise the issue.
5) Why not examine the DVD performance issue with the KM/PM133?
Obviously something is wrong with the benchmark or the drivers or both. It's a bit misleading because CPU utlization while watching a DVD shouldn't be a big deal anyway -- you're not trying to play Quake at the same time. But there should have been some research into the problem, because the Savage4 has historically featured excellent DVD performance, and its Hardware Motion Compensation and iDCT features should at least allow it to beat the ALi Alladin TNT2 which lacks them.
Modus