Is x86 architecture hampering performance?

[CTho9305]

Originally posted by: Sahakiel

Originally posted by: imgod2u
Unfortunately, whether the cache blocks or not, the processor will still run into problems when the data needed is not there. This is the major bottleneck in modern systems with processor idle time being in the 200+ cycles. That's way more of a hit than any branch mispredict.

I think you missed the significance of a nonblocking cache, but you do bring up another good point. Although stalls on memory accesses are pretty rare (ignoring compulsory misses), their associated penalties are large enough to be a problem.

The thing is, even with a non-blocking cache, if you have, say, a 50-entry re-order buffer, any delays longer than 50 cycles are guarateend to stall you (assuming none of those intructions take multiplies or otherwise slow you down), because you can't track more instructions than the missing memory access and the next 49 instructions/uops. You also have to consider that it's very likely that within a few instructions, you'll hit something that depends on the result of the memory access, and you have to delay that instruction, and therefore any later ones that depend on it.

 

[Sahakiel]

Originally posted by: CTho9305
The thing is, even with a non-blocking cache, if you have, say, a 50-entry re-order buffer, any delays longer than 50 cycles are guarateend to stall you (assuming none of those intructions take multiplies or otherwise slow you down), because you can't track more instructions than the missing memory access and the next 49 instructions/uops. You also have to consider that it's very likely that within a few instructions, you'll hit something that depends on the result of the memory access, and you have to delay that instruction, and therefore any later ones that depend on it.

Yes, I know there are no magic pills, only a large number of anesthetics. Like I said, once the program is up and running, fetches to main memory are rare, but the penalties are high enough to be a problem. Non-blocking caches simply help alleviate the problem to a certain degree. With a OOOE design, the processor can continue with other calculations until it is forced to stall. Once the data is finally retrieved, the processor can then play catch up and pick up some of the slack, represented by pipeline bubbles. Depending on the processor design (based on speed and costs), you can pick up anywhere from 0-100+% of the slack (probably closer to 20% IRL). That's the flexibility of OOOE.