Yes, you have briefly described how IBM has designed additional hardware to prevent physical aliasing across different sets in the first-level data cache. This additional hardware is not free and would not be required in a physically-indexed cache.L1 cache is only accessed with VIVT tags - actually mictotags which are only partial as 64bit address space ain't needed to access few kilobytes of memory. Every L1 cache line has also PIPT tags which are used when cache line is loaded or checked for coherency- either in L2 when L2 is inclusive or in L1 tag directory. Only data access from core to L1 cache is accessed with virtual address - all other cache handling through PIPT tags and absolutely same as doing it with only PIPT.
IBM is documented extremely well their micto-tagged access with directory based full physical tags sceheme for their Z-series cpus, I recommend looking it if want to know more about it.
We're talking about potential here. Revenue and marketshare wise the x86 vendors have more incentive to focus on servers, and ARM has less.But thats the point , you cant do that with good performance because you have to have a cache control protocol that actually works across that. So then when you go and look at the latest ARM large scale fabrics they are significantly worse for data locality then what AMD are doing let alone what the ARM client cores look like.
It's the same reason I gave Core 2 as an example. Core 2 was a very good client core. If Intel never developed Nehalem's P2P interconnect, and the memory controller you could have also given the same argument.I'm not suggesting here that ARM is inherently superior, but the take some have that ARM's performance is only applicable to mobile and laptops and synthetic benchmarks, but it is unsuited for "real world performance" in high end gaming/workstation/servers is laughable.
Fair, but where is this M4 192 core ARM server processor?That's a terrible take. You're comparing a state of the art x86 core against a core far worse than the state of the art in ARM land. Put 192 M4 cores together with the same amount of inter-core communication and memory controller resources that the Zen 5c Epyc has and you'd see a totally different story. Or for a comparison where both sides are fighting with one hand tied behind their backs put 192 Bulldozer cores up against that AmpereOne.
Your example has absolutely nothing to do with "ARM not being able to translate between synthetic benchmarks and real world performance" and everything to do with AmpereOne having a crappy core, and its marketers having cherry picked a few benchmarks that show it off in the best light.
Yes I know what I'm suggesting to compare with doesn't exist, but it isn't because Apple couldn't build that 192 core monster it is because they have no interest in competing in the high end server market (or for that matter in the low end server market)
I'm not suggesting here that ARM is inherently superior, but the take some have that ARM's performance is only applicable to mobile and laptops and synthetic benchmarks, but it is unsuited for "real world performance" in high end gaming/workstation/servers is laughable.
Fundamentally, modern CISC and RISC based processors are not all that different. The idea that somehow ARM with its RISC instruction set will somehow blow away every x86 server chip that came before it is just BS.
Everyone seems to think that there is something magical about ARM and that magic will work everywhere.
Pretty sure most people here aren't naive enough to believe these things. Perhaps in other places they do.It seems like lots of people are very willing to believe that AMD and Intel are simply full of stupid engineers and ARM companies are just so smart
As a member of Arm’s highest performance core line, Neoverse V2 is a large core with width and reordering capacity on par with AMD’s Zen 4. However, it stops short of being a monster like Golden Cove, Oryon, or Apple’s Firestorm. As implemented in Graviton 4, Neoverse V2 runs at up to 2.8 GHz.
Several people have posited that an ARM core wouldn't benefit from SMT as much as an X86 core would. Not necessarily due to the ISA, but due to other microarchitectural design choices. Indeed, the fact that none of ARM's server cores (except the first gen Neoverse E1) have SMT, might be a testament to that.As I have said, lets start with getting an ARM processor that has decent SMT
ARM designs are wide, so yes it can take advantage of SMT.Several people have posited that an ARM core wouldn't benefit from SMT as much as an X86 core would. Not necessarily due to the ISA, but due to other microarchitectural design choices. Indeed, the fact that none of ARM's server cores (except the first gen Neoverse E1) have SMT, might be a testament to that.
How is it an afterthought? Apple M4 does pretty damn well on Cinebench 2024, at fraction of the power.ARM has risen to supremacy in phones and tablets where ultra low power operation is key and multi-thread is an afterthought.
Do you know what was the excuse before this? "Oh ARM chips can never reach x86 level IPC in any way shape or form". Well they did. And it's beating in absolute single thread performance against desktop processors using 3-4x the power, and sometimes killing itself to reach those clocks.The care and feeding of a massively parallel DC processor is more complex than just a high IPC single core design. It seems like lots of people are very willing to believe that AMD and Intel are simply full of stupid engineers and ARM companies are just so smart that of course they can make a core design originally targeting a cell phone that surpasses them with little to no effort just by using the current best in class ARM core.
Have you thought maybe they shouldn't have done 512-bit vectors? AVX512 should have been AVX3-256.As I have said, lets start with getting an ARM processor that has decent SMT and a SIMD engine that can do more than 128bit paths (you know, where x86 was over a decade ago).
Well QC did dip their toes in with Falkor - but like several others at the time they bought in to a market that wasn't yet ready for ARM servers.Will be interesting to see whether Nuvia as part of Qualcomm ever gets to enter that market.
AVX512 to AVX2 is what SVE is to Neon. You would need to recompile anyway. What was stupid was the artificial market segmentation where client Skylake had AVX512 disabled. It could either handle it as Zen1 did handle AVX2 or Zen4 did handle AVX512. Another not so wise thing, was making the 512b mandatory, and 256/128b optional, as this lacked foresight as we can now see.Have you thought maybe they shouldn't have done 512-bit vectors? AVX512 should have been AVX3-256.
ARM just added more 128-bit units, which is superior when you can do so because it benefits all FP code, even legacy, while wider vectors need recompiling.
Intel probably did not have plans to make heterogenous CPU configurations when AVX512 was first proposed.Another not so wise thing, was making the 512b mandatory, and 256/128b optional, as this lacked foresight as we can now see.
I agree, but at the same time they wanted to keep Skylake die as small as possible to maximize profit so if they made 256b default they could provide new instructions and bigger architectural register pool on client too. And that would widen adoption. But its if and maybes nowIntel probably did not have plans to make heterogenous CPU configurations when AVX512 was first proposed.
Yes, you have briefly described how IBM has designed additional hardware to prevent physical aliasing across different sets in the first-level data cache. This additional hardware is not free and would not be required in a physically-indexed cache.
Virtually-indexed caches must provide alias resolution hardware, while physically-indexed caches must obey certain size constraints. These are the trade-offs.
Yes, after excluding the hardware and design choices that solve the aliasing problem in any particular processor, there's no additional hardware requirement. This is a tautology.There's no hardware other than physical tagging to solve aliasing and homonym problems. Every memory coherent multi-core systems needs that every cache line in system has physical tags. AMD has also documented how they do their L1 virtual tagging - L2 is inclusive of L1 and every L1 cache line has physical tags in L2, and L2 handles L1 loads and evictions. Doing TLB conversion for every load means that for every 64 bit cache load there's need to load and compare thousands of bits metadata(from TLB - which is cache for cpu MMU) which ain't needed when L1 is accessed from core side with linear address. IBM did also describe that doing physical tags and tlb conversions for many loads per cycle will drive those TLB accesses hottest part of the cpu limiting clock potential. For comparison doing 8 bit mictotags for L1 like AMD does only needs comparing 64 bits of L1 tags before L1 load.
Each memory operation that executes in the processor must check permissions in order to provide precise exceptions. Accordingly, it's not possible to fully escape the TLB. However, the TLB output is not always required to begin cache access. This is the critical advantage of virtual indexing, including cases where the virtual and physical indices are guaranteed to be identical, as in the caches discussed above.
The use of a smaller tag for fast, "good enough" hit/miss analysis does not mean that a full tag match is not required. A reduced-size tag can correctly detect the vast majority of cache misses, but it cannot prove that a cache hit has definitely occurred. This permits the full tag comparison to be delayed without significant harm to the performance of the processor, because it's rare for cache misses to be detected with the full tag.
Yes, you have briefly described how IBM has designed additional hardware to prevent physical aliasing across different sets in the first-level data cache. This additional hardware is not free and would not be required in a physically-indexed cache.
Virtually-indexed caches must provide alias resolution hardware, while physically-indexed caches must obey certain size constraints. These are the trade-offs.
The explanations above appear sound to me. Duplicating information in a CPU to reduce timing criticality very frequently worth the area cost. However, none of these techniques provide any explanation about the originating question on page 1: why is Qualcomm's 96 KiB, 6-way L1D cache so much larger than the competition? The fact remains that IBM (32 KiB, 8-way), Intel (48 KiB, 12-way), and AMD (48 KiB, 12-way) are not paying the tax to handle aliases across multiple cache sets in their first-level data caches.
Looking up the duplicate tags to prevent aliases, regardless of when the lookup is performed, has a power cost. That cost scales with the number of tags that have to be checked. All of the other techniques that you have mentioned are important parts of the state of the art of microprocessor design, but they do not clearly answer the original question.
It may simply be the case that the reduced associativity of the cache compensates for the additional coherency management logic. Coherency traffic has to check 4 cache sets * 6 ways = 24 tags per cache line, but normal cache access for loads and stores is limited to the 6 ways in a single cache set. The density of the cache itself could also be a relevant factor.
I previously had searched for only the cache sizes in IBM z17 (reportedly 32K L1I and 32K L1D), but I didn't look at it closely enough: the snippets from Hot Chips are for the DPU, a separate part of the system architecture from the main CPU cores. I apologize for the error. Indeed, IBM zSeries has been paying the power cost of the extra tag lookups that enable a larger L1 cache for several generations.But that cache scheme is design choice not technical limit. L1 cache ways could easily made larger than page size as Qualcomm - Apple and IBM demonstrates(IBM Z-series L1d is 128KB -8way and 4KB pages are supported). x86-vendors just seems to be sticking one scheme they have used and changing to other design seems to be hard uphill. But they need to as that cache solution they now use ain't power efficient - all data movement needs power and keeping more data at L1 instead of shuffling it back and forth between cache levels will be more power efficient. Having TSO-policy on memory coherency might have part of why x86 sticks to that kind of scheme - L1 ways aliased stores are big problem as there's always possibility that after L1 ways run out whole cpu could be halted. Increasing L1 capacity doesn't solve problem, more ways help as do possibility to swap way cache lines with upper cache levels. TSO doesn't do anything useful after all so it's just a handicap that x86 seems to not get rid of.
Has little to do with that. 256-bit should have been the maximum. In Skylake they skipped 512 bit because it needed too many compromises. Clocks, area. Why bother with it when vast majority is better off with a GPU? More and more so.I agree, but at the same time they wanted to keep Skylake die as small as possible to maximize profit so if they made 256b default they could provide new instructions and bigger architectural register pool on client too. And that would widen adoption. But its if and maybes now
Because the two x86 vendors are broken. Can't stay a decade without fumbling to the point where people start wondering whether the company can exist.Intel probably did not have plans to make heterogenous CPU configurations when AVX512 was first proposed.
Well, first off. The decoders in Zen5 are used differently than the decoders in Skymont.Question:
What are the Pros and Cons of;
(1) Unified Decoders ?
(2) Clustered Decoders ?
Two latest microarchitectures that use clustered decoders are Zen5 (4+4) and Skymont (3+3+3).
On the other hand, most other microarchitectures use Unified decoders: Apple M4-P (10), Cortex X925 (10), Oryon-L (8), Lion Cove (8), etc...
Both Zen5 and Skymont are x86 cores. So it is not a coincidence that they use clustered decoders, because clustered decoders take up less transistors than a unified decoder of equivalent width. Considering how complex decoding on x86 can be, using clustered decoders is the economical path.
But some people are suggesting that we might eventually see clustered decoders on ARM cores too.
View attachment 112673
Superficially, Zen 5’s frontend looks like the ones in Intel’s latest E-Cores. However, they don’t work the same way. Each Zen 5 cluster only handles a single thread, and maximum frontend throughput can only be achieved if both SMT threads are loaded.
Dual decode clusters came up in sideline discussions. The core only uses one of its decode clusters when running a single thread, regardless of whether the sibling thread is idle or SMT is turned off. From those sideline conversations, apparently the challenge was stitching the out-of-order instructions streams back in-order at the micro-op queue. The micro-op queue is in-order because it has to serve the renamer, and register renaming is an inherently serial process. I asked if having 8-wide decode for a single thread would substantially help IPC. They acknowledged there were some cases where 8-wide decode might have performance benefits, but also noted Zen 5 aims to run a lot of code out of the micro-op cache. I also brought up Intel’s E-Cores, which can use both decode clusters for a single thread. They pointed out that Zen 5 has completely different frequency targets, and I can see their point. Something easy to do on a low performance, 3.8 GHz core like Crestmont might be harder on a high performance 5.7 GHz core like Zen 5.
Claim makes zero sense to me.Question:
What are the Pros and Cons of;
(1) Unified Decoders ?
(2) Clustered Decoders ?
Two latest microarchitectures that use clustered decoders are Zen5 (4+4) and Skymont (3+3+3).
On the other hand, most other microarchitectures use Unified decoders: Apple M4-P (10), Cortex X925 (10), Oryon-L (8), Lion Cove (8), etc...
Both Zen5 and Skymont are x86 cores. So it is not a coincidence that they use clustered decoders, because clustered decoders take up less transistors than a unified decoder of equivalent width. Considering how complex decoding on x86 can be, using clustered decoders is the economical path.
But some people are suggesting that we might eventually see clustered decoders on ARM cores too.
View attachment 112673
What do you mean by "100% correct stream of instructions"?Claim makes zero sense to me.
But I'm sick of arguing with people who are so stuck in the x86 way of doing things that they utterly refuse to understand alternatives.
Suppose (forget how, imagine an oracle) you have a 100% correct stream of enough instructions fetched from the I-cache to be fed to an ARM-style decoder.
Then WTF do you need clustered decode, even if you want 20-wide decode? Each decode is independent!
ALL the magic in a decent design is in achieving that 100% correct stream of instructions, especially if you want that stream to pull in more than a basic block per cycle.
This (of course..., sigh) is also completely misunderstood by people who live exclusively in x86 land, but it's a whole different issue from clustered decoding.
As both Zen5 and *mont are focused on core count this is simply a necessary path to increasing per socket density in order to compete with ARM solutions on this score.Both Zen5 and Skymont are x86 cores. So it is not a coincidence that they use clustered decoders, because clustered decoders take up less transistors than a unified decoder of equivalent width. Considering how complex decoding on x86 can be, using clustered decoders is the economical path.