Info 64MB V-Cache on 5XXX Zen3 Average +15% in Games

[Kedas]

Well we know now how they will bridge the long wait to Zen4 on AM5 Q4 2022.
Production start for V-cache is end this year so too early for Zen4 so this is certainly coming to AM4.
+15% Lisa said is "like an entire architectural generation"

 

[Hitman928]

Microsoft had MilanX servers on Azure for public access which were months ago. Doubt they are creating those out of ES/QS chips lol. Only reason the 5800X3D is coming later is cause there is just so much demand for MilanX from just MS and likely initial production wasn't high. General avaliablity of MilanX is coming about the same time as 5800X3D which points to a ramp in production but the initial MilanX shipments weren't engineering samples even if the volume was still low.

When Microsoft announced MilanX on Azure, it was not a public access announcement. It was by special access only and it is still not available for public use as far as I know. I don't know why you don't think they couldn't be using QS in an essentially "beta release" of a new cloud offering. Once the design is finalized and validated, all of the following ES/QS are the same as the final retail product, the only difference is more logistical at that point. A single wafer is enough for over 80 fully enabled Epyc dies, then you need a second wafer for the V-cache. How many wafers do you think are needed to support a single preview instance for Microsoft?

Common terminology is that everything before release for volume manufacturing is called an engineering run, so saying they were producing low volume but it wasn't engineering runs is contradictory.

Like @desrever said, they were shipping Milan-X to hyperscalars in volume last year. TSMC made that comment public, but obviously they were letting AMD use the tech a little early . . .

Can you define volume? Can you name the hyperscalers they were shipping to, besides Microsoft? Can you name 1 cloud instance I, as an average user, can fire up today that uses MilanX? Are you saying TSMC was able to deliver volume manufacturing ahead of schedule but never announced it?

 

[DrMrLordX]

Can you define volume?

Nope. Can you? Since you brought it up . . .

Can you name the hyperscalers they were shipping to, besides Microsoft?

No. You either take the industry reports at face value or you don't, and scrape up other info as best you can. MS apparently snagged most of the earliest run of Milan-X. But there have been EPYC Milan-X leaks here and there, including of ES and commercial silicon. There was that one site (linked in this thread!) of a Chinese etailer taking pre-orders on Milan-X last December. And of course numerous industry insiders have been reporting that hyperscalars have been getting early shipments of Milan-X since last . . . October I think?

You should know how ODM sales work by now. We're not supposed to know these things, nor are the competitors of the hyperscalars placing these clandestine orders. They still happen, and have been for years.

 

[Hitman928]

Nope. Can you? Since you brought it up . . .

On the fab side, risk production numbers go up to around 30K wafers per month or about 1K wafers per day. High volume production is considered to be achieved when they hit between 40K to 50K wafers per month. Now, I imagine they don't have the same standard for something like die stacking as it will only be a small fraction of their total dies for a while, but even if they consider only 5% of that to be HVM for their stacked die process, that's still 50 wafers per day which could make, at minimum, 4000 Milan-X per day. Do you really think Microsoft is taking 4000 Milan-X CPUs per day?

No. You either take the industry reports at face value or you don't, and scrape up other info as best you can. MS apparently snagged most of the earliest run of Milan-X. But there have been EPYC Milan-X leaks here and there, including of ES and commercial silicon.

What is the difference to you between the commercial silicon and ES leaks? In other words, what differentiated the actual silicon? Was it just some random posters claiming to have one or the other?

There was that one site (linked in this thread!) of a Chinese etailer taking pre-orders on Milan-X last December.

This doesn't amount to anything without some kind of validation. Retailers from many different countries have preemptively taken preorders for lots of different items they didn't have an actual schedule for or some times even a distributor lined up for.

And of course numerous industry insiders have been reporting that hyperscalars have been getting early shipments of Milan-X since last . . . October I think?

Which fits in nicely to the exact schedule I said was expected all along. Hyperscalers will get early samples to stand up their own systems and put them through their validation requirements. If AMD/TSMC was in HVM, they wouldn't be called early shipments, they'd just be called shipments.

You should know how ODM sales work by now. We're not supposed to know these things, nor are the competitors of the hyperscalars placing these clandestine orders. They still happen, and have been for years.

So what is the argument? That AMD/TSMC already hit HVM in October, but the entire inventory was sent to hyperscalers and after a few months, they were done buying inventory so then it could be released to retail? Yet there's no evidence of these large orders in either AMD's financial statements or availability of instances.

Is it really that hard to believe that TSMC was completely honest in that volume production of stacked dies wasn't ready until 4Q2021 and that AMD used some pre-volume production to supply key partners for validation and bring up of their systems and that AMD is releasing to retail the silicon they get once the process was ready for volume production? Which just happens to be the same schedule they've followed for every Epyc release so far?

Edit:

BTW, this is what Ian said in his May 31, 2021 (in a June 1 update) article on the AMD V-cache unveiling.

Those processors will start production at the end of the year. No comment on availability, although Q1 2022 would fit into AMD's regular cadence.

 

[DrMrLordX]

On the fab side, risk production numbers go up to around 30K wafers per month or about 1K wafers per day.

Okay, but that's from the fab's perspective. TSMC may have had other customers in on risk production, and there is ramping involved. AMD probably isn't taking 30kwpm of N7 just for Milan-X and Vermeer-X, and likely won't at any point.

What is the difference to you between the commercial silicon and ES leaks? In other words, what differentiated the actual silicon? Was it just some random posters claiming to have one or the other?

Firmware mostly, plus again, there was a pre-order for Milan-X back in December which is pretty extraordinary when you consider that commercial availability of server-class hardware usually lags ~6 months behind ODM shipping "in volume" (from the perspective of the design firm, not the fab; take a look at what Intel considered to be "volume" shipments of IceLake-SP, for example).

This doesn't amount to anything without some kind of validation.

We're not going to get validation. ODMs keep their mouths shut. AMD and their hyperscalar clients don't want anyone to know what's going on. Typically ODMs ship to the hyperscalars 6 months before commercial availability, so take the commercial launch date for Milan-X and backtrack to last September (though most of the leakers seem to be saying, it was really October). That's when ODMs were probably sending out the first batches of A0 silicon (or whatever stepping actually shipped, it might have been designated B2 because reasons). Also remember that Milan was allegedly shipping to hyperscalars around . . . 9 months before it finally became available last year? AMD really sat on Milan for awhile. It was an outlier.

Retailers from many different countries have preemptively taken preorders for lots of different items they didn't have an actual schedule for or some times even a distributor lined up for.

Yeah but they typically don't do that until they have a SKU. This pre-order had the model numbers and everything.

Hyperscalers will get early samples to stand up their own systems and put them through their validation requirements.

They get more than just that.

So what is the argument? That AMD/TSMC already hit HVM in October, but the entire inventory was sent to hyperscalers and after a few months, they were done buying inventory so then it could be released to retail? Yet there's no evidence of these large orders in either AMD's financial statements or availability of instances.

Yes, and again . . . they're not going to tell us these things.

Is it really that hard to believe that TSMC was completely honest in that volume production of stacked dies wasn't ready until 4Q2021

Yes. I mean if TSMC wants to call the initial run something other than volume production then so be it. But we can't honestly claim that stacked N7 was impossible when multiple insiders have indicated that thousands of Milan-X CPUs have been going to hyperscalars since last October or so. If AMD had really wanted to prioritize Vermeer-X, it didn't have to launch this late, and they didn't have to launch one SKU.

 

[Hitman928]

Okay, but that's from the fab's perspective. TSMC may have had other customers in on risk production, and there is ramping involved. AMD probably isn't taking 30kwpm of N7 just for Milan-X and Vermeer-X, and likely won't at any point.

But AMD is by far TSMC's largest, if not only, stacked die customer at this point. That doesn't mean they have to take the full production, but it does mean that they would have had more than enough to supply the cloud and bring some to retail if it really was available in high volume numbers.

Firmware mostly, plus again, there was a pre-order for Milan-X back in December which is pretty extraordinary when you consider that commercial availability of server-class hardware usually lags ~6 months behind ODM shipping "in volume" (from the perspective of the design firm, not the fab; take a look at what Intel considered to be "volume" shipments of IceLake-SP, for example).

What evidence is there of these claims? What is this pre-order back in December? When was expected delivery? Who was the retailer? What was the firmware difference? I'm not talking internet rumors because you are arguing against public statements made directly by the manufacturers, so you need more than internet rumors.

We're not going to get validation. ODMs keep their mouths shut. AMD and their hyperscalar clients don't want anyone to know what's going on. Typically ODMs ship to the hyperscalars 6 months before commercial availability, so take the commercial launch date for Milan-X and backtrack to last September (though most of the leakers seem to be saying, it was really October). That's when ODMs were probably sending out the first batches of A0 silicon (or whatever stepping actually shipped, it might have been designated B2 because reasons). Also remember that Milan was allegedly shipping to hyperscalars around . . . 9 months before it finally became available last year? AMD really sat on Milan for awhile. It was an outlier.

So we're again trusting internet rumors over public statements of the manufacturers with no real evidence to base it on. BTW, these early batches of Milan were also pre-volume production.

Yeah but they typically don't do that until they have a SKU. This pre-order had the model numbers and everything.

Still doesn't mean much unless they could give you an actual delivery date with any kind of assurance.

They get more than just that.

So what is your view of how much and when they receive these shipments?

Yes, and again . . . they're not going to tell us these things.

So where is AMD hiding this revenue and where are all the products ending up?

Yes. I mean if TSMC wants to call the initial run something other than volume production then so be it. But we can't honestly claim that stacked N7 was impossible when multiple insiders have indicated that thousands of Milan-X CPUs have been going to hyperscalars since last October or so. If AMD had really wanted to prioritize Vermeer-X, it didn't have to launch this late, and they didn't have to launch one SKU.

Yes, that's exactly what is meant. You seem to think that pre-volume production means a few dozen samples are sent out which is not true at all. I haven't seen any reports of thousands of Milan-X CPUs floating around before the end of last year, but even if that were true, if Milan-X were in HVM, those shipments would only take up roughly 10%-15% of a single day's wafer order from AMD (which is 0.5% - 0.75% of a monthly wafer order).

If anyone bothered to read Ian's take on the situation after the announcement and his own call with AMD, he said that AMD would go into production (which means volume production) at the end of the year which would fit into Q12022 retail availability according to their usual cadence. So, everything from TSMC's statements, to AMD's statements, to consistency with their typical schedule, points to it always being a Q1 2022 release, but we're supposed to ignore all that because some people on the internet insisted it was actually in volume production earlier and AMD just chose to not only not report this in their financial results, but delay both the Ryzen3D release and the general Milan-X release because they had secret deals with the ODMs that somehow both made them no significant revenue and produced no available instances. This goes way beyond Occam's razor.

 

[DrMrLordX]

But AMD is by far TSMC's largest, if not only, stacked die customer at this point. That doesn't mean they have to take the full production, but it does mean that they would have had more than enough to supply the cloud and bring some to retail if it really was available in high volume numbers.

They still aren't going to sell 30 kwpm worth of Milan-X. If I recall correctly,their entire allocation of N5 for Genoa et al is 20 kwpm?

What is this pre-order back in December?

It's in this thread. If you've been reading it from page 1, you'd have seen it. We discussed it a bit. Do I have to go dig it up?

So we're again trusting internet rumors over public statements

Again, TSMC can label one event as "volume production" while still providing AMD enough die stacking to feed early ODM shipments. Or early runs of Vermeer-X. AMD simply elected not to bring Vermeer-X to the market any earlier. Otherwise, all the public statements being made by insiders/leakers are entirely consistent with AMD's behavior in the past wrt early shipments of CPUs (not ES, not QS) to hyperscalars. They get the new stuff before everyone else, and they get thousands of units before AMD even launches the product publicly.

So where is AMD hiding this revenue and where are all the products ending up?

Same place they always do: Enterprise&Embedded. There is no product-by-product breakdown. You just get a number. Intel does the same thing with DCG revenues.

I haven't seen any reports of thousands of Milan-X CPUs floating around before the end of last year

I have. Again, a lot of the stuff was posted in this thread.

but we're supposed to ignore all that

Nobody's ignoring that. AMD is ramping up production for general availability. They've fulfilled their initial ODM shipments and now they're reading to sell to the public.

This goes way beyond Occam's razor.

Bollocks.

 

[Hitman928]

They still aren't going to sell 30 kwpm worth of Milan-X. If I recall correctly,their entire allocation of N5 for Genoa et al is 20 kwpm?

No, but the argument was that the Milan-X was already produced in high volume for pretty much the entire fourth quarter and that the cloud providers bought everything AMD could produce, which would mean tens of thousands of CPUs.

It's in this thread. If you've been reading it from page 1, you'd have seen it. We discussed it a bit. Do I have to go dig it up?

You're the one pointing to it as evidence, so it's up to you. I did a forum search and couldn't find it.

Again, TSMC can label one event as "volume production" while still providing AMD enough die stacking to feed early ODM shipments. Or early runs of Vermeer-X. AMD simply elected not to bring Vermeer-X to the market any earlier. Otherwise, all the public statements being made by insiders/leakers are entirely consistent with AMD's behavior in the past wrt early shipments of CPUs (not ES, not QS) to hyperscalars. They get the new stuff before everyone else, and they get thousands of units before AMD even launches the product publicly.

Again, volume production is what drives general availability because, in what I thought would be self evident, you need volume production to support volume sales. Never have I once said that AMD wasn't shipping products to hyperscalers last year. In fact, I said that they were and that this was expected. Shipping product from engineering runs, or limited runs (whatever you want to call it) to key partners to stand up their systems/flows and build the ecosystem before you shift to volume manufacturing is the exact thing that is expected. Once the partners have some to do this, you start volume shipping for general availability. AMD wasn't going to release products to retail before having volume production to support general availability, so they used some limited runs to support some key partners until they could ramp to volume production at the end of last year which, after validation, assembly, packing, etc. leads to a release around the end of 1Q22. What evidence has been provided to the contrary? Sure, AMD could have theoretically chosen to release some limited runs to retail, but that's not how AMD, or Intel, or Apple, etc. have ever operated except out of complete desperation (see Cannon Lake). To suggest this was a real possibility is ridiculous.

Same place they always do: Enterprise&Embedded. There is no product-by-product breakdown. You just get a number. Intel does the same thing with DCG revenues.

Again, the argument is that AMD has Milan-X in high volume production since October and that AMD couldn't bring it to retail because the cloud providers were buying everything they could make of Milan-X. That 100% would make an impact in their Enterprise numbers and AMD would have made note of it in their statements.

I have. Again, a lot of the stuff was posted in this thread.

I don't follow threads in excruciating detail. If something outside of rumors about this have been posted, I haven't seen them.

Nobody's ignoring that. AMD is ramping up production for general availability. They've fulfilled their initial ODM shipments and now they're reading to sell to the public.

Why would they need to ramp if they were already shipping at volume by last October, enough to support a retail release? Do you not see the contradiction?

Bollocks.

Believe what you want. I'm out of this discussion. If you find some actual evidence to support these claims, let me know.

 

[Timorous]

Microsoft released a video a while ago where they said they took pretty much as many Milan-X parts as AMD could manufacture so they could validate before AWS / GCP and have a rather large competitive advantage for clients that can take advantage of the performance gains Milan-X provides.

In that video they stated they had several platforms up and running already and they had had them a while.

Pretty aure volume production would have started around December time as AMD indicated it would and part of the reason for 'delayed' Vermeer-X is that with Milan-X demand they can only use dies that don't bin well enough to work in a Milan-X sku and it takes a while to build the amount of stock they want.

 

[Kedas]

Keep in mind that volume production of V-cache dies is a subset of the current volume production line, it's not a parallel extra production line it's extra work on a limited number of the volume production dies.
This means that AMD decides how many will be made or not.
And first batches were at least made in early November.
So 'volume' for V-cache doesn't mean the same as when they will say volume production for zen4.

Also April 20 is Q2 not Q1

 

[moinmoin]

But AMD is by far TSMC's largest, if not only, stacked die customer at this point.

That's the public impression, since AMD advertised the tech. We have zero actual insight about the amount of customers TSMC has for the tech and know nothing about their volume, be it at the ramp up or at high volume production. Everything is pure guesswork so far.

 

[Hitman928]

Keep in mind that volume production of V-cache dies is a subset of the current volume production line, it's not a parallel extra production line it's extra work on a limited number of the volume production dies.
This means that AMD decides how many will be made or not.
And first batches were at least made in early November.
So 'volume' for V-cache doesn't mean the same as when they will say volume production for zen4.

Also April 20 is Q2 not Q1

First V-cache dies were made well before November. Otherwise Microsoft couldn't have had a preview instance running in November. But again, we're talking about when the die stacking tech went into volume production. Yes, V-cache SKUs of course will be a subset of AMD's wafers, but until the die stacking was available for volume production, AMD couldn't decide how many they could produce as the upper limit, that was reliant on TSMC ramping up their capability to produce stacked dies at volume, which TSMC themselves said came online in 4Q21 and most likely mid to late 4Q as well.

For Q1/Q2 I kept saying end of Q1 was the earliest possible, but got tired of repeating myself so just started saying Q1. End of Q1 or early Q2 was always going to be the earliest we saw Ryzen3D.

 

[Hitman928]

That's the public impression, since AMD advertised the tech. We have zero actual insight about the amount of customers TSMC has for the tech and know nothing about their volume, be it at the ramp up or at high volume production. Everything is pure guesswork so far.

If any other company that could come close to AMD's volume was using it, we'd know. There hasn't been a single other product announced by any of the major, or even mid-major, companies that use this tech. I'm sure multiple companies have stuff in the pipeline, or at least are 'toying around' with it, but nothing substantial at this time.

 

[igor_kavinski]

What are the chances of this tech appearing in a GPU?

 

[Hitman928]

part of the reason for 'delayed' Vermeer-X is that with Milan-X demand they can only use dies that don't bin well enough to work in a Milan-X sku and it takes a while to build the amount of stock they want.

I'm not really sure this is the case. You only need 1 fully enabled CCD for a 5800x3d versus 8 CCDs for MilanX, but all but 1 of the SKUs use partially enabled CCDs. Also, as others mentioned, these SKUs aren't supplied from separate wafer lines than Ryzen/Milan, so to suggest that Ryzen is purely Milan cast-offs doesn't really work since Zen3 Ryzen has much higher volume than Milan. The process is way too mature for there to be more Milan rejects than acceptable CCDs. Even if we assume that this is true, we still go back to the same wafer lines being used, and there is plentiful supply of normal Ryzens in the market, so even if it is true, needing Milan cast-offs doesn't seem to be an issue.

 

[Hitman928]

What are the chances of this tech appearing in a GPU?

Pretty high for some form of stacking, though probably a different configuration.

 

[moinmoin]

If any other company that could come close to AMD's volume was using it, we'd know.

Wouldn't be so sure about that. There are plenty people you'd think should know better for whom e.g. MediaTek is/was a completely blind spot for whatever reason. Also it's not really about a single company coming close to AMD's volume but the overall volume, regardless of the amount of customers.

 

[jpiniero]

AMD EPYC 7773X "Milan-X" Benchmarks Show Very Strong HPC Performance Upgrade Review - Phoronix

www.phoronix.com

Phoronix has some benchmarks of Milan-X. Compared to the regular Milan most are modest gains but there's also ones where it's a lot faster... and ones where it's slower. Also draws more power.

 

[Hitman928]

Wouldn't be so sure about that. There are plenty people you'd think should know better for whom e.g. MediaTek is/was a completely blind spot for whatever reason. Also it's not really about a single company coming close to AMD's volume but the overall volume, regardless of the amount of customers.

You are right that I/we don't know for sure that there aren't other companies using a lot of stacking volume, but given the evidence we do have and how long AMD has been working with TSMC on this tech, I feel pretty confident in my statement. I am perfectly willing to be wrong on this if there is any evidence at all of any other company, or companies, producing products with this tech in volume.

 

[Saylick]

ServeTheHome has a good write up as well. Some of the plots are rather interesting, like the effective IPC and L3 hit rate:

AMD Milan-X Delivers AMD EPYC Caches to the GB-era

AMD Milan-X ushers in the GB-era of server chip caches with up to 0.75GB of L3 cache per chip or 1.5GB in a system. We get to test hands-on.

www.servethehome.com

 

[igor_kavinski]

That internal IPC visualization tool being used by AMD is da bomb!

 

[gdansk]

That MariaDB performance seems promising. Would like to see more database benchmarks

 

[deasd]

Phoronix test Milan-X and intuitive summary:

https://twitter.com/x/status/1505903193460547592

 

[DisEnchantment]

Phoronix has some benchmarks of Milan-X. Compared to the regular Milan most are modest gains but there's also ones where it's a lot faster... and ones where it's slower. Also draws more power.

That was the overall test, there are some loss of perf due to 11% lower clocks of the X SKUs. Had there been no clock regression it would have been really impressive.So your typical gRPC/RESTful server should stick with the regular SKUs.

But for the targeted loads, the gains are significant to say the least.

AMD Milan-X Upgrade In The Cloud Makes Microsoft Azure HBv3 Very Compelling For HPC Review - Phoronix

www.phoronix.com

 

[xilli_fiberbit]

What is explain the weird price between the 7473X (with 24c/48t) and the 7373X (with 16c/32t) ?

 

[Saylick]

What is explain the weird price between the 7473X (with 24c/48t) and the 7373X (with 16c/32t) ?

Some customers require high ST performance and the 16c 7373X has the FULL 96MB of L3 just like all of the other SKUs, i.e. these are all 8 CCD SKUs. AMD doesn't save on silicon for any of these parts. To make the 16c SKU, AMD has to find eight CCDs with the best clocks and essentially disable 6 out of the 8 cores on each of the eight CCDs. Also, some software is licensed on a per-core basis.