But that changed with Bionics, right?you forgot to mention the funniest part, A10 had LITTLEs grow off the bigs like a cancerous tumor.
AMD's NPU has memory tiles, and the XDNA 1 implementation could do 30 GBps read and 30 GBps write per memory tile as per Ryzen AI column architecture and tiles — AMD Riallto 1.0 documentation. There is an interface tile that connects this memory tile to system memory.How much memory bandwidth (GB/s) is needed to feed Strix's 50 TOPS NPU?
Will Strix Point be able to feed it at that speed?AMD's NPU has memory tiles, and the XDNA 1 implementation could do 30 GBps read and 30 GBps write per memory tile as per Ryzen AI column architecture and tiles — AMD Riallto 1.0 documentation. There is an interface tile that connects this memory tile to system memory.
Going by the 2x8 layout from the recent versal XDNA 2 launch, and that there are 1 less interface tile - it should have a limit of 210 GBps. But the interface tiles seem to support compression and can't say how that changes things. Don't know how XDNA 2 changes things.
But yeah should ideally be fed by ~200GBps bandwidth.
You are assuming that we will get 8533 LPDDR5X on Strix Point APUs, not 6400 SO-DIMM DDR5.FYI, Strix Point will have 136 GB/s of main memory bandwidth.
And it has no SLC.
Uh, uh, uh.
So the GPU isn't going to be the only thing that's bandwidth starved huh?
We are not limited by compute capabilities anymore. We are limited by memory.I wonder how well strix halo will age, given that 2026 will prolly have an lpddr6 release of it, which would bump bandwith to like 300 to 400 ish area, considering amd products age fine, i bet we can squeeze a 25% improvement over base strix halo, by probably oc the lpddr5x modules to 9600 or hell to 10000, i heard the ram configuration was in 24 and 48gb for some asus laptops, which will probably be the first ones to release with strix halo, but i do hope we get a 32 or 64gb model, also undervolting this thing will be a massive task
AMD's NPU has memory tiles, and the XDNA 1 implementation could do 30 GBps read and 30 GBps write per memory tile as per Ryzen AI column architecture and tiles — AMD Riallto 1.0 documentation. There is an interface tile that connects this memory tile to system memory.
Going by the 2x8 layout from the recent versal XDNA 2 launch, and that there are 1 less interface tile - it should have a limit of 210 GBps. But the interface tiles seem to support compression and can't say how that changes things. Don't know how XDNA 2 changes things.
But yeah should ideally be fed by ~200GBps bandwidth.
If you look past the current sales of gaming PCs, including prebuild systems and DIY, AMD is still barely > 50%You'll have to wait for a year.
It seems like you are contradicting yourself. If the only way to run this code is Mac with 64 GB of memory, then how is dGPU advantage if dGPU can only have 16-24 GB?Do we have an idea of how RAM will work in Strix Halo? Fully unified, as in a Mac? Or with a part of it reserved for the GPU, as in integrated graphics solutions?
I'm interested in the part for IA RAM heavy workloads, but that would only be a differentiator under the Mac fully unified model. If a portion of RAM is to be reserved, then I could go for a laptop with an Nvidia GPU - since Strix Halo wouldn't have the advantage of a huge pool of RAM vs a GeForce card.
Edit: like the only way to run Llama 3 70B is with an M2/3 Max with at least 64 GB, and from what I see on Twitter even that's rather short on RAM.
Intel more competitive CPU arch won't intercept with Zen 5 time on market though? By the time they have a more competitive thing, AMD will also be releasing Zen 6.If you look past the current sales of gaming PCs, including prebuild systems and DIY, AMD is still barely > 50%
Then, there is the installed base. Prior to 5800x, AMD was that compelling for gaming. Installed base is 65-70% Intel (on Steam Surveys).
So there is a good sized market out there for AMD to sell to, if Intel is not competitive. And this market may be people who never in their lives bought a non-Intel CPU. It seems like an opportunity that does not come around that often.
Here is a dumb way to think: "AMD does not need to release the V-Cache processor until Intel has a more competitive CPU". Smart way to think is for AMD to maximize their sales, while their advantage is the greatest.
Because if they wait until Intel does release a more competitive CPU, it will only give the Inte-only buyers excuse to stay with Intel.
Think in terms of "surrender", when the fans of one brand flip to the opposite side. It is also a function of the gap in performance (not just the length of time with trivial lead). No need to "surrender" for 1% or 5% delta.Intel more competitive CPU arch won't intercept with Zen 5 time on market though? By the time they have a more competitive thing, AMD will also be releasing Zen 6.
Yeah, I suspect that Apple ditched (if you want to call it that) the concept because they realized their E cores were pretty buff on their own so it made more sense to expose them to the OS so that apps could leverage them in MT situations. With the A10 Fusion, it was either the E core or the P core, but never both.This patent reminds me a lot of A10 Fusion, where the little cores ended up being almost transparent to software because if a workload was heavy enough it would transition from the little cores to the big cores, and primarily use those instead (not sure how that was determined by the OS/hardware, but I wouldn't be surprised if it relied upon the types of instructions run - like in that patent - or the workload duration).
A very hardware solution to the whole big.LITTLE scheduling problem. But Apple still ended up dropping it pretty quickly afterwards. It's why it was such a surprise to see AMD patent almost the same idea back then too.
I suspect they will wait until supply backlog for Turin-X tapers off.Because if they wait until Intel does release a more competitive CPU, it will only give the Inte-only buyers excuse to stay with Intel.
Not in the foreseeable future. Advanced packaging will start to be mainstream once chipmakers are forced due to ever increasing complexity x performance. But early adoption is off the table as your costs increase too much compared to monolithic. See Meteor Lake.When will APUs get 3D V-cache?
My daughter's friend's uncle's newphew said Zen 5 > Arrowlake by a good chunk
Now that AMD has separated Zen 5 Client into Classic and Dense, they can customize Classic to use eLVT for higher clocks while Dense use uLVT for higher efficiency at lower clocks.
Zen6 takes this even further, if you can figure out the one area AMD is not entirely leading in.
Well, advanced packaging, which Zen 6 is supposed to bring, should help with power consumption in various load scenarios including idle state, at least. Consumption in Connected Standby alias Modern Standby may still be an issue though.In what way does Zen6 take it further? And are such details already known about Zen6?
no one cares about the dark Si blobs.So the GPU isn't going to be the only thing that's bandwidth starved huh?
really now what he implied.Well, advanced packaging, which Zen 6 is supposed to bring, should help with power consumption in various load scenarios including idle state, at least. Consumption in Connected Standby alias Modern Standby may still be an issue though.
never.When will APUs get 3D V-cache?
well yeah A10 was an utterly schizophrenic design.But that changed with Bionics, right?
I was referring to the theoretical peak of weights being streamed in and out of the NPU given the 7 interface-memory tile pairs. I am confused about what you are trying to say here.Cache bandwidth is mostly irrelevant for client inference. Caching helps when you can batch requests, but for a client inference setup you mostly just can't usefully do that. A single inference job needs to touch gigabytes of parameters, the actual usable bandwidth is just how much is left of system ram bandwidth after all other users.
now ./. not?really now what he implied.
That's an OEM issue (and AMD no longer makes tablet parts anyway).
sigh, AMD Hawk Point sucks for tablets, this tablet also has a fan and has heating problems despite having enough heatpipes. I heard this song and dance before but Zen 5 really needs excel in tablets/thin n lights.
That idle avg and battery life is pathetic.