Discussion Apple Silicon SoC thread

[Eug]

M1
5 nm
Unified memory architecture - LP-DDR4
16 billion transistors

8-core CPU

4 high-performance cores
192 KB instruction cache
128 KB data cache
Shared 12 MB L2 cache

4 high-efficiency cores
128 KB instruction cache
64 KB data cache
Shared 4 MB L2 cache
(Apple claims the 4 high-effiency cores alone perform like a dual-core Intel MacBook Air)

8-core iGPU (but there is a 7-core variant, likely with one inactive core)
128 execution units
Up to 24576 concurrent threads
2.6 Teraflops
82 Gigatexels/s
41 gigapixels/s

16-core neural engine
Secure Enclave
USB 4

Products:
$999 ($899 edu) 13" MacBook Air (fanless) - 18 hour video playback battery life
$699 Mac mini (with fan)
$1299 ($1199 edu) 13" MacBook Pro (with fan) - 20 hour video playback battery life

Memory options 8 GB and 16 GB. No 32 GB option (unless you go Intel).

It should be noted that the M1 chip in these three Macs is the same (aside from GPU core number). Basically, Apple is taking the same approach which these chips as they do the iPhones and iPads. Just one SKU (excluding the X variants), which is the same across all iDevices (aside from maybe slight clock speed differences occasionally).

EDIT:



M1 Pro 8-core CPU (6+2), 14-core GPU
M1 Pro 10-core CPU (8+2), 14-core GPU
M1 Pro 10-core CPU (8+2), 16-core GPU
M1 Max 10-core CPU (8+2), 24-core GPU
M1 Max 10-core CPU (8+2), 32-core GPU

M1 Pro and M1 Max discussion here:

Page 78 - Discussion - Apple Silicon SoC thread

Page 78 - Seeking answers? Join the AnandTech community: where nearly half-a-million members share solutions and discuss the latest tech.

forums.anandtech.com

M1 Ultra discussion here:

Page 109 - Discussion - Apple Silicon SoC thread

Page 109 - Seeking answers? Join the AnandTech community: where nearly half-a-million members share solutions and discuss the latest tech.

forums.anandtech.com

M2 discussion here:

Page 127 - Discussion - Apple Silicon SoC thread

Page 127 - Seeking answers? Join the AnandTech community: where nearly half-a-million members share solutions and discuss the latest tech.

forums.anandtech.com

Second Generation 5 nm
Unified memory architecture - LPDDR5, up to 24 GB and 100 GB/s
20 billion transistors

8-core CPU

4 high-performance cores
192 KB instruction cache
128 KB data cache
Shared 16 MB L2 cache

4 high-efficiency cores
128 KB instruction cache
64 KB data cache
Shared 4 MB L2 cache

10-core iGPU (but there is an 8-core variant)
3.6 Teraflops

16-core neural engine
Secure Enclave
USB 4

Hardware acceleration for 8K h.264, h.264, ProRes

M3 Family discussion here:

Page 215 - Discussion - Apple Silicon SoC thread

Page 215 - Seeking answers? Join the AnandTech community: where nearly half-a-million members share solutions and discuss the latest tech.

forums.anandtech.com

M4 Family discussion here:

Page 263 - Discussion - Apple Silicon SoC thread

Page 263 - Seeking answers? Join the AnandTech community: where nearly half-a-million members share solutions and discuss the latest tech.

forums.anandtech.com

 

[Doug S]

Apple for sure will need to address their small NAND though if that's their plan.

Even the biggest LLMs are less than 1 TB in size, and Apple already offers 1 TB phones with reports that 2 TB will be offered this fall. Obviously we're getting WAY WAY out there in speculation land (that Apple is going to do something with AI this fall, and it will involve their patents for running LLMs out of NAND) but if that was the case they could limit it to running only on models that have at least 1 TB of NAND. It would almost certainly be a Pro only feature at least at first, and it wouldn't be too crazy to make 1 TB the minimum config for iPhone 16 Pro/Max - tell people if you buy that you will have only xxx GB left for other apps so you may want to consider the 2 TB model.

 

[gdansk]

Apple's patents in that area did not involve using NAND flash as swap. That's so obvious that even in a world where far too many "obvious" patents are granted that it would never be granted.

Read the paper? It outlined a way of using flash as memory as seldom as possible with domain-specific compression and optimizations.

i.e. a patentable, non-obvious technique but at the end of the day all methods to reduce the performance and ssd shredding impact of using a disk as memory

 

[darkswordsman17]

How do you read data from disk without it being written? And these models aren't small even when compressed and used only as needed as described in the paper. It really seems like a poor fit unless you have a more reliable form of storage for consumer devices.

I don't even know how you're getting that thought from what I wrote? The issue is constant writes, which wouldn't be happening is what I'm saying. It'd obviously be written, but I don't think it'd be constantly being rewritten, it'd be more like a game loaded on a gaming system, where you'd write large chunk of files which the game would read/load, but you'd only be updating the save files and some other files here and there and not the whole of the install constantly, which is what would wear out NAND. But I might be mistaken on how much AI processing writes.

Apple's patents in that area did not involve using NAND flash as swap. That's so obvious that even in a world where far too many "obvious" patents are granted that it would never be granted.

It wouldn't need to specify NAND, would it? But the fact of the matter is, Apple's chips use embedded NAND as its main storage, so something that potentially takes up a sizeable chunk of it (and possibly presents extra wear/writes) presents an issue with how Apple has priced the NAND in their devices.

 

[gdansk]

But I might be mistaken on how much AI processing writes.

The technique is better than typical demand paging because it understands the problem space but it still adds writes to NAND flash compared to simply running a simpler model that does fit memory. Is a 5% better image fill really worth terabytes of cumulative writes to a consumer's phone? I seriously question that. Their description of the solution seemed focused on much larger devices than phones in any case (where 24GB of VRAM was considered a serious limitation).

Re: NAND alternatives. Instead of switching to something esoteric and expensive (plus who makes an alternative anymore?) they could simply double the LPDDR on the Pro for marketing prestige image fill features. Or do it on their servers where they can hot-swap storage devices as they fail.

 

[FlameTail]

Or do it on their servers where they can hot-swap storage devices as they fail.

HOT QUESTION: What silicon does Apple use in their datacenters? Do they use their own custom silicon (like the other hyperscalers) or do they buy from Intel/AMD/Nvidia?

 

[moinmoin]

HOT QUESTION: What silicon does Apple use in their datacenters? Do they use their own custom silicon (like the other hyperscalers) or do they buy from Intel/AMD/Nvidia?

They outsource to AWS, Google Cloud etc.

 

[jpiniero]

They outsource to AWS, Google Cloud etc.

They have their own datacenter too for some stuff, IIRC. I believe they use HP.

 

[moinmoin]

They have their own datacenter too for some stuff, IIRC. I believe they use HP.

Yeah, they do. But the majority is outsourced to the "cloud".

I think everybody expected Apple to scale up its own DCs and eventually move to its own server tech over time as well. But instead of that happening first server hardware and software were shuttered and then the well known lead Arm designer exodus occurred.

I'd say it's clear Apple doesn't see its business in running DCs, otherwise it could have built a rival to Amazon's AWS over the decades. Very ironic considering how big a part of its business subscription services make up today.

 

[jpiniero]

I'd say it's clear Apple doesn't see its business in running DCs, otherwise it could have built a rival to Amazon's AWS over the decades. Very ironic considering how big a part of its business subscription services make up today.

The "Services" subscription revenue isn't that much. Most of that department is the Google Search money.

 

[FlameTail]

I think everybody expected Apple to scale up its own DCs and eventually move to its own server tech over time as well. But instead of that happening first server hardware and software were shuttered and then the well known lead Arm designer exodus occurred.

And the thing is, those engineers actually wanted to make server silicon (atleast that was their goal when they left Apple).

Ironic.

 

[moinmoin]

The "Services" subscription revenue isn't that much. Most of that department is the Google Search money.

What? It's Apple's biggest business right after iPhone.

And the thing is, those engineers actually wanted to make server silicon (atleast that was their goal when they left Apple).

Ironic.

Exactly.

 

[jpiniero]

What? It's Apple's biggest business right after iPhone.

Because the Google Search money is that much. Seriously, if (when) the DOJ bans it, Apple's stock is going to tumble.

 

[moinmoin]

Because the Google Search money is that much. Seriously, if (when) the DOJ bans it, Apple's stock is going to tumble.

Do you seriously think "Google search money" is more than the whole App Store money, the iCloud money, the Apple TV money, the Apple Music money, the Apple Arcade money and whatever further service of the day money they have cooking?

 

[Tigerick]

The "Services" subscription revenue isn't that much. Most of that department is the Google Search money.

Apple Services FY2023 was $87.6 billions, slightly less than Mac, iPad and Wearables combined....clearly you have no idea how big Apple Services Department is.

PS: Google paid Apple $26.3 billions in 2021, go figures.

 

[Doug S]

Yeah, they do. But the majority is outsourced to the "cloud".

I think everybody expected Apple to scale up its own DCs and eventually move to its own server tech over time as well. But instead of that happening first server hardware and software were shuttered and then the well known lead Arm designer exodus occurred.

I'd say it's clear Apple doesn't see its business in running DCs, otherwise it could have built a rival to Amazon's AWS over the decades. Very ironic considering how big a part of its business subscription services make up today.

Well Apple has tried to put as much on device as possible, at least for anything with a privacy impact, so they went in a different direction where the cloud is concerned. Sure you still want a backup of your device, and unlike 5 or 10 years ago now you can have it backed it using key you control so it is 100% secure from Apple (or whoever they outsource those iCloud backups to) being able to snoop in your data.

But stuff like "AI" they don't want that in the cloud, which may limit its capabilities but once your queries are in the cloud you no longer control that data (i.e. if you suddenly start asking it a lot of questions about cancer, that's potentially highly profitable information for a company that derives a lot of revenue from advertising like Google, Facebook or to a lesser extent Microsoft)

I was among those who thought it made sense for Apple to use its own stuff to insource its cloud needs and be able to eventually use that cloud as an extension of your iPhone's computational abilities for certain applications, but that's not how they see their future. Apple's needs are for "commodity cloud" like storing those backups and providing additional capacity for the apple.com website after they announce the new iPhone, etc. They can get it cheaper buying it from one of the big providers than by becoming one of the big providers themselves.

 

[Mopetar]

You don't even need NAND for a trained model since you can just put it on some kind of ROM chip. Granted that means you can't update it, but the whole model doesn't need to change and patches could be kept on the regular HDD.

For a phone, it's probably good enough as a solution since the lifespan isn't more than 10 years in the most extreme cases.

Are there any ROM technologies that have high capacity, high bandwidth, and low power consumption? If not, Apple is certainly the kind of company that could invest in something like that.

 

[mikegg]

Because the Google Search money is that much. Seriously, if (when) the DOJ bans it, Apple's stock is going to tumble.

Which would be a huge overreach of power.

Quite honestly, I use ChatGPT more often than Google now. Google has competition. Extremely strong competition. Maybe even existential competition.

 

[Doug S]

You don't even need NAND for a trained model since you can just put it on some kind of ROM chip. Granted that means you can't update it, but the whole model doesn't need to change and patches could be kept on the regular HDD.

For a phone, it's probably good enough as a solution since the lifespan isn't more than 10 years in the most extreme cases.

Are there any ROM technologies that have high capacity, high bandwidth, and low power consumption? If not, Apple is certainly the kind of company that could invest in something like that.

Why would you put static data on a ROM when you can put it on NAND? The updates would come as part of iOS upgrades, sure they're a lot bigger but bandwidth is much less of a problem for most than it was in the early days of iOS.

I don't think there is such a thing as ROMs with hundreds of GB of capacity - at least not that cost less per bit than NAND. OK "in theory" ROM should be cheaper because it does less since it doesn't require the ability to erase/rewrite (or makes it difficult like EEPROM) but economies of scale are a thing. The NAND market is much much bigger than the ROM/EEPROM market.

NAND that isn't being erased/written has the same power consumption as ROM, and best of all it would help the longevity of your NAND overall. Flash blocks age out when they've been erased/rewritten too many times, the FTL optimizes for that by spreading that out equally across all blocks. Even if you used half your flash as a "ROM" writing something once and never changing it, those blocks will get migrated around over time by the FTL so you'd effectively get double the lifetime from only actively using half your flash capacity.

I don't believe it is correct that a trained model can be stored in ROM, at least not one that you actually use. A query requires a lot of updates, though the extent to which you can use indirection to store the updates in RAM and leave the "ROM" unchanged may allow that sort of thing to work so long as you will accept some tradeoff in how much the model learns from your queries.

 

[FlameTail]

NAND is a type of ROM, no?

 

[DrMrLordX]

NAND is a type of ROM, no?

Read Only Memory? No. You could potentially use it in the place of a ROM but the fact that you can repeatedly write to NAND means it doesn't really fit the definition.

 

[GC2:CS]

Hmmm... not seeing any UltraFusion D2D on this shot of the M3 Max. Unless that image is cropped, it looks like there won't be an M3 Ultra based on that die. I don't recall ever seeing a Palma 2C codename anywhere either.

Source: @techanalye1 on X:

https://twitter.com/x/status/1740142759942750246

Also surprised nobody seems to be talking about the R1 / APL1W08, which certainly looks like a 2.5/3D tiled package design.

View attachment 93749

Source: iFixit: https://www.ifixit.com/Guide/Apple+Vision+Pro+Chip+ID/169813

What about HBM memory being in there ? Top, bottom center die. Looks quite symmetrical nice and got too many tiles for a first gen tile design though.

What if M3 Ultra is a new design like M3 Pro or M3 Max is a chop of M3 Ultra like M2 Pro is a chop of M2 Max.

 

[Doug S]

What about HBM memory being in there ? Top, bottom center die. Looks quite symmetrical nice and got too many tiles for a first gen tile design though.

What if M3 Ultra is a new design like M3 Pro or M3 Max is a chop of M3 Ultra like M2 Pro is a chop of M2 Max.

I have never bought that Apple Silicon was planned be on an 18 month cadence, it just worked out that way for the first few. Releasing M3/M3P/M3M at the same time shortly after their little brother A17P tells me they've got their schedule ironed out and it'll be yearly from now on.

They can skip Ultra in the M3 generation because M2 Ultra will be barely a year old when the M4 family arrives this fall and it won't have to suffer from N3B's yield issues. If there are any exceptions to the yearly cadence it would probably be Ultra, and if so they'd only include the I/O pads in every other Max.

 

[mikegg]

I have never bought that Apple Silicon was planned be on an 18 month cadence, it just worked out that way for the first few. Releasing M3/M3P/M3M at the same time shortly after their little brother A17P tells me they've got their schedule ironed out and it'll be yearly from now on.

The vast majority of members here were sure that Apple Silicon was on an 18 month cadence.

Take for example:

What's wrong with ~18 months? That was roughly the time frame for the iPad Pros, which used pre-M type chips.

The average between releases is over 500 days, and the very shortest is over 400 days.

2015-11 - A9X
2017-06 - A10X
2018-10 - A12X
2020-03 - A12Z
2021-04 - M1
2022-10 - M2

I think many members here assumed that the big AX iPad chips had 18 months+ update cadence so Apple Silicon must also be the same. That line of reasoning ignores the fact that Apple Silicon is more important than ever and goes into everything from multiple iPad models to Macs of all sizes to Vision Pro and beyond. Not updating Apple Silicon yearly means not having a new chip yearly for the vast majority of Apple's devices.

 

[SteinFG]

I think many members here assumed that the big AX iPad chips had 18 months+ update cadence so Apple Silicon must also be the same. That line of reasoning ignores the fact that Apple Silicon is more important than ever and goes into everything from multiple iPad models to Macs of all sizes to Vision Pro and beyond. Not updating Apple Silicon yearly means not having a new chip yearly for the vast majority of Apple's devices.

M1 macbook - Nov, 2020
M2 macbook - Jul, 2022 (20 months)
M3 macbook - Nov, 2023 (16 months)
It's not a lot of samlple points, but the average is 18 😛
If the M4 macbook comes out before March 2025, we can revise that theory, but so far it's right.

 

[FlameTail]

Look at the huge rise in power consumption with A17P.

N3B yields might be a reason, but it's not the only one.

The strategy of relying on clock speed increases to increase performance is not a sustainable one.