Question By 2030, we will be buying massive NPUs with a CPU and GPU attached to it.

mikegg · Mar 12, 2023

Today, NPUs such as Apple's neural engine takes less space than the CPU or GPU in a SoC. By 2030, I predict that we won't be buying "CPUs". We will all be buying NPUs with a CPU and a GPU attached to it.

NPUs will become the new CPUs.

More applications will start to make massive use of AI inference. Soon, consumers will demand that their laptops and mobile phones infer models as big as GPT or LLaMA or Stable Difusion or future large models. It has been theorized that the current iPhone 14 Pro could infer Meta's LLaMA model, though slowly and with much less accuracy.

In order to do this, chip makers will focus on making NPUs and making them huge.

We are in the beginning of a complete paradigm shift in chip requirements.

Apologies if this is the wrong place to post this. There is no NPU forum on Anandtech.

cytg111 · Apr 13, 2024

What inference task is latency limited to the point that it must occur on die rather than decentralized(like now)?

mikegg · Apr 28, 2024

I’m hearing there is a strong possibility that the chip in the new iPad Pro will be the M4, not the M3. Better yet, I believe Apple will position the tablet as its first truly AI-powered device — and that it will tout each new product from then on as an AI device. This, of course, is all in response to the AI craze that has swept the tech industry over the last couple years.

Apple Rivals Retool to Challenge the iPhone and Vision Pro

Apple’s top rivals are revamping their strategies for smartphones and headsets in a sign they smell an opportunity. Also: The new iPad Pro will usher in Apple’s shift into AI hardware; new details emerge on iOS 18 and the company’s AI plan; and the Vision Pro’s lead marketer retires.

www.bloomberg.com

M4 coming out early with a fat NPU?

Doug S · Apr 28, 2024

mikegg said:
Apple Rivals Retool to Challenge the iPhone and Vision Pro

Apple’s top rivals are revamping their strategies for smartphones and headsets in a sign they smell an opportunity. Also: The new iPad Pro will usher in Apple’s shift into AI hardware; new details emerge on iOS 18 and the company’s AI plan; and the Vision Pro’s lead marketer retires.

www.bloomberg.com

M4 coming out early with a fat NPU?

I don't buy it. I suppose theoretically there would be enough time for TSMC to have run them enough N3E wafers for a lower volume product like the iPad Pro, and if they had the core design for the M4/A18 done a few months earlier than usual it would be possible.

But "theoretically possible" doesn't make it true.

mikegg · Apr 28, 2024

Doug S said:
I don't buy it. I suppose theoretically there would be enough time for TSMC to have run them enough N3E wafers for a lower volume product like the iPad Pro, and if they had the core design for the M4/A18 done a few months earlier than usual it would be possible.

But "theoretically possible" doesn't make it true.

I think it's more likely than not.

First, it's very unlike Apple to schedule an whole event with media invites just for an iPad using OLED. It'd be a very "meh" event.

But if M4 is getting introduced, then they have to schedule a new event.

The iPad Pro is exactly the kind of device that a very early M chip could debut with. First, it's high volume enough that Apple doesn't want to keep using the lower yield N3B node. But it's not so high volume that it would overwhelm a new node (like the iPhone would).

I'm guessing Apple knew years in advance that M3 would be a short lived generation due N3B cost issues. Therefore, they could have easily dedicated a larger team to make sure the M4 gets its design finished 5-6 months earlier.

Also, the A18 Pro is likely already in full production using N3E because Apple always starts new iPhone chip production in Spring. That's why I don't think it's farfetched for M4 to debut next month.

Lastly, there are signs everywhere that Apple wants to move away from N3B as soon as possible. The Mac Mini doesn't have an M3 version and is rumored to skip it. The M3 Max doesn't have UltraFusion silicon, indicating that Apple might skip the M3 Ultra. M3 only went into its most important Macs - Air & Pros. Yes, the iMac got the M3 but I think it's a very low volume product that did not receive the M2 upgrade so it was due.

Doug S · Apr 29, 2024

mikegg said:
I think it's more likely than not.

First, it's very unlike Apple to schedule an whole event with media invites just for an iPad using OLED. It'd be a very "meh" event.

But if M4 is getting introduced, then they have to schedule a new event.

The iPad Pro is exactly the kind of device that a very early M chip could debut with. First, it's high volume enough that Apple doesn't want to keep using the lower yield N3B node. But it's not so high volume that it would overwhelm a new node (like the iPhone would).

I'm guessing Apple knew years in advance that M3 would be a short lived generation due N3B cost issues. Therefore, they could have easily dedicated a larger team to make sure the M4 gets its design finished 5-6 months earlier.

Also, the A18 Pro is likely already in full production using N3E because Apple always starts new iPhone chip production in Spring. That's why I don't think it's farfetched for M4 to debut next month.

Lastly, there are signs everywhere that Apple wants to move away from N3B as soon as possible. The Mac Mini doesn't have an M3 version and is rumored to skip it. The M3 Max doesn't have UltraFusion silicon, indicating that Apple might skip the M3 Ultra. M3 only went into its most important Macs - Air & Pros. Yes, the iMac got the M3 but I think it's a very low volume product that did not receive the M2 upgrade so it was due.

Well one thing that would be in favor of this is if Apple plans to show off their new cores in Apple Silicon first, and iPhone gets them later in the year - the opposite of how things have been done up until this point.

Because TSMC's node timing seems to be totally off from what they have been since Apple became their biggest customer. They always started mass production Q2 so they could ship tens of millions of SoCs in time for a new iPhone cycle starting at the end of Q3. One could excuse the timing issues with N3 due to the problems TSMC encountered, but it sounds like that wrong timing (mass production starting in Q4) is continuing on with N2 and even to the new 'A16'. It makes no sense they wouldn't have taken steps to line up with the needs of their biggest and most important customer one way or another. Unless it turns out that starting mass production in Q4 is how their biggest customer would prefer things going forward.

mikegg · Apr 29, 2024

Doug S said:
Well one thing that would be in favor of this is if Apple plans to show off their new cores in Apple Silicon first, and iPhone gets them later in the year - the opposite of how things have been done up until this point.

I don't think this would continue next year. I think this year is special due to N3B yields and Apple needing to be very aggressive in catching up in GenAI.

I would expect M5 to launch in Fall 2025 again.

Doug S · Apr 29, 2024

mikegg said:
I don't think this would continue next year. I think this year is special due to N3B yields and Apple needing to be very aggressive in catching up in GenAI.

I would expect M5 to launch in Fall 2025 again.

Its irrelevant to Apple's fortunes whether they release something with "AI" in May or October, there is no need to "catch up". Especially when AMD and Intel have nothing at all on that front yet. Who is their supposed competition, Nvidia? They play in datacenter AI which is a whole different realm and one in which Apple will never compete.

If M4 appears early it won't be because of AI. Maybe it will be because of N3B yields (or more specifically, perhaps because Apple's "known good die" pricing for N3B has an expiration date, which has been rumored) or maybe it will be because Apple wants to switch up the schedule.

I still maintain there's no way that TSMC is going to a process release schedule that's six months opposite of what Apple wants. You might get into that situation because you run into issues, but you'd make efforts to correct that schedule as soon as possible to match the wants and desires of a company that's fully 25% of your revenue (and probably a greater share of your profit)

Hulk · Apr 29, 2024

Good. I am tired of thinking. Thinking is for puny Banner.

mikegg · Apr 29, 2024

Doug S said:
Especially when AMD and Intel have nothing at all on that front yet. Who is their supposed competition, Nvidia? They play in datacenter AI which is a whole different realm and one in which Apple will never compete.

Microsoft, Google, Meta, OpenAI.

It’s not AMD and Intel. It isn’t even about the hardware. It’s what the hardware will allow Apple to do with GenAI.

Right now, investors value stocks that have a stake in the GenAI race and Apple doesn’t have any. Hence, their stock has lagged when compared to Nvidia, Microsoft, Google recently.

Doug S · Apr 29, 2024

mikegg said:
Microsoft, Google, Meta, OpenAI.

It’s not AMD and Intel. It isn’t even about the hardware. It’s what the hardware will allow Apple to do with GenAI.

Right now, investors value stocks that have a stake in the GenAI race and Apple doesn’t have any. Hence, their stock has lagged when compared to Nvidia, Microsoft, Google recently.

That's all cloud based, and all depending on Nvidia hardware. Apple's on device AI will be a totally separate category. If they were gaining a year I might agree with you, but four months is nothing. They wouldn't make major roadmap changes to gain fourth months in a market that doesn't yet exist.

moinmoin · Apr 29, 2024

Doug S said:
Who is their supposed competition, Nvidia?

The biggest impact of the AI hype train is the stock market valuation of stock companies hyping it. And AAPL's valuation is not doing so well compared to other tech stocks so far.

Hulk · Apr 29, 2024

It is my belief that AI will have the largest impact behind the scenes for most people and not from direct queries. What I mean by that is that software will work better and do want we intend without so many keystrokes. Topaz Photo AI is one example. I don't care if it's AI or not, it makes the photos look better. Or to use another example I've posted before when your computer goes BSOD it reboots and tells what happened and how it fixed itself or you can avoid the issue moving forward.

Is Google search AI? I don't know but it seems pretty darn intelligent.

Doug S · Apr 30, 2024

moinmoin said:
The biggest impact of the AI hype train is the stock market valuation of stock companies hyping it. And AAPL's valuation is not doing so well compared to other tech stocks so far.

Sure, but Apple's management has been pretty good about not having a short term outlook focused on quarterly stock prices. They aren't going to go to a bunch of effort to push out M4 early because they want to goose the stock price in May rather than October.

Question By 2030, we will be buying massive NPUs with a CPU and GPU attached to it.

mikegg

Golden Member

cytg111

Lifer

mikegg

Golden Member

Apple Rivals Retool to Challenge the iPhone and Vision Pro

Doug S

Platinum Member

Apple Rivals Retool to Challenge the iPhone and Vision Pro

mikegg

Golden Member

Doug S

Platinum Member

mikegg

Golden Member

Doug S

Platinum Member

Hulk

Diamond Member

mikegg

Golden Member

Doug S

Platinum Member

moinmoin

Diamond Member

Hulk

Diamond Member

Doug S

Platinum Member

TRENDING THREADS