News Ampere Altra Launched with 80 Arm Cores for the Cloud（Performance Estimates）

[csbin]

Ampere Altra Launched with 80 Arm Cores for the Cloud

The Ampere Altra 80-core Arm server CPU is upon us with 128 PCIe Gen4 lanes per CPU, CCIX, and dual-socket server capabilities. We assess the impact.

www.servethehome.com

 

[naukkis]

The actual EPYC CPU turned out to have eight 75mm^2 chiplets and a ~450mm^2 14nm IO die. A hypothetical monolithic die to fit those together might straight up exceed what can be fabbed on 7nm due to the size alone.

AMD did tell us at Rome and chiplet introduction that max monolithic version they could have produced was 48c version, and that would have been extremely low yielding product.

 

[DrMrLordX]

I think it's funny that nVidia hasn't been mentioned once in this thread.

Still, kudos to someone ACTUALLY releasing hardware that you can buy that is a competitive ARM server solution. In fact it makes Intel look pretty awful, not that that's hard. As an added bonus, a 1P server featuring the Q32-17 might almost be affordable as a home tinkering machine. I would rather have one of those at home than mess around with Graviton2 instances. Call me old-fashioned.

@RasCas99

I'm a bit skeptical about what will be the future of ARM servers. If the sale goes through to nVidia, that becomes a potential threat to a lot of the in-house development efforts unless companies like Google; Microsoft; and Amazon (like Apple) snag an architecture license and stay on ARMv8.x in perpetuity. I strongly suspect that nVidia intends to hijack the entire Neoverse platform and use it to their advantage.

And that makes me worry a bit for Ampere. They're using reference N1 cores and they've licensed the bus from ARM as well. Any changes to future licensing schemes will affect their product portfolio and their bottom line.

Also, anyone notice that Ampere is using a CCIX link to connect the two CPU sockets?

edit: and now that I think about it, I would really like to be able to buy a Q32-17 machine. I'm sick of waiting for Rockchip to release the RK3588. Not anywhere near in the same ballpark, but hey if you want something you can actually buy and own at home that isn't an A72 or similar . . . what else are you going to do? I mean, besides use a phone?

 

[JoeRambo]

AMD did tell us at Rome and chiplet introduction that max monolithic version they could have produced was 48c version, and that would have been extremely low yielding product.

The Ampere Altra Review: 2x 80 Cores Arm Server Performance Monster

www.anandtech.com

This page exposes the price AMD has to pay for chiplets. Altra has same characterics of 8C DDR4 3200 controller and is beating AMD with Intel like memory performance. 97ns vs 133ns is a nasty deficit. And this is from what is a startup versus company that is in MC business for years.
The inter core communication reveals even more deficit for AMD, latency is double and even triple in same socket once beyond CCX. ZEN3 will double CCX domain size, but at the cost of L3 cumulative bandwidth available to the chip.

The reality still is: if price and performance are similar, people will continue to choose monolithic chips, once ARM per core performance improves some ( N2 promises 40% ) and if they deliver 128C 3+Ghz chip -> it will the beginning of the end for X86 in servers.

AMD and Intel really need to jump ARM bandwagon with their IP in IMC, interconnects and RAS.

 

[lobz]

The more impressive thing is that Ampere can put 80 (maybe 128?) N1 cores into one single die w/ I/O. AMD on the same process had to resort to a ton of chiplets for 64 cores, and Intel's Ice Lake-SP supposedly only fits 42 cores into a single die.

That is just silly. They didn't 'have to' do it like that. Not sure how much less profit Ampere could hope for if they offered CPUs ranging from 6 to 64 cores and had to design, tape out, validate and market a lot of different monolithic chips. Yeah everyone can say about Intel what they want, but it's obvious that their margins are shrinking. AMD made a business choice based on the markets they are present in.

 

[DisEnchantment]

I am waiting for some vendor to present us eval kits but not sure if this is available outside of the Hyperscalars.
As of now I have access only to our Azure premium SKUs so not going to see it either.
All our on prem clusters are x86 based, best part... nobody ever touch the code, live migration after live migration
If at some point we decide to migrate something, our guys will have some real work!!
When that happens I am expecting super fat discounts from our vendors.

 

[amrnuke]

I'm a bit skeptical about what will be the future of ARM servers. If the sale goes through to nVidia, that becomes a potential threat to a lot of the in-house development efforts unless companies like Google; Microsoft; and Amazon (like Apple) snag an architecture license and stay on ARMv8.x in perpetuity. I strongly suspect that nVidia intends to hijack the entire Neoverse platform and use it to their advantage.

If nvidia are able to purchase Arm, I can't imagine nvidia cutting off or manipulating any of the major vendors like Apple, Amazon, MS etc. - 1) because they all have (or in the case of Amazon/Annapurna, likely have) architectural licenses, and 2) because nvidia is a fleck of corn in Apple, MS, and Amazon's stool when it comes to monetary weight. If nvidia try to manipulate those licenses or create problems, it will probably only serve to have Apple, MS, and Amazon take nvidia through an extensive and expensive legal spanking.

Anyway, my expectation is that the future of Arm servers might actually be more robust. nvidia are already in the data center marketplace and I have to imagine that their acquisition of Arm will only accelerate their work in the data center. That Arm chips are now offering competitive performance in the server space goes along with that, and as long as nvidia are driving data center development of whole systems (nvidia Arm based CPU to pair with their GPU), the server space will naturally follow, at least as best I can foresee, which may not be very well admittedly.

 

[DrMrLordX]

If nvidia are able to purchase Arm, I can't imagine nvidia cutting off or manipulating any of the major vendors like Apple, Amazon, MS etc. - 1) because they all have (or in the case of Amazon/Annapurna, likely have) architectural licenses, and 2) because nvidia is a fleck of corn in Apple, MS, and Amazon's stool when it comes to monetary weight. If nvidia try to manipulate those licenses or create problems, it will probably only serve to have Apple, MS, and Amazon take nvidia through an extensive and expensive legal spanking.

Not sure who has perpetual architectural licenses, but I would be surprised if anyone had such licenses for ARMv9. It's not that they'll try to manipulate existing licenses . . .

Anyway, my expectation is that the future of Arm servers might actually be more robust.

Maybe now that Altera is showing hardware, yes. We'll see if they can continue to launch product, or if they'll flame out like Marvell.

 

[NTMBK]

Not sure who has perpetual architectural licenses, but I would be surprised if anyone had such licenses for ARMv9. It's not that they'll try to manipulate existing licenses . . .

If they try to play that game, I would not be surprised if the ecosystem stayed with ARMv8. Remember when Intel tried to move everyone over to their locked down Itanium ISA, that only Intel could produce chips for? Just because the platform holder tries to make something happen, doesn't mean that everyone else will go along with it.

 

[lobz]

If they try to play that game, I would not be surprised if the ecosystem stayed with ARMv8. Remember when Intel tried to move everyone over to their locked down Itanium ISA, that only Intel could produce chips for? Just because the platform holder tries to make something happen, doesn't mean that everyone else will go along with it.

But boy did they try to make it hapoen painfully hard...

 

[moinmoin]

I honestly cannot see any of the existing and potential ARM chip manufacturers be willing to argue for Nvidia's takeover of ARM. Nvidia will have a lot of work to convince regulators everywhere that this won't negatively affect competition in the ARM ecosystem. The timing of these new announcements of ARM solutions (now by Ampere and Microsoft, very likely more to come) tells me they are done now so that regulators are actually aware of and recognize these efforts in their decision for or against Nvidia's takeover of ARM.

 

[Mopetar]

I don't think Nvidia would be wise to try any shenanigans since RISC-V will be an option for companies and even though it may be a bit of an "also ran" for a while, pushing companies to adopt it will just cut down on the amount of time it takes for it to become competitive.

 

[Doug S]

if i've learnt anything over 15 years of designing/selling Datacentre infrastructure is the market in general always cares way less then you think they do, you have to have massive undeniable advantage over multiple generations to gain inertia.

Yes, and its been that way forever. Back in the RISC workstation days when you'd see PA-RISC or Alpha or POWER gain a temporary advantage, it never moved the market much - the rule of thumb was you needed a 2x advantage in performance or half price for same performance to get people to consider going through the hassle of migration. The market shares moved around a bit but didn't make any huge moves until Intel was able to deliver that 'half price for same performance' when their fab advantage became too much for the RISC workstation model to survive.

Some people want to point to these Ampere results and say "well, ARM has caught up, and costs less, x86 is doomed". Things don't happen that quickly, the people making the multi million dollar purchase decisions are cautious by nature, and aren't going to make such a choice without a lot of study and preparation and seeing others with a higher risk tolerance take the plunge first.

 

[Schmide]

So I'm re reviewing the results. In compiling there seems to be divergent results.

At anand

at phoronix (not an image graph so no linky)

AMD EPYC 7742 2P                200.99
Intel Xeon Platinum 8280 2P     229.32
Ampere Altra Q80 - 33 2P        282.64

Yes they are compiling different versions 10 vs 11, different disks and ramdisks were used, and some NUMAs may vary, but compiling should be relatively linear per platform. Seems to be quite a contradiction.

Edit: Distros are diff too

 

[HurleyBird]

AMD did tell us at Rome and chiplet introduction that max monolithic version they could have produced was 48c version, and that would have been extremely low yielding product.

That's a naïve extrapolation. Altra has 32MB L3 while theoretical monolithic 48c Rome would run at a whopping 192 MB L3. An actual high core count monolithic Zen 2 server processor, in order to make any sense at all, would need to look more like a Xeon in terms of layout and cache hierarchy.

 

[JoeRambo]

Some people want to point to these Ampere results and say "well, ARM has caught up, and costs less, x86 is doomed". Things don't happen that quickly, the people making the multi million dollar purchase decisions are cautious by nature, and aren't going to make such a choice without a lot of study and preparation and seeing others with a higher risk tolerance take the plunge first.

In x86 tug war of Intel vs AMD there were plenty of moments of "Intel bankrupt in 2007" and "AMD is done for". Plenty. Once 64C AMD server chips were leaked by that awesome french site that is sadly no longer with us -> people were shouting Intel is done for etc. Fast forward several years and that is not the case.

But in ARMs case -> the custom cloud, Apple ecosystem, every damn Android phone, Fugaku all point tentacles at the writing on the wall that the end of x86 having technological ( read perf, mindshare) superiority is upon us.

Even when looking at nearest future in the context of this thread - next year Altra will throw 128C 3Ghz chip. When this thread was posted, i was not taking it seriuos cause it was projections, but they have delivered and despite doom and gloom about 32MB of L3 it performs rather well as 1S machine.
In 2021 we will see AMD ZEN3 server chip with probably same 64C ~same clocks. Tough place to compete with 128C ARM chip that is no longer retarded in single threaded perf. And AMD is not exactly mindshare/ecosystems champion either.

Then there is elefant in the room -> the "other" cloud guys, the also rans. They can't afford to put custom ARM chips themselves, those cloud guys who can - show them middle fingers. Yet as this AT review clearly outlines - there's now 250W 80C chip on the shelves ~Graviton2 competence, without strings attached. Amazon and now Apple already started the ball running, but it will take open market chips to really move things.

 

[HurleyBird]

Even when looking at nearest future in the context of this thread - next year Altra will throw 128C 3Ghz chip. When this thread was posted, i was not taking it seriuos cause it was projections, but they have delivered and despite doom and gloom about 32MB of L3 it performs rather well as 1S machine.
In 2021 we will see AMD ZEN3 server chip with probably same 64C ~same clocks. Tough place to compete with 128C ARM chip that is no longer retarded in single threaded perf. And AMD is not exactly mindshare/ecosystems champion either.

The problem that ARM vendors are facing is basically the same as what AMD is facing: massive conservatism from customers. Only it's worse because of the ISA boundary. By all rights AMD should have more server marketshare than Intel, but they don't, and it's not even close. If AMD can stay ahead of Intel for long enough, it will eventually happen, but it's a slow trog. And if Intel retakes the lead at any point, it will be a big setback.

Ironically, AMD's server chips decimating Intel's might be the best thing for Intel right now. It helps enable customers to stay conservative in terms ISA by comparing ARM solutions with AMD's.

So, maybe 128C Altra can beat Milan. More likely it will be really close, winning some and losing others. Can it beat server Zen 4? Almost certainly not. Sure, Zen 4 will be out later and it's not a fair comparison, but it is the right comparison because the market isn't even close to fair. In order to seriously threaten the x86 server market it isn't enough that ARM solutions sometimes beat x86 solutions. They need to win considerably and consistently. If there's a seriously impressive ARM chip but x86 catches up a year or two later, that still plays in favour of x86.

 

[Schmide]

But in ARMs case -> the custom cloud, Apple ecosystem, every damn Android phone, Fugaku all point tentacles at the writing on the wall that the end of x86 having technological ( read perf, mindshare) superiority is upon us.

Its predecessor the K computer did nearly the same thing with SPARC inside and now this is the end of x86 suddenly.

Even when looking at nearest future in the context of this thread - next year Altra will throw 128C 3Ghz chip. When this thread was posted, i was not taking it seriuos cause it was projections, but they have delivered and despite doom and gloom about 32MB of L3 it performs rather well as 1S machine.
In 2021 we will see AMD ZEN3 server chip with probably same 64C ~same clocks. Tough place to compete with 128C ARM chip that is no longer retarded in single threaded perf. And AMD is not exactly mindshare/ecosystems champion either.

It took forever to get this 80c Altra out and suddenly they're going to easily add 60% more cores on a single piece of silicon? Seems very aggressive.

We basically have a few points on the graph now, interpolating deterministic outcomes from that is a fool's errand.

 

[DrMrLordX]

If they try to play that game, I would not be surprised if the ecosystem stayed with ARMv8.

If SVE2 is effectively locked-up behind ARMv9 and its successors, then ARMv8 license holders will be stuck on NEON forever. Unless they can somehow incorporate a different SIMD ISA extension without violating their license. Which Apple sort-of did with AMX.

I honestly cannot see any of the existing and potential ARM chip manufacturers be willing to argue for Nvidia's takeover of ARM.

I'm not sure how it would help Altera. The ARM server market is not a zero-sum game, but rest assured that nVidia hopes to capture as much of that market's growth potential for itself. And it's going to try to force as many ARM server vendors as possible to shape their platforms around what will likely be a vertically-integrated CPU/GPU solution featuring proprietary nVidia hardware/standards wherever possible. Altera is already using CCIX for inter-socket communications. NV can change that in a New York minute.

 

[JoeRambo]

So, maybe 128C Altra can beat Milan. More likely it will be really close, winning some and losing others. Can it beat server Zen 4? Almost certainly not.

Anandtech results are not the only ones on the web. At Phoronix Altra has some incredible results in variuos disciplines. All while having much better power consumption?
It's not like 128C is end of road for ARM or Altra. Even if ARMs promise of 40% ST advancement is bogus and real advancement is half of that - ARM is in the land of "good enough" right now, can only get better with maturing ecosystem.

I think what they need the most is not more or faster cores, but rather high end workstations with MacOS or Windows, so people start optimizing important software for those chips.

 

[Doug S]

If SVE2 is effectively locked-up behind ARMv9 and its successors, then ARMv8 license holders will be stuck on NEON forever. Unless they can somehow incorporate a different SIMD ISA extension without violating their license. Which Apple sort-of did with AMX.

SVE2 is already an optional extension for AArch64, it is too late to "lock it up". There is even support for it in ARM's compiler. While GCC currently supports only SVE I have to imagine they are working on SVE2 support. Any ARMv8 architectural licensee is free to implement it like any other optional extension, though AFAIK no one has publicly stated that they have or will (though Nuvia would be a strong bet given their target market)

We still don't know what AMX is, and whether it is an ISA extension running on Apple's ARM CPU cores. It is unclear whether such a thing is permissible for an ARM architectural license, though possibly Apple's architectural license is different from the others given their history with ARM. AMX might be something that runs outside their ARM cores in another block on the SoC. Perhaps it is identical to SVE2 - nothing says Apple's marketing has to refer to it "SVE2". Maybe it is "mostly identical" to SVE2 but with some stuff added and/or deleted, so they call it something different because ARM won't let you call it "SVE2" unless it is exactly SVE2.

So I think the jury is still out on whether Apple went their own way with AMX or not.

 

[DrMrLordX]

SVE2 is already an optional extension for AArch64, it is too late to "lock it up"

Interesting. Pity nobody's bothered incorporating it into available designs yet. So those with an ARMv8 perpetual license will have that option in the future? You sure it's not linked intrinsically to a different license?

We still don't know what AMX is, and whether it is an ISA extension running on Apple's ARM CPU cores.

The allegations I have read claim that it is an ISA extension running on the core itself (rather than as a part of the fixed-function hardware units in the SoC).

 

[beginner99]

I'm a bit skeptical about what will be the future of ARM servers. If the sale goes through to nVidia, that becomes a potential threat to a lot of the in-house development efforts unless companies like Google; Microsoft; and Amazon (like Apple) snag an architecture license and stay on ARMv8.x in perpetuity. I strongly suspect that nVidia intends to hijack the entire Neoverse platform and use it to their advantage.

Maybe wouldn't be a surpirse given NVs past and obsession with closed, proprietary stuff. heck one can even imagine they will only sell you full servers if you want a precious card for AI. It's NV, wouldn't surprise me either.

But that will only go so far. Delayed or no licensing of ARMv9 will simply just kill the ARM server ecosystem. Any trust in a fair treatment and fair chance to compete will be gone. And with NV being then the only vendor of ARMv9 who will invest in software in it? Let's be honest. That's the biggest problem. Yes it wouldn't surprise me if NV is this short-sighted but it will fail. Even more so the fully closed server approach. That would be a big win for AMD as essential big corps will probably then help AMD with their rocm with developer time.

edit: and now that I think about it, I would really like to be able to buy a Q32-17 machine. I'm sick of waiting for Rockchip to release the RK3588. Not anywhere near in the same ballpark, but hey if you want something you can actually buy and own at home that isn't an A72 or similar . . . what else are you going to do? I mean, besides use a phone?

new M1 mac?

 

[mikegg]

But that will only go so far. Delayed or no licensing of ARMv9 will simply just kill the ARM server ecosystem. Any trust in a fair treatment and fair chance to compete will be gone. And with NV being then the only vendor of ARMv9 who will invest in software in it? Let's be honest. That's the biggest problem. Yes it wouldn't surprise me if NV is this short-sighted but it will fail. Even more so the fully closed server approach. That would be a big win for AMD as essential big corps will probably then help AMD with their rocm with developer time.

The truth is, every company strives for proprietary tech since open tech gives you no business moat. You don't see Windows OS, Microsoft Word, Apple SoCs, Adobe Photoshop, etc. being open-sourced. Heck, AMD doesn't open source most of their own inventions. The only things that AMD makes open are things that it knows Nvidia is way better at so making them open is just a way to gain more adoption.

One exception I can think of is Android and Chrome being open-source. But Google makes money through the Android App Store, emphasizes its own services on Android, and makes Google the default search engine on Chrome.

Nvidia has been hugely innovative with things such as CUDA, RTX, DLSS, Tensor Cores, etc. There's no reason to expect Nvidia to open-source their inventions so that other companies can copy them. Nvidia also did the heavy lifting in getting Ray Tracing software support onto DirectX and Vulkan, which AMD can now benefit from.

This "closed ecosystem" label thing for Nvidia only exists in the DIY PC Master race or elite gamer community. These communities want inventions to be made standard so that many companies can produce parts and compete on price. In other words, they want Nvidia tech but pay AMD prices. In B2B, most businesses don't care. If closed, proprietory Nvidia tech can solve their business problems, then they will buy Nvidia regardless.

Lastly, Nvidia is now the biggest semiconductor company in the world by marketcap if you don't count TSMC. They didn't get to this size by being stupid. They got here because they provided a lot of value to their customers.

 

[lobz]

The truth is, every company strives for proprietary tech since open tech gives you no business moat. You don't see Windows OS, Microsoft Word, Apple SoCs, Adobe Photoshop, etc. being open-sourced. Heck, AMD doesn't open source most of their own inventions.

One exception I can think of is Android and Chrome being open-source. But Google makes money through the Android App Store, emphasizes its own services on Android, and makes Google the default search engine on Chrome.

Nvidia has been hugely innovative with things such as CUDA, RTX, DLSS, Tensor Cores, etc. There's no reason to expect Nvidia to open-source their inventions so that other companies can copy them. Nvidia also did the heavy lifting in getting Ray Tracing software support onto DirectX and Vulkan, which AMD can now benefit from.

This "closed ecosystem" label thing for Nvidia only exists in the DIY PC Master race or elite gamer community. These communities want inventions to be made standard so that many companies can produce parts and compete on price. In B2B, most businesses don't care. If closed, proprietory Nvidia tech can solve their business problems, then they will buy Nvidia regardless.

Lastly, Nvidia is now the biggest semiconductor company in the world by marketcap if you don't count TSMC. They didn't get to this size by being stupid. They got here because they provided a lot of value to their customers.

Customers who bought Turing cards under the false hope they could game with RT on in less than 2 years and more than 5 worthwile games did the heavy lifting regarding real-time RT thus far.

 

[DrMrLordX]

new M1 mac?

I've considered the Mac Mini, but I'm not overly-fond of the operating system. Having something like the Q32-17 that can native-boot Linux would be a plus.