Help me understand PCIE lanes.

[pantsaregood]

Let's look at Skylake-DT and Z170, for example. The CPU has 16 dedicated PCIE lanes integrated into it and the Z170 chipset has an additional 24. That's a total of 40 lanes.

So, what's stopping x8/x8/x8/x8 PCIE? why can't the lanes on the chipset be utilized?

Now, even though I don't properly understand why, I also know there are weird chipsets that "add" PCIE lanes and enable weird things lime four-way SLI on mainstream CPUs. How does this work, exactly?

 

[ShintaiDK]

DMI 3.0 = PCIe 3.0 x4.

 

[john3850]

DMI 3.0 = PCIe 3.0 x4.

Would there be any overhead or reduced speeds from the z170 at x4.

 

[pantsaregood]

If DMI 3.0 has the same bandwidth at PCIE 3.0 x4, then how can DMI 3.0 supply 20 PCIE 3.0 lanes? Does it do so via some kind of switching?

 

[Shivansps]

Its not 20 either, its shared with other I/O devices, if all I/O is populated only 9 remain.

And all of that is limited to the DMI 3.0 speed, you cant never get full x4 anyway.

The max you can do with a Z170 is 8x/4x/4x/4x pci-e plus a few x1 that will be sharing the last x4.

 

[Tuna-Fish]

If DMI 3.0 has the same bandwidth at PCIE 3.0 x4, then how can DMI 3.0 supply 20 PCIE 3.0 lanes?

The short version is that it can't, really.

Does it do so via some kind of switching?

PCIE is a packet-based switched protocol. The 20 PCIE lanes connect at full speed to the chipset. Should all the devices attached to them try to simultaneously communicate with the CPU (or the RAM, as the memory controller is on-die with the CPU), this will happen at 4xPCIE 3.0 speeds.

Since these days there is little that PCIE devices want to do that doesn't require them to communicate with the CPU, "up to 20 PCIE lanes" is kind of shitty marketing.

If you want lots of PCIE lanes, head for the higher-end platforms.

 

[Shivansps]

mmmm hard disk access is not direct?

 

[ShintaiDK]

mmmm hard disk access is not direct?

It goes via the chipset and DMI.

 

[zir_blazer]

Unless you use a PCIe SATA Controller or PCIe SSD in a PCIe Slot connected to the Processor.

For as long that you use a single Video Card, you can do 8x/4x/4x. You're not really going to lose Video Card performance, and you can put a PCIe SSD directly connected to the Processor. Its a better idea than using a Intel 750 on the Chipset and saturating the DMI, maybe even better performing since Latency should be lower with one less hop.

I'm curious about how much DMA has advanced, since at this point anything that is connected to the Chipset and needs to talk to another PCIe device, may want to completely avoid going to the Processor for the DMI bottleneck reason.

 

[h4rm0ny]

Unless you use a PCIe SATA Controller or PCIe SSD in a PCIe Slot connected to the Processor.

For as long that you use a single Video Card, you can do 8x/4x/4x. You're not really going to lose Video Card performance, and you can put a PCIe SSD directly connected to the Processor. Its a better idea than using a Intel 750 on the Chipset and saturating the DMI, maybe even better performing since Latency should be lower with one less hop.

I'm curious about how much DMA has advanced, since at this point anything that is connected to the Chipset and needs to talk to another PCIe device, may want to completely avoid going to the Processor for the DMI bottleneck reason.

Do graphics cards even need x16 anymore? Genuine question but it seems to me that with PCIe 3.0, capacity has exceeded need. Though if it's not really getting the full x16 PCIe 3.0...? Any reason we wouldn't see graphics cards with a x8 length connector at some point? Is it just for when it's plugged into an older PCIe 2.0 slot, or am I completely off-base here.

 

[ShintaiDK]

Do graphics cards even need x16 anymore? Genuine question but it seems to me that with PCIe 3.0, capacity has exceeded need. Though if it's not really getting the full x16 PCIe 3.0...? Any reason we wouldn't see graphics cards with a x8 length connector at some point? Is it just for when it's plugged into an older PCIe 2.0 slot, or am I completely off-base here.

There is a test here, with a GTX980.
https://www.techpowerup.com/reviews/NVIDIA/GTX_980_PCI-Express_Scaling/

 

[h4rm0ny]

There is a test here, with a GTX980.
https://www.techpowerup.com/reviews/NVIDIA/GTX_980_PCI-Express_Scaling/

Thank you for that. It was an interesting read. It also confirms what I suspected (nice to be right for once). x16 at PCI-e 3.0 is significant overkill for current graphics cards. x8 at PCI-e 2.0 even seems to be largely sufficient though I suspect we're on the cusp of that no longer being true.

 

[CluelessOne]

Does this means that on any motherboard all the sata port, usb and ethernet goes through the chipset (aka DMI interface)? Also on a motherboard do all PCIe slot goes direct to CPU (not going via chipset)?

 

[ShintaiDK]

Does this means that on any motherboard all the sata port, usb and ethernet goes through the chipset (aka DMI interface)? Also on a motherboard do all PCIe slot goes direct to CPU (not going via chipset)?

No.

It depends what chipset/socket/motherboard.

But overall, on LGA115x/FMx. Only the graphics is directly connected to the CPU, everything else via the chipset.

 

[freeskier93]

Thank you for that. It was an interesting read. It also confirms what I suspected (nice to be right for once). x16 at PCI-e 3.0 is significant overkill for current graphics cards. x8 at PCI-e 2.0 even seems to be largely sufficient though I suspect we're on the cusp of that no longer being true.

What I find really surprising from that article is just how performance bottlenecked GPUs are at 4K. The difference between PCIe 1.1 x4 and PCIe 3.0 x16 at 4K is only 2 or 3 fps in almost all the games tested.