By Debendra Das Sharma, PCI-SIG Board Member

The upcoming PCI Express® (PCIe®) 6.0 specification will continue PCI-SIG’s® longstanding history of innovation for the next generation of products to keep up with evolving needs in a wide range of markets. One of the significant innovations in PCIe 6.0 specification, in addition to several needed to double the data rate to 64.0 GT/s with PAM-4 signaling, is a new Low Power State (L0p) to support scalable power consumption with bandwidth usage without interrupting traffic. The new state L0p is symmetric and maintains at least one active Lane that ensures uninterrupted traffic flow, even during width transitions. The Link always trains in the highest possible width and subsequently can modulate its width depending on the bandwidth need in Flit mode. This post includes answers to questions about L0p that were asked during the PCIe 6.0 Specification webinar.

1. During the presentation, it was mentioned that L0p provides proportionate power consumption to bandwidth usage. Can you explain why the PCIe 6.0 specification will not be supporting x3, x5, etc. widths and why it will only be supporting symmetric widths?

PCIe only supports x1, x2, x4, x8, and x16 Links. That is a strong enough reason to not include the support for other widths that are currently not supported since it means significant investment to enable L0p. The reason for this specific support is due to tradeoffs of power savings vs. complexity. PCI-SIG focuses on supported PCIe architecture widths in order to avoid creating rules around new widths and all the associated design and validation complexities. For example, we must deal with transactions starting and ending in Lanes where previous architectures did not. This is feasible, but at the expense of complexity and support for different widths can be added later if the market requires it. Currently, PCI-SIG’s goal is to leverage the existing infrastructure and gain power savings so that users are able to create the best designs while saving power. A simpler approach allows developers to focus on the essential aspects and also reduces interoperability risks in an open ecosystem.

Simplicity was also the reason for choosing to leverage symmetric Link width in this release of the specification. Without it, users would have to deploy a Link layer mechanism to convey training messages from the lower PHY layer. For example, consider trying to go from four Lanes in either direction to four Lanes upstream and eight Lanes downstream. The four downstream Lanes that need to train do not have counterparts in the training sets informing them of the progress. As a result, you would need to use the upper Link layer to track and send progress. While such a flow is possible, it will be new. Due to the way the L0p protocol is built, asymmetry can be easily incorporated using this approach in a future version of the specification. Between the unsupported widths and asymmetric L0p, the latter is easier to perform and will most likely will be adopted going forward.  

2. Is the idea of L0p intended to be used for previous revisions of PCIe specifications e.g. the PCIe 5.0 specification?

L0p is part of the PCIe 6.0 specification and is only enabled in Flit Mode. One can design a component with a maximum data rate of 32.0 GT/s (or lower), support Flit Mode and L0p and still conform to PCIe 6.0 specification without any support for 64.0 GT/s data rate.

3. Why not support L0p for 8b/10b?

At the time of the webinar, PCI-SIG was considering including 8b/10b support for L0p and have since included it.

4. Will L0p be supported in a Link with Retimers? If so, how will the Retimer handle the link width change on both pseudo-links?

L0p will be supported in a Link with Retimers. Retimers supporting the Flit Mode are mandated to support L0p. The L0p width is negotiated between the Ports – both pseudo-ports in the same direction from a Retimer will be identical in width. The transitions between active and idle states of each Lane are completed with Ordered Set(s), which the Retimer can view and then act accordingly. For example, when the Link width is reduced from x8 to x4, a special type of Ordered Set (EIOS) will be sent on the four Lanes going electrically idle prior to the Port making those Lanes electrically idle. The Retimer will see the EIOS, send it out and will tristate its drivers in the four outgoing Transmitter Lanes.

While upsizing a Link (say from x4 to x8), we need to activate some inactive Lanes. For activating the inactive Lanes, the Ordered Sets are sent which the Retimer will pass through and wake up those Lanes. These transitions between active and inactive Lanes are identical to what happens during existing low-power states in PCI Express specifications.

5. What is the entry and exit latency of L0p lane width transition? 

There is no impact to traffic flow with either entry or exit of L0p since PCI-SIG uses the existing periodic SKP Ordered Set to orchestrate when a Lane switches to an inactive state (width down configuration) or enters into the mix of other active Lanes sending traffic (width up configuration). Once a determination is made to reduce the Link width with L0p, one must wait until the next SKP Ordered Set boundary, which in the worst case will be 1.5 micro-seconds away. On the way to up-configure, it will be identical to L1 exit latency, which is dependent on the design and the amount of aggressive power savings the device implements. PCI-SIG expects this number to be in the micro-seconds range.

6. If the Link width changes back to L0, why not use Fast Training Sets (FTS) like L0s?

Fast Training Sets use an assumption-based approach to training after exiting from L0s. Thus, it forces designers to be cautious since the design needs to complete training within the given number of Ordered Sets. As a result, power savings tend to be modest because designs will cover for worst-case processes, voltage, temperature and other environmental condition variations. Similarly, designs also provide a higher number for the number of FTS for unforeseen field variations. On the other hand, a handshake-based training like L1 yields better power savings, though the exit latency can vary depending on the length of electrical idle and the variations in process, voltage, temperature and other environmental conditions.

7. What is the expected power envelopment for a 16x Lanes?

The expected power envelopment depends on the design and the channel reach the silicon is working with. As of now, we have seen power ranging in single-digit pJ/b. PCI-SIG expects similar power efficiency numbers to continue with the PCIe 6.0 specification. In terms of idle power, L1 sub-states are expected to consume single digit to very low double digit micro-watts of power per Lane. For wider links like x16, we expect similar power numbers, though it is possible that people can drive better power efficiency by amortizing the overhead of circuits shared across Lanes such as PLL, calibration, test, etc.    

8. L0p would impose L0s with restrictions post L0p exit for unused Lanes. What is the minimum residency for L0p?

L0s is unsupported in Flit Mode and L0p is supported only in Flit Mode; they do not co-exist. The minimum residency time in electrical idle per Lane in L0p is expected to be the same as L1. 

Do You Want to Learn More About the PCIe 6.0 Specification?

The recording of the PCIe 6.0 Specification webinar is available to watch anytime on the PCI-SIG YouTube channel. Also, this series of Q&A blogs will continue to provide answers to the questions asked by attendees during the live presentation. Follow PCI-SIG on Twitter on LinkedIn for updates about these blogs.