What interconnects are used with memory for HPC and AI?

By Jeff Shepard

Collected at: https://www.designworldonline.com/what-interconnects-are-used-with-memory-for-hpc-and-ai/

High-performance computing (HPC) for artificial intelligence (AI) applications places extreme demands on memory connectors. They must deliver high-speed connectivity, high levels of signal integrity (SI), and support high-density solutions. This article reviews connector needs for Peripheral Component Interconnect Express 6.0 (PCIe 6.0), Serial Attached SCSI (SAS) including SAS 3.0, 4.0 and SAS/PCIe 5.0, and Data Rate 5 Synchronous Dynamic Random-Access Memory (DDR5 SDRAM).

While SAS/PCIe 5.0 connectors support data transfer speeds of 32GT/s for PCIe lanes and 24Gb/s for SAS lanes, next-generation PCIe 6.1 achieves 64 GT/s. Several new technologies were used to reach that new performance plateau including:

Pulse Amplitude Modulation with four levels (PAM4) signaling combines 2 bits per clock cycle for 4 amplitude levels to encode 00, 01, 10, and 11, compared with non-return to zero (NZR) modulation used in earlier generations of PCIe, which used 1 bit per clock cycle and two amplitude levels to encode 0 and 1 (Figure 1).

Even with connectors that deliver superior SI, the move from NZR to PAM4 can result in a higher bit error rate (BER). To mitigate the increased BER, PCIe 6.1 has added a forward error correction (FEC) mechanism that works in coordination with a cyclic redundancy check (CRC). The new approach adds less than 2 ns of latency and enables PCIe 6.1 to maintain the same reach as earlier generations.

SAS connectors

SAS connectors feature reduced cross talk and enhanced performance for use with enterprise solid state drives (SSDs) and PCIe Gen 4 and Gen 5 connectivity. There are three primary SAS connector standards: Storage Networking Industry Association (SNIA) small form factor 8680 (SSF 8680), SFF 8681, and SFF 8639.

SAS 3.0 connectors have 29 positions and handle data rates up to 12 Gbps. They are compatible with the SFF 8680 specification and support both SAS and SATA drives. Features include hot plugging, blind mating misalignment compensation, and a retention mechanism.

SAS 4.0 connectors also have 29 positions and support faster data rates up to 24 Gbps. They have features like SAS 30 connectors, but they are compatible with the SFF 8681 specification. They are often used for SATA drives in mission-critical applications like high-performance computing for AI and bulk storage systems.

SAS/PCIe 5.0 connectors are compatible with SFF 8639 and support U.2 and U.3 applications. U.2 supports standard SAS and SATA-based spinning disks and solid-state drives (SSDs). U.3 uses the same connector and is a “tri-mode” standard that combines SAS, SATA, and nonvolatile memory express (NVMe) support into a single controller. These connectors support data transfer speeds of 32GT/s for PCIe lanes and 24Gb/s for SAS lanes.

DDR5

DDR5 modules are available in three electrical formats: unbuffered (U), registered (R), and load-reduced (L). DDR5 connectors for U, R, and L modules supporting channels with transfer rates up to 6.4 GT/S are defined in the Joint Electron Device Engineering Council (JEDEC) Standard PS-005B — DDR5 288 Pin U/R/LR DIMM Connector Performance Standard.

The JEDEC standard defines the form, fit, and function of DDR5 connectors, including mechanical, electrical, and reliability requirements for a connector mated to a 1.27 mm thick module. The standard supports mounting multiple DDR5 modules on a circuit board, usually with 288-pin connectors. As with earlier generations of DDR memory, DDR5 is available in several DIMM configurations.

A notch in the center of DDR modules aligns with the DDR socket for proper mating. The notch in DDR5 sockets also prevents non-DDR5 modules from being installed in the system. DDR5 notches differ between U and R module types (Figure 2).

Summary

A variety of interconnect types are required to support the memory needs of high-performance computing and AI systems. Increasing storage densities and speeds have resulted in new connector types being introduced for PCIe 6.1, SAS 4.0, and DDR5 memory technologies.

References

DDR5 Memory Standard, Kingston Technology
DDR5 SDRAM, Wikipedia
Enabling High Speed with DDR5 DIMM Sockets, TE Connectivity
PCIe 6.1 – All you need to know about PCI Express Gen6, Rambus
PCIe Gen 6 High Speed Interconnect Solutions, Amphenol
Standard PS-005B – DDR5 288 Pin U/R/LR DIMM Connector Performance Standard, DDR5, JEDEC