The Nitro cards are a family of cards that offloads I/O functions such as networking, security, and virtualization for service acceleration and to improve the system performance. The Nitro cards are built for specific purposes and will have their own System of Chip (SoC) software. There are 4 different types of Nitro cards as below:
- Nitro card for VPC
- Nitro card for EBS – NVMe based storage.
- Nitro card for instance storage
- Nitro card for system control (brain behind the Nitro system)
Nitro card for VPC is a network card to handle VPC traffic. It is similar to a physical PCIe card with network ports on one end and a PCIe bus on the other end. It uses SR-IOV to create different virtual network functions to offer enhanced network functionalities. In addition to the network I/O functionality, the nitro cards also perform the other functionalities of the VPC such as:
- Overlay Encapsulation and Decapsulation
- Security Groups
- Traffic Rate Limiter
- Traffic Routing
These OS-bypass capable Nitro cards come with an optional capability to create Elastic Fabric Adapter (EFA) which are primarily targeted for High Performance Computing (HPC) and Machine Learning (ML) applications. The traditional behavior of processing the network packets via the kernel’s TCP/IP stack is not sufficient to address such low latency requirements. This is addressed using the below enhancement:
A new open-source library known as libfabric was developed as part of Open Fabric Interface (OFI) work group which aims to create a family of application program interfaces (APIs) to exposes the network data directly from the NIC cards to the middleware and/or applications. The libfabric API is leveraged (and installed as a module on the host) to bypass the host kernel TCP/IP stack and directly create message queues to the Nitro card drivers. This helps achieve high performance and low latency for the applications.
Any HPC or ML application may have distributed workloads performing the computing functionalities and may need to exchange data between them in a low-latency manner. While the use of OFI/Libfabric helps bypass the kernel, this is not sufficient for inter-host communication over the fabric. To address such requirements, a new cloud optimized transport protocol referred as Scalable Reliable Datagram (SRD) is used by the Elastic Fabric Adapter (EFA) for the overlay virtual network for communication between workloads distributed across different servers. SRD which is inspired by Infiniband combines the positive and working characteristics of both UDP and TCP to improve the overall performance. SRD allows ECMP spraying to take advantage of the availability of multiple paths between the host. It also supports packet out-of-order delivery and let the upper layer handle the re-ordering.
More details about SRD are available here - https://ieeexplore.ieee.org/document/9167399
The 2nd generation of EFA was recently introduced in the late 2022 for 6th generation compute and memory optimized instances. EFA v2 enables full RDMA semantics and bumps the bandwidth from 100 to 200 GBps.
A quick comparison of Nitro generation cards below:
No comments:
Post a Comment