Note

The screenshots below are examples only and show how a dialog for a feature described here might look like. Not every NIC/NIC driver will support every feature discussed here and different drivers might use different names for the features presented. The hints are just meant to give an idea for what to look for.

Network Interface Card

Using the correct network interface card is crucial. In case of the BVS CA-GT this would be a 10 Gigabit Ethernet controller which is capable of Remote Direct Memory Access (RDMA) technology.

Network Interface Card Driver

Also the network interface card manufacturers provide driver updates for their cards every now and then. Using the latest NIC drivers is always recommended and might improve the overall performance of the system!

Maximum transmission unit (MTU) / Jumbo Frames / Network Direct Packet Size

The MTU / Jumbo Frames is a common configuration value of mostly all Ethernet adapters. The Network Direct Maximum Transmission Size is an extra configuration parameter for Ethernet adapters which are supporting RDMA. In general, the MTU must not be smaller than the Network Direct Maximum Transmission Size. The resulting packet size for RoCEv2 data transfers is negotiated at connection establishment as part of the RoCEv2 protocol. Best practice is to set the MTU = 9000 and the Network Direct Maximum Transmission Size = 4096.

On the network interface card's side, this might look like this:

Setting "Jumbo Packet" using an Nvidia Connect-X6 network interface

There are other configuration settings for RDMA capable Ethernet adapters which must be assured to be set correctly.

The Network Direct Maximum Transmission Size must be set:

The Network Direct Functionality must be enabled:

The Network Direct Technology must be set: