RoCEv2 and the path to 100GigE

1GigE has been, and remains, a useful and convenient camera protocol, allowing long, industrial cables and easily-managed multi-camera or multi-host systems in a familiar interface without proprietary technology – switches, network interface cards are available from many vendors with many configurations to suit different applications.

Over time the requirement for more data has meant that dual GigE, N-BaseT and 10GigE cameras have become widespread. This begins to place demands on the host machines that were not significant with 1GigE. The management of packets and movement of data place increasing requirements on the host’s CPU.

High-speed Ethernet

Systems based on 1GigE to 5GigE cameras have generally had manageable levels of CPU load on the host, with filter-drivers giving an efficient method of dealing with image-packets and jumbo packets reducing the CPU load and increasing the payload-efficiency of the Ethernet links.

With 10GigE cameras we see advantages to DMA (direct memory access) methods to overcome the bottleneck. RDMA (remote direct memory access) is a way to do this over a network, and RoCEv2 allows the data to be routed to manage the bandwidth. With increasing transfer speeds (25 and 100GigE), this becomes an important way to control the network load and the load on the host.

This scalability allows Ethernet-based cameras to compete with the fastest framegrabber-based systems. In the following comparison, we take CoaXPress over Fiber (CoF) as the high-speed benchmark.

RoCEv2 Fundamentals

RDMA means that network data transfers happen with no CPU intervention. This is absolutely different to other Ethernet-based transfers where each packet must be interrogated to find out its source, destination, payload-size and so on. All of these are CPU tasks. With RDMA the CPU load is minimal – less than 1% even at 100GigE speeds. In addition the lack of CPU intervention allows lower latency and makes the interface the limiting factor, not the host’s processing capability. Removing this bottleneck makes high-speed Ethernet cameras a scalable, practical reality.