RoCEv2 booster for 25 GigE and the path to 100 GigE
1. Introduction
1 GigE is a useful and practical camera protocol that enables long industrial cables and easy-to-manage multi-camera or multi-host systems in a familiar interface without proprietary technology – switches and network cards are available from many suppliers in numerous configurations for different applications.
Over time, the need for more data has led to the widespread use of dual GigE, N-BaseT, and 10 GigE cameras. This places demands on the host computers that were not significant with 1 GigE. Packet management and data transfer are placing increasing demands on the host's CPU.
2. High-Speed Internet
Systems based on 1 GigE to 5 GigE cameras generally have manageable CPU utilization on the host, with filter drivers providing an efficient method for processing image packets and jumbo packets. This reduces CPU utilization and increases the payload efficiency of Ethernet connections.
With 10 GigE cameras, direct memory access (DMA) methods overcome the data bottleneck. Remote direct memory access (RDMA), for example, is one way to achieve this over a network, enabling data transfers with low latency and low overhead. RoCE (RDMA over Converged Ethernet) or RoCEv2 goes one step further and allows data to be routed to manage the bandwidth. With transmission speeds increasing from 25 to 100 GigE, this is becoming an important method for controlling network utilization and host utilization.
Thanks to this scalability, Ethernet-based cameras can compete with the fastest frame grabber-based systems.
3. RoCEv2 Basics
RDMA means that network data transfers take place without CPU intervention. This differs fundamentally from other Ethernet-based transfers, where each packet must be queried to determine its source, destination, payload size, etc. All of these are tasks performed by the CPU. With RDMA, CPU utilization is minimal - even at 100 GigE speeds, it is less than 1%. In addition, eliminating CPU intervention enables lower latency and makes the interface the limiting factor, not the host's processing power. By eliminating this bottleneck, high-speed Ethernet cameras become a scalable, practical reality.
Figure 1: Streaming without RoCEv2 
Figure 2: Streaming with RoCEv2
4. Comparison of 25 and 100 GigE cameras with CoaXPress over fiber optics
|
|
25 or 100 GigE |
CoaXPress over fiber optic (CoF) |
Comments |
|
Bandwidth |
100 GigE interface is 100 Gbit/s, of which approximately 99 Gbit/s are usable |
~100 Gbps |
Comparable interface speeds |
|
Transmission medium |
Fiber optic cable, QSFP connectors |
Fiber optic cable, QSFP connectors |
|
|
CPU overhead |
<1% due to RDMA |
Minimal due to DMA |
Comparable CPU overhead |
|
Error message |
The network architecture allows the use of multiple cameras and hosts. With RoCEv2, CPU utilization is not dependent on bandwidth |
Errors can usually be reported but not corrected |
|
|
Scalability |
Network architecture allows multi-camera and multi-host. RoCEv2 allows the bandwidth to be routed and CPU load is not bandwidth-related |
Frame grabber-based systems are point-to-point systems, but splitters are available |
Flexibility advantage with Ethernet-based systems |
|
Synchronization |
Hardware triggering on the camera and IEEE1588 PTP for clock synchronization of separate devices |
Hardware triggering generally at the camera and frame grabber |
|
|
Processing |
There are NICs with integrated FPGA processing. RDMA also facilitates GPU-direct and NVMe-direct workflows |
Many frame grabbers have certain processing functions such as Bayer conversion, and some also have general processing functions |
|
|
CPU overhead for capture |
RoCEv2 reduces this to a minimum of about 1% |
Minimal due to DMA |
|
|
Consumption (camera) |
25 GigE cameras in the range of 11 to 13 W 100 GigE cameras in the range of 18 W |
Estimated: 34 W |
Lower power consumption with Balluff Ethernet cameras |
|
Consumption (system) |
100 GigE NIC typically 10 to 29 W |
Estimated: 30 W |
Similar power requirements |
Figure 3: Data rates depending on network protocol