Obtaining the lowest latency with your Ethernet IP autofocus-zoom camera

Image capture and video data transmission will always be subject to latency because of the unavoidable acquisition, transport and processing delays between various system components. Here, and in more detail in our technical note, we explain how latency enters a vision system and how to set up components to minimize it. The technical note also includes measured latency for our Harrier IP AF-Zoom camera modules.

Sources of latency and optimizations

Many different elements of the vision system add latency to the overall data transmission. The diagram below shows where latency may be introduced.

Getting the best results from the camera

AF-Zoom block cameras are sophisticated imaging instruments and they have many image processing features (e.g. auto white balance, auto exposure, lens distortion compensation, noise reduction, and more). However, the processing required for these features takes time, increasing the latency of the camera video.

A typical AF-Zoom camera will have latency of 2-6 frames depending on the camera and the processing operations performed. The latency is also usually related to the video frame rate; e.g. features that require the processing of two image frames can only be applied after two frames have been captured, giving a minimum latency of at least a two frame time period. If the frame rate is faster, this time limitation is smaller – hence there is usually a direct relationship between frame rate and video output latency.

To minimize latency in the block camera you should set a fast frame rate and not use complex image processing features; your camera may have low latency modes that can be enabled using a VISCA command. Most Harrier Ethernet IP cameras have a low latency mode.

The right IP processor

The Harrier IP interface board on the Harrier IP cameras receives the image data from the camera module and can perform additional image processing such as the addition of overlays or image scaling. The images are then H.264 compressed and converted to a data packet format (IP/UDP) that can be sent to the network. Latency is reduced by using hardware to encode the video, and by optimizing/minimizing the video processing required.

Network requirements

Avoid using a complex and busy network; a point-to-point (camera to host PC) connection will deliver the lowest latency and jitter.

Choosing a PC

As shown above, the receiving computer/PC will add further latencies; data decompression and (optional) image processing contributing the greatest delays within the PC. Selecting a high-performance computer that can receive, decode, process and display the images quickly and without interruption will help deliver low latency, low jitter video transmission.

The video rendering software can also make a significant difference to the latency. To reduce latency, ensure that the software can be, and is, set up to do minimal buffering and processing of the video stream before it sends the images to the screen. Check the properties of your video renderer and potentially use UDP packets instead of TCP.

The role of the monitor

Finally, the time taken for the monitor electronics to receive and display the image on the LCD screen and can vary between 5ms and 68ms. Some televisions and monitors/displays buffer a whole frame before displaying the image, adding a minimum latency of one frame period (~16.7ms @60fps). Others may also perform image processing to scale the image and/or reduce image artefacts. For minimum latency, carefully check the features of your display, turn off all processing options that add latency and set the video resolution to be the same as the native monitor resolution.

Obtaining the lowest latency in your vision system

Harrier IP cameras with a Harrier camera interface board leverage high-quality components to optimize for providing low latency video. In addition, they offer a special low latency configuration, which delivers 1080p30 video with lower latency. Some other AF-Zoom block cameras, including Tamron and Sony cameras, have low latency modes as well.

While each element of a vision system will add to the overall latency, developers can address aspects of each component to reduce this as much as possible. Below is a summary of our recommendations to minimize latency:

• Set a high frame rate on the AF-Zoom camera (60Hz).
• Do not use processing-intense AF-Zoom camera features.
• Enable low latency mode if the AF-Zoom camera has one.
• If the volume of the data at the high frame rate is too high, set the Harrier IP Camera Interface board Encoding Interval to 2 to deliver 1080p30 with ~1080p60 latency.
• Use a hardware accelerated IP processing interface board such as the Harrier IP Camera Interface Board.
• Operate a simple network that doesn’t carry high amounts of unrelated data.
• Use a point-to-point network connection for the lowest latency and jitter.
• Select a high-performance computer that can receive, decode, process and display images quickly without interruption.
• Select software which can be set to perform minimal buffering and video processing.
• Ensure that the display screen has minimal latency.

Our technical note goes in to more detail on which elements add the most latency to video transmission and how to reduce this as much as possible. You will also find glass-to-glass measurements of latency for many of our Harrier IP AF-Zoom cameras.