Image Data Is Lost (Incomplete Frames)

Symptoms

While acquiring data from the device image data is lost e.g. the counter for incomplete or lost images is increasing or request objects inside an application are returned with errors.

Cause

There are A LOT of potential causes for this so a general cause as well as a general resolution cannot be given:

• Switches, routers or cables in between the device and the NIC of the host system might simply not be able to cope with the amount of data coming from the device.
• The NIC might not be able to get rid of the incoming data fast enough.
• The underlying driver might not be fast enough to send the captured data up to the application.
• The application might not process the received data fast enough eventually blocking all the memory that could be used for capturing data into thus the driver must drop incoming data.

Resolution

Since there are so many potential reasons for data loss, the following section will try to provide some guidance to track down the root cause on a specific system step by step.

1. Make sure that everything described here has already been set up in an optimal way: Optimizing Network Performance.
2. Check your BIOS for stuff mentioned here: The error counter increases.
3. Make sure you have understood the concepts for bandwidth management described here and that you have set up your system and application accordingly.

If the problem persists after everything described in the sections mentioned above has been configured correctly and it has been double checked that the bandwidth coming from the device(s) should be digestible by the NICs they are connected to:

First of all try to reproduce the issues using ImpactControlCenter or multiple instances of ImpactControlCenter with the same settings (or at least almost the same) as your application. Depending on that:

• If ImpactControlCenter is working fine but your application is not try to find out what is different:
  1. Check the amount of request objects used by ImpactControlCenter and your application.
  2. Check if your application sometimes blocks a lot of request objects because of extensive image processing so the driver has no more buffers to capture into.
  3. Check if queuing and retrieving buffers from the Impact Acquire driver in your application actually works as expected by checking for error codes returned from the various imageRequestXYZ functions.
• If ImpactControlCenter is showing the same behaviour:
  • When every frame is corrupted check the actual link speed reported by the device and NIC for link speed negotiation issues. Maybe one of the components for some reason is running at a reduced (100 MBit/s, ...) link speed.
  • Check if the MTU has actually been negotiated as expected, so look for the GevSCPSPacketSize property and check if the value is close to the MTU configured for the NIC. If it is not try to set it to a higher value (close to the MTU of the NIC) manually but be aware that a switch in between might not be capable of dealing with jumbo packets. If that is the case you will no longer receive ANY data! Think about exchanging components that cannot deal with jumbo frames with components that can.
  • Check if reducing the image size, frame rate or the bandwidth (see Limiting the bandwidth of the imaging device) helps. IMPORTANT: Reducing the image size and reducing the bandwidth can result in the same amount of data being sent per second, but still the transmission on the wire is different! Smaller ROIs or a reduced frame rate will transmit the images at full link speed but with LARGE GAPS in between the individual images while using the bandwidth limit features the network data will be sent using a lot of SMALL GAPS in between all individual network packets (and one image typically consists of a lot of network packets). So reduced ROIs might actually have no effect on the amount of lost data while reducing the bandwidth might! If that is the case your network connection in combination with the NIC is most likely the cause for the problem. It is also important to understand that when 2 devices are transmitting their data simultaneously on the same link without reducing their bandwidth usage, the peak data might be beyond the transmission links capacity! E.g. 2 1 GBit/s devices sending a very small image at exactly the same time might generate 2 GBit/s peak bandwidth! As most switches have very small buffers this might result in data loss very quickly! The only downside of reducing the bandwidth usage is that the full image will arrive in the application slightly later, since the packets are sent with some delay, but for multi-camera scenarios this very often is the only feasible option.
  • Try to remove/exchange certain components like switches.
  • Look for updated NIC or Impact Acquire drivers.