Due to random process deviations, sensor manufacturers cannot guarantee that all pixels will behave the same under the same lighting conditions. These deviations are called blemishes or defective pixels. Balluff offers several methods for correcting these errors. By default, these are host-based calculations, but some device families support device-based corrections, which saves CPU load and reduces latency:

The removal of defective pixels generally requires two subtasks:

• Detection of defective pixels

• Correction of defective pixels

Both tasks can be performed at different "locations":

• Detection and correction on the host with Impact Acquire

• Detection on the host with Impact Acquire, correction on the device with mvDefectivePixelCorrectionControl of the device in list-based mode

• Detection and correction on the camera using mvDefectivePixelCorrectionControl in algorithm-based mode.

If the detection does not take place in real time, i.e. during the image acquisition itself, it is necessary to save the detected defects somewhere. This can be either on the device or on the host or both.

As mentioned above, the defect pixel list can be created with Impact Acquire. Since there are three types of defects, Impact Acquire offers three calibration methods for detection:

1. Leaky pixels (in the dark)
   are pixels that produce a higher readout code than average

2. Hot pixels (under normal lighting conditions)
   are pixels that generate a higher non-proportional readout code when the temperature rises

3. Cold pixels (under standard lighting conditions)
   are pixels that generate a lower readout code than the average when the sensor is exposed (e.g. due to dust particles on the sensor)

The following steps are required to detect leaky pixels:

1. Set Gain ("Setting -> Base -> Camera -> GenICam -> Analog Control -> Gain = 0 dB") and Exposure "Setting -> Base -> Camera -> GenICam -> Acquisition Control -> ExposureTime = 360 msec" to the given operating conditions. The total number of defective pixels found in the array depends on the gain and the exposure time.

2. Darken the lens completely.

3. Set the (filter) "Mode = Calibrate leaky pixel".

4. Acquire the image (e.g. by pressing Acquire in the ImpactControlCenter with "Acquisition Mode = SingleFrame").

The filter checks:

Pixel > LeakyPixelDeviation_ADCLimit // (default value: 50)

All pixels above this value are considered leaky pixels.

With "Mode = Calibrate Hot And Cold Pixel", both detections can be carried out simultaneously.

The following steps are necessary to detect hot or cold pixels:

• A uniform sensor illumination of approx. 50 - 70 % saturation (i.e. an average gray value between 128 and 180) is required.

• Set the (filter) "Mode = Calibrate Hot Pixel" or "Mode = Calibrate Cold Pixel" or "Mode = Calibrate Hot And Cold Pixel".

• Acquire an image (e.g. by pressing Acquire in the ImpactControlCenter with "Acquisition Mode = SingleFrame").

The filter checks:

Pixel > T[hot] // (default value: 15 %)

// T[hot] = deviation from the average gray value

Pixel < T[cold] // (default value: 15 %)

// T[cold] = deviation from the average gray value

Figure 1: Image corrections: DefectivePixelsFilter

By repeating the corrections for defective pixels, the correction data is accumulated, resulting in a higher value in "DefectivePixelsFound". If you want to reset the correction data or repeat the correction process, you must set the filter mode to "Reset Calibration Data". The "DefectivePixelsMaxDetectionCount" property can be used to limit the number of defective pixels detected.

Corresponding functions are available for saving and loading the defective pixel data:

• "int mvDefectivePixelDataLoad( void )"

• "int mvDefectivePixelDataSave( void )"

The "Settings -> Basic -> ImageProcessing -> DefectivePixelsFilter" area has also been expanded (see Figure 2). First of all, the "DefectivePixelsFound" indicates the number of defective pixels found. The coordinates are now available via the "DefectivePixelOffsetX" and "DefectivePixelOffsetY" properties. In addition, it is possible to manually edit, add and delete these values (by right-clicking on the "DefectivePixelOffset" and selecting"Append Value" or"Delete Last Value"). Secondly, with the functions

• "int mvDefectivePixelReadFromDevice( void )"

• "int mvDefectivePixelWriteToDevice( void )"

you can exchange the filter data with the device and vice versa.

Simply right-click on "mvDefectivePixelWriteToDevice" and click on "Execute" to write the data to the device (and pass the data to the Storing pixel data on the device function). To save the data permanently in the non-volatile memory of the device, "mvDefectivePixelDataSave" must also be called afterwards!

Figure 2: Image corrections: DefectivePixelsFilter (from driver version 2.17.1 and firmware version 2.12.406)

When the device is opened, the device loads the defective pixel data from the device. If pixels are present in the filter (via calibration), you can still load the values from the device. In this case, the values are merged with the existing ones. This means that new values are added and duplicates are removed.

Figure 3: Defective pixel data is written to the device (from driver version 2.17.1 and firmware version 2.12.406)

Once a defect list has been created, a host-based correction can be carried out with Impact Acquire.

Various replacement methods are available to correct the defective pixels:

1. "Replace 3x1 average"

   Replaces the detected defective pixels with the average value of the left and right neighboring pixel (3x1)

2. "Replace 3x3 median"

   Replaces the detected defective pixels with the median value, which is calculated from the nearest neighboring pixels in a 3x3 region.

3. "Replace 3x3 Filtered Data Averaged"

   Replaces the detected defective pixels and treats them as if they had been processed with a 3x3 filter algorithm before they reached this filter.

   Only recommended for devices that do not offer compensation for defective pixels; packed RGB or packed YUV444 data is required. See enumeration value dpfmReplaceDefectivePixelAfter3x3Filter in the corresponding API manual for additional details about this algorithm and when and why it is needed

As already described, it is possible to load lists of defective pixels onto the device. Various algorithms can be used to determine whether a pixel is defective or not, which depends on how much a pixel is allowed to deviate, the temperature, the gain and the exposure time. As already described, list-based correction is deterministic, i.e. it is known exactly which pixels will be corrected.

However, list-based correction has some disadvantages:

• A standard list is saved in the device during production, but this may not suit the target application as the temperature/exposure time settings are very different.

  → It is necessary to create the list using a recognition algorithm (or to support Impact Acquire)

• With time and aging of the sensor, new defects can/will appear.

• It does not work in binning/decimation modes

• The memory for storing defective pixels is limited

In this case, the device performs the detection and correction on-the-fly without using an error list.

Adaptive correction addresses the disadvantages of the list-based method mentioned above. While the correction itself (i.e. which pixels are used to correct a detected defect) is the same, no static information from a list is used, but it is detected on the fly.

In order to use reasonable thresholds, knowledge of the noise statistics of the sensor is used to detect the outliers. These are also corrected on the fly. Since this is a dynamic approach, it also works in binning/decimation mode and would also detect newly occurring errors.

Nevertheless, there are some disadvantages:

• It is not deterministic

• False positives can be detected, i.e. non-defective pixels could be treated as defective.

• If pixels are at the edge of the threshold values used, they can be corrected in one image but not in the next.

On BVS CA-SF devices, adaptive correction is always used if:

• no list is stored on the device

• binning or decimation is used

Each pixel of an image sensor is an individual detector with its own properties. This applies in particular to the sensitivity or spectral response. To solve this problem (including lens and illumination variations), a monochrome calibration plate (e.g. white or gray) is recorded as a flat field, which is used to correct the original image. The optics must not be changed between the flat-field correction and the subsequent application. To reduce errors during flat-field correction, a saturation between 50 % and 75 % of the flat-field in the histogram is useful.

The following steps are required to perform a flat-field correction:

• A monochrome and uniformly "colored" calibration plate (e.g. white or grey) is required.

• Not a single pixel may be saturated - we therefore recommend setting the maximum gray value in the brightest area to max. 75% of the gray scale (i.e. to gray values below 190 when using 8-bit values)

• Select BayerXY under "Settings -> Basic -> Camera -> GenICam -> Image format control -> PixelFormat".

• Set the (filter) "Mode = Calibrate" (Figure 4).

• Start live recording ("Acquire" with "Acquisition Mode = Continuous").

• Activate the correction: Set the (filter) "Mode = On".

• Set the settings including the correction data via "Action -> Capture Settings -> Save Active Device Settings"

  (The settings can be saved in the Windows registry or in a file)

Figure 4: Image corrections: Host-based flatfield correction

After restarting the device, you must reload the capture settings.

The filter takes a number of images (corresponding to the value of CalibrationImageCount, e.g. 5) and averages the flat-field images to form a correction image.

In some cases, it may be necessary to use only a specific area within the device's field of view to calculate the correction values. In this case, only a specific AOI is used to calculate the correction factor.

The "Host-based flat-field correction" can be performed in the following way:

• All necessary settings can be found under "ImageProcessing"-> "FlatfieldFilter".

• Stop the "Continuous" recording mode.

• Set "CalibrationImageCount" to e.g. 5.

• Set the "Mode" to "Calibrate".

• Set "CalibrationAoiMode" to "UseAoi".

• Set the properties ("X, Y and W, H") that appear under "CalibrationAOI" to the desired AOI.

• Start the "Continuous" acquisition mode.

• Activate the correction: Set the "Mode" to "On".

Figure 5: Image corrections: Host-based flatfield correction with calibration AOI

In some cases, it may be necessary to correct only a specific area in the device's field of view. In this case, the correction values are only applied to a specific area. For the rest of the image, the correction factor is only 1.0.

The "Host-based flatfield correction" can be set as follows:

• All necessary settings can be found under "ImageProcessing"-> "FlatfieldFilter".

• Stop the "Continuous" recording mode.

• Set "CalibrationImageCount" to e.g. 5.

• Set the "Mode" to "Calibrate".

• Start the "Continuous" capture mode.

• Activate the correction: Set the "Mode" to "On".

• Set "CorrectionAOIMode" to "UseAoi".

• Set the properties ("X, Y and W, H") that appear under "CorrectionAOI" to the desired AOI.

Figure 6: Image corrections: Host-based flatfield correction with correction AOI

The device-based flat-field correction supports full AOI and 14-bit to 14-bit correction (12-bit coefficients). This enables pixel-to-pixel correction and saves tens of % CPU load and reduces latency. To reduce noise, averaging over a series of "mv Flat-Field Correction Calibration Images" is possible. A correction image is then calculated in the device. This may take some time / number of images, but the device flashes green. A correction image can be saved for all user settings.

The device-based flat-field correction is independent of the offset and uses a trigger in run mode (e.g. external trigger).

Figure 7: ImpactControlCenter - Settings

The device-based flat-field correction can be set as follows:

• "mv Flat-Field Correction Calibration Image Count" to e.g. 5.

  This averages 5 images before the FFC factors are calculated to reduce the effects of noise.

• Stop the "Continuous" acquisition mode.

• Right-click on "int mvFFCCalibrate()" -> "Execute".

• Activate the correction: set "mv Flat-Field Correction Enable = 1".

Depending on the sensor, this may take some time as the data is stored in the internal flash memory (yellow LED lights up).

In some cases, it may be necessary to use only a specific area within the device's field of view to calculate the correction values. In this case, only a specific AOI is used to calculate the correction factor.

The "device-based flat field correction with AOI" can be set as follows:

• Start the "Continuous" acquisition mode.

• Use the properties ("OffsetX, OffsetY and W, H") to configure the desired AOI for the FFC calculation.

  First stop the acquisition to change W and H in Image Format Control. Only OffsetX and OffsetY can be changed during operation.

• Set "mvFFCCalibrationImageCount" to e.g. 5.

  This averages 5 images before calculating the FFC factors to reduce the effects of noise.

• If not already done in step 2, stop the "Continuous" acquisition mode.

• Then right-click on "int mvFFCCalibrate()" -> "Execute".

• Use the properties ("OffsetX, OffsetY and W, H") to set the full image size or the desired AOI.

• Activate the correction: Set the value "mvFFCEnable = 1".

The flat field correction is applied to the entire image.

Figure 8: ImpactControlCenter - Setting the calculation AOI for flat-field correction

If the FFC works as expected, it is possible to save the FFC calibration values in the device if the mvFFCImageSave property is available. Depending on the sensor, this will take some time as the data is stored in the internal flash memory (yellow LED lights up). The FFC values can then be loaded from the flash using the mvFFCImageLoad command. If the values are to be loaded each time the device is switched on, this can be done by setting "mvFFCAutoLoadMode = On".

Color devices with an FW version lower than 2.40 use a multiplication factor of 1.3 on the correction factor of each color. This factor minimizes colour artefacts when the scene becomes brighter. Newer firmware versions use a multiplication factor of 1.0 and use the AOI method described above to minimize colour artefacts.

Figure 9: ImpactControlCenter - Save/load FFC values