Noise Filters
When to use a noise filter
Noise Filters are usually employed to clean the input from DVS cameras; due to the architecture of the sensors, they are sensitive to Background Activity noise produced by temporal noise and junction leakage currents. Background activity increases with low light or higher sensitivity, so for different applications a filter is often needed. In some cases, a noise filter can be useful for eliminating real events due to minor light changes and keep a better separation between the moving subject and the background.
Use cases:
- low light conditions
- high sensitivity biases
- dark background
How to use a noise filter
Adding a noise filter is as easy as adding any other module. To add a noise filter you have to go to Structure, select Add module, then choose the desired filter and name it. Now you can connect the filter wherever you want it.
Filter modules can be added in every position in your workflow, they can be added before or after any module that processes events. Of course we recommend to use it directly after the input, as shown to the left of this image, doing so you will reduce the number of unwanted events in the pipeline and this will make the system run faster.
Filters
Dvsnoisefilter
The Dvsnoisefilter is a standard spatiotemporal filter that stores a map of event timestamps. The filter also provides a hot-pixel filter with learning to filter "broken" (always-active) pixels in the sensor, as well as a refractory period filter, which limits the rate at which single pixels can consecutively generate events.
This filter provides a fixed mask size of 8 neighboring events, as well as several advanced configurable options.
The most important parameters of the filter are:
| Parameter | Description |
|---|---|
| backgroundActivityTime | Maximum time difference in µs for events to be considered correlated and not be filtered out |
| backgroundActivityEnable | Enable the background activity filter |
| refractoryPeriodTime | Minimum time between consecutive events to not be filtered out |
| refractoryPeriodEnable | Enable the refractory period filter |
| hotPixelLearn | Learn the position of current hot (abnormally active) pixels |
| hotPixelEnable | Enable the hot pixel filter |
Best used for:
- Generic filter
- Static camera
Knoise Filter
The Knoise filter is based on the paper "O(N)-Space Spatiotemporal Filter for Reducing Noise in Neuromorphic Vision Sensors"; it is a spatiotemporal filter with memory optimizations. By only storing one timestamp per row and column, the memory usage and accesses are greatly reduced.
This filter provides only 2 parameters, the time interval to look up the spatiotemporal neighbors (deltaT) and the number of supporting pixels (supporters). This filter is very memory efficient but, as the paper shows, is prone to false negatives, meaning that real events can be filtered out because of missing past information. Currently the default parameter for deltaT is 1ms.
The parameters of the filter are:
| Parameter | Description |
|---|---|
| deltaT | Time delta for filter in µs |
| supporters | Number of supporting pixels |
Best used for:
- Moving camera
- Embedded systems
Ynoise Filter
The Ynoise filter is based of the paper "Event Density Based Denoising Method for Dynamic Vision Sensor", that describes a variant of the standard spatiotemporal filter.
At the moment we have only implemented what they call coarse filtering to achieve better performance. The complete algorithm adds one further step that they call fine filtering, used to remove some flickering noise. We also use the L∞ norm because it is easier to compute with a representation of the density matrix in an array structure.
The parameters of the filter are:
| Parameter | Description |
|---|---|
| deltaT | Time delta for filter in µs |
| lParam | Value L of LxL matrix for density matrix |
| threshold | Threshold value for density pixels |
Best used for:
- Generic filter
- Static camera
- Image reconstruction
Comparison
Comparing noise filters can be tricky because the performance and quality of the filter depend heavily on the parameters. As a general rule we can assume that better quality means worse performance. For the following comparison, we used the most balanced settings, that are currently the default settings in DV-Runtime.
We artificially generated a sequence of events without any noise, then added Gaussian noise to it, so that we could generate meaningful and precise metrics on noise filtering, signal filtering and so on. While not a perfect setup, it gives indicative results and is deterministic and reproducible.
Visual comparison
As shown in the image below, we can see that the Dvsnoisefilter keeps the most real events after filtering. The Ynoise filter looses some real events, but most of the shape of the plane and tree are intact. Instead Knoise is already corrupting the shape of the plane, due to the massive filtering going on.
Note: data100k is an artificially generated event file without noise, data_noise100k is that file with added Gaussian noise, the other images are the output of the discussed filters.
Signal to Noise Ratio (with default parameters)
Below we present a table with the different signal to noise ratios, calculated with default settings (most balanced) with our sample event files. As we can see the two most balanced filters are the Dvsnoisefilter and the Ynoise filter, they are slower than the Knoise filter, but they have a better SR (Signal Ratio). The SNR (Signal to Noise Ratio) is higher in the Knoise filter, because most of the events are eliminated (noise included), making the SNR higher and the SR lower; though it still is the fastest of the tested filters due to its memory trade-offs.
The formulas for calculating the SR, NR and SNR are the following:
Where:
Ef: stand for number of events after filtering
Eo: stand for number of events before filtering
Eq: can be f or o
signal: stand for original video
noise: stand for video with noise
The parameters for Dvsnoisefilter are (for both Event File):
- hotPixelEnable: false
- backgroundActivityEnable: true
- backgroundActivityTwoLevels: false
- backgroundActivityCheckPolarity: false
- backgroundActivitySupportMin: 1
- backgroundActivitySupportMax: 8
- backgroundActivityTime: 2000
- refractoryPeriodEnable: true
- refractoryPeriodTime: 100
The parameters for Knoise are (for both Event File):
- deltaT: 1000
- supporters: 1
The parameters for Ynoise are (for both Event File):
- deltaT: 10000
- lParam: 3
- threshold: 2
Event File 1 (duration 5s, 1280x960):
| Filter | SR | NR | SNR | CPU time (kernel time + user time) |
|---|---|---|---|---|
| Dvsnoisefilter | 0.9 | 0.08 | 208.06 | 7.92s |
| Knoise | 0.19 | 0.0 | 791.9 | 4.35s |
| Ynoise | 0.58 | 0.03 | 388.69 | 5.25s |
Event File 2 (duration 5s, 640x480):
| Filter | SR | NR | SNR | CPU time (kernel time + user time) |
|---|---|---|---|---|
| Dvsnoisefilter | 0.87 | 0.22 | 38.06 | 4.70s |
| Knoise | 0.15 | 0.01 | 101.56 | 2.02s |
| Ynoise | 0.63 | 0.07 | 86.93 | 4.92s |
Note: CPU time is relative to the system in use, so is only to be used as an indicator for comparison between filters.
Quality and Performance comparison chart
We recommend for standard usage (with static cameras or without significant scene changes) the Dvsnoisefilter and the Ynoise filters. For dynamic camera situations or significant scene changes, we recommend the Knoise filter, as it is faster and will provide a better compromise for keeping good filter performance and real-time processing speeds.