Accumulator Module
What does the accumulator do?
The accumulator module takes events and prints them onto a frame. Removal of the information is done with a configurable decay function.
Use cases:
- Create high-speed event frames
- Frame reconstruction
- Long term noise accumulation
- Creating representations to feed into a convolutional neural network
Accumulation can be used whenever there is a need to convert events into frames
Setting up an accumulator
Adding accumulator module
The default Visualize configuration of DV already has an accumulator set. To add another accumulator, click on the Structure tab in DV, and add a Dv accumulator module by clicking on Add module.
After giving the new accumulator a name, it should show up in the module configuration. Connect the accumulator to your configuration, by dragging a connection between the blue events output and the newly created accumulator module. Also drag a connection from the accumulator module frames output to an empty visualizer input.
Start the visualizer by clicking onto the play icon.
Adjusting accumulator settings
After the accumulator has been added, you can change the settings of the accumulator in the right bar. To access all of the accumulator settings, click on the plus sign on the top right.
Position of the accumulator sidebar. Click the purple plus button to access more options.
The pop up window with all the accumulator options after clicking to plus button
Accumulator settings overview
Accumulation time
This sets the time window to accumulate events before sending out a new frame in milliseconds. This effectively sets the frame rate of the accumulator. The accumulator does not reset the frame at generation time, the contents of the old frame carried over to the next frame. To achieve an effect where the frame gets cleared out after each frame, set the decay function to Step and make sure the Decay param is set to the same value as the accumulation time. Note that this value is in milliseconds, while the other one is in microseconds.
Decay function / Decay param
One of None, Linear, Exponential, Step. Defines the data degradation function that should be applied to the image. For each function, the Decay param setting assumes a different function:
Function | Decay param function | Explanation |
---|---|---|
None | No function | Does not apply any decay |
Linear | The slope a of the linear function, in intensity per microsecond | Assume intensity of a pixel is I0 at time 0 , this function applies a linear decay of the form of I0 - (t * decayparam) or I0 + (t * decayparam) until the value hits the value specified in neutral potential |
Exponential | The time constant tau of the exponential function in microseconds | Assume intensity of a pixel is I0 at time 0 , this function applies an exponential decay of the form I0 * exp(-(t/decayparam)) . The decay approaches a value of neutralPotential over time. |
Step | No function | Set all pixel values to neutral potential after a frame is extracted. |
Event Contribution
The contribution an event has onto the image. If an event arrives at a position x
, y
, the pixel value in the frame at x
, y
gets increased / decreased by the value of Event contribution, based on the events polarity.
Except:
- The resulting pixel value would be higher than Max potential, the value gets set to Max potential instead
- The resulting pixel value would be lower than Min potential, the value gets set to Min potential instead
- The event polarity is negative, and Rectify polarity is enabled, then the event is counted positively
Min potential / Max potential
Sets the minimum and maximum values a pixel can achieve. If the value of the pixel would reach higher or lower, it is capped at these values. These values are also used for normalization at the output. The frame the module generates is an unsigned 8-bit
grayscale image, normalized between Min potential and Max potential. A pixel with the value Min potential corresponds to a pixel with the value 0
in the output frame. A pixel with the value Max potential corresponds to a pixel with the value 255
in the output frame.
Neutral potential
Sets the neutral potential. This has different effects, depending on the decay function:
Function | Neutral potential function |
---|---|
None | No function |
Linear | The value the linear function tends to over time |
Exponential | No function |
Step | The value the frame gets reset to after getting a frame |
Rectify polarity
If this value is set, all events act as if they had positive polarity. In this case, Event contribution is always taken positively.
Synchronous Decay
If this value is set, decay happens in continuous time for all pixels. In every frame, each pixel will be eagerly decayed to the time the image gets generated. If this value is not set, decay at the individual pixel only happens when the pixel receives an event. Decay is lazily evaluated at the pixel.
Note that both decay regimes yield the same overall decay over time, just the time at which it is applied changes. This parameter does not have an effect for Step decay. Step decay is always synchronous at generation time.
Color demosaicing
Adds a Bayer color demosaicing filter onto the generated frame. Note that this setting only makes sense when a color dvs camera is connected.
Example configurations
High fps event frames
For a use case with a very fast frame-based processing engine, one can get high-speed hdr images of the moving edges. The following settings give 1000 fps with frames that are exposed over 1ms integration time
Setting | Value |
---|---|
Accumulation time | 1 (ms) |
Decay function | Step |
Decay param | Ignored |
Min potential | 0 |
Max potential | 1 |
Neutral potential | 0.5 |
Event contribution | 0.5 |
Rectify polarity | No |
Synchronous decay | Ignored |
Exponentially decayed frame reconstruction
The following configuration gives a good frame reconstruction if there is a lot of activity in the scene. The exponential decay is pretty strong, which leads to strong shadows when a movement changes direction. The strong decay suppresses noise quite effectively.
Setting | Value |
---|---|
Accumulation time | 33 (ms) |
Decay function | Exponential |
Decay param | 1000000 (us) |
Min potential | 0 |
Max potential | 0.3 |
Neutral potential | 0 |
Event contribution | 0.04 |
Rectify polarity | No |
Synchronous decay | Yes |
The fps can be arbitrarily set. In this example it is set to 33ms to achieve 30 fps. The quality is independent of the fps, if you require 1000 fps, you can just set the Accumulation time to 1 (ms).
Linear decayed frame reconstruction
A reconstruction can also be achieved with a linear decay. Generally, this tends to preserve less detail, but suffers less from shadows.
Setting | Value |
---|---|
Accumulation time | 33 (ms) |
Decay function | Linear |
Decay param | 0.000001 (intensity / us) |
Min potential | 0 |
Max potential | 1 |
Neutral potential | 0.5 |
Event contribution | 0.15 |
Rectify polarity | No |
Synchronous decay | Yes |
Analogous to the exponential case, one can set the fps by changing the Accumulation time parameter.
Further configurations
It is advised to experiment with the accumulator settings until the desired effect is achieved.