Abstract:Event-based vision sensors are appealing because of their time resolution, higher dynamic range, and low-power consumption. They also provide data that is fundamentally different from conventional frame-based cameras: events are sparse, discrete, and require integration in time. Unlike conventional models grounded in established geometric and physical principles, event-based models lack comparable foundations. We introduce a method to generate event-based data under controlled transformations. Specifically, we subject a prototypical object to transformations that change over time to produce carefully curated event videos. We hope this work simplifies studies for geometric approaches in event-based vision. GERD is available at <a class="link-external link-https" href="https://github.com/ncskth/gerd" rel="external noopener nofollow">this https URL</a>
What problem does this paper attempt to address?
The core problem that this paper attempts to solve is: **How to generate controllable event - based data to promote the understanding and improvement of event - based vision systems?**
Specifically, event - based visual sensors (such as Dynamic Vision Sensors, DVS) are different from traditional frame - based cameras. They capture sparse and asynchronous data points. These data points are unevenly distributed in time and space and need to be processed by time integration. However, existing event - based models lack a solid foundation in geometric and physical principles like traditional frame - based models. Therefore, it is difficult for researchers to deeply understand the spatio - temporal structure behind these event - based data, which limits the development of event - based computer vision models.
To solve this problem, the author proposes a method for generating event - response data under controlled transformation, called **GERD (Geometric Event Response Data generation)**. Through this method, it is possible to:
1. **Generate event videos under controlled transformation**: By applying time - varying transformations (such as translation, scaling, rotation, etc.) to typical objects, carefully planned event videos are generated.
2. **Study spatio - temporal structure**: Study the spatio - temporal structure of event data streams under controlled conditions, providing a prerequisite for processing real - world data.
3. **Test model robustness and generalization ability**: The generated data set is suitable for testing the robustness and generalization ability of event - based computer vision models.
### Main contributions
- **Simulator development**: Developed a simulator for generating event - response data, which can generate object event records under controlled transformation.
- **Spatio - temporal structure study**: By controlling the amount of transformation, event records of any sparsity can be generated to study the changes of objects in space - time.
- **Noise simulation**: Three types of noise (background noise, shape sampling noise, event sampling noise) are introduced to more realistically simulate the randomness in actual scenarios.
- **Application examples**: Multiple application scenarios are provided, including simulation stimulation, transformation invariance and covariance testing, etc.
### Related work
The paper also reviews existing event - based data sets and generation methods, pointing out that the current number of event - based data sets is small and the quality varies. Existing generation methods are mainly divided into two categories: recording with physical sensors and generating through simulation. The method proposed in this paper belongs to the latter, aiming to generate more controllable and high - quality event data.
In conclusion, this paper fills a major gap in event - based computer vision research by introducing the GERD simulator, providing a powerful tool for future research.