Abstract:Occlusions and complex backgrounds are common factors that hinder many computer vision applications. In a street scene, the challenge of accurately predicting pedestrian trajectories comes from the complexity of human behavior and the diversity of the external environment. It is difficult, if not impossible, to extract relevant information to accurately predict pedestrian trajectories in dynamic scenes. Synthetic aperture imaging (SAI) uses an array of cameras to mimic a camera with a large virtual convex lens by projecting images of a scene from different views onto a virtual focal plane. It is commonly used to reconstruct occluded objects, and in a street scene, can provide observation of pedestrians occluded by other objects and pedestrians. In this paper, we propose a joint prediction method based on autofocusing of SAI to predict pedestrian trajectories in dynamic scenes. The main contributions of this paper include: 1) The task of pedestrian trajectory prediction in dynamic scenarios is redefined as pedestrian trajectory prediction and SAI autofocusing from a practical but more challenging perspective. 2) The proposed method is based on an existing SAI-based method to extract information in heavily occluded views, which can obtain more accurate results but with less computational cost and without using other sensors such as LiDAR or depth cameras. 3) A new pedestrian trajectory prediction model, an attention-based trajectory prediction variational autoencoder (ATP-VAE), is proposed to extract complex human behavior and social interactions in dynamic scenes through a new Intention Attention Unit. The experimental results on multiple public datasets show that the proposed method achieves state-of-the-art results in the first-person perspective and in aerial view.

Synthetic Aperture Imaging with Events and Frames

Event-Based Fusion for Motion Deblurring with Cross-modal Attention

Image De-occlusion via Event-enhanced Multi-modal Fusion Hybrid Network

EISNet: A Multi-Modal Fusion Network for Semantic Segmentation with Events and Images

Event-Based Low-Illumination Image Enhancement

EVI-SAM: Robust, Real-time, Tightly-coupled Event-Visual-Inertial State Estimation and 3D Dense Mapping

LaSe-E2V: Towards Language-guided Semantic-Aware Event-to-Video Reconstruction

Asymmetric Event-Guided Video Super-Resolution

Efficient Light Field Angular Super-Resolution with Sub-Aperture Feature Learning and Macro-Pixel Upsampling.

Event Enhanced High-Quality Image Recovery

Multi‐Event‐Camera Depth Estimation and Outlier Rejection by Refocused Events Fusion

Learning to Super-Resolve Blurry Images with Events

Towards Robust Event-guided Low-Light Image Enhancement: A Large-Scale Real-World Event-Image Dataset and Novel Approach

Autofocusing for Synthetic Aperture Imaging Based on Pedestrian Trajectory Prediction

E-detector: Asynchronous Spatio-temporal for Event-based Object Detection in Intelligent Transportation System

E2VIDX: improved bridge between conventional vision and bionic vision

Event-Based Vision Enhanced: A Joint Detection Framework in Autonomous Driving

Coherent Event Guided Low-Light Video Enhancement

Passive Non-line-of-sight Imaging for Moving Targets with an Event Camera

Event-guided Low-light Video Semantic Segmentation

Event-enhanced Snapshot Mosaic Hyperspectral Frame Deblurring