Abstract:This paper presents the framework and results from the team “Florida International University-University of Miami (FIU-UM)” in the TRECVID 2020 Disaster Scene Description and Indexing (DSDI) task. We submitted four runs, each applying the same framework but using different score aggregation methods to rank each video shot. The score aggregation methods used in these runs are summarized as follows. • run1: the sum of the feature scores obtained from the video frames to rank the shots; • run2: the average of the feature scores obtained from the video frames to rank the shots; • run3: the maximum of the feature scores obtained from the video frames to rank the shots; • run4: the average of the top three features’ scores obtained from the video frames to rank the shots. Our framework includes the following processing steps: (1) pre-processing imagery from the provided LADI (Low Altitude Disaster Imagery) dataset; (2) generating soft labels for imagery in the LADI dataset through the fusion of annotations from both human and machine annotators as well as image time/location-based concept lookups of open datasets; (3) categorizing the frames in the LADI imagery by five Convolutional Neural Network (CNN) models (i.e., damage, environment, infrastructure, vehicles, and water), each focused on a subset of the 32 features; and (4) aggregating the predictive scores of the frame-level to the shot-level through sum, avg, max, and top. To improve the performance of the CNN models, we adopt various training strategies, including (1) the model pre-trained on ImageNet; (2) propagating the labels during training, following the sequence nature of the LADI dataset; and (3) retrieving more relevant data using an image crawler to enhance the training data. This year, the FIU-UM team achieved the first place among all the submitted runs, regardless of the training type. Among a total of four prioritized submitted runs with different relevancy sorting techniques, three of our runs ranked the top 3. The submission details are listed as follows. • Training type: LADI + Others (O) • Team ID: FIU-UM (Florida International University University of Miami) • Year: 2020

Multi-Source Weak Supervision Fusion for Disaster Scene Recognition in Videos

Deep Learning With Weak Supervision for Disaster Scene Description in Low-Altitude Imagery

Multilevel Spatial-Temporal Feature Aggregation for Video Object Detection

Multimodal and multiscale feature fusion for weakly supervised video anomaly detection

Multimodal Deep Learning Based on Multiple Correspondence Analysis for Disaster Management

Florida International University - University of Miami TRECVID 2020 DSDI Track.

Cross-Scale-Guided Fusion Transformer for Disaster Assessment Using Satellite Imagery.

Multi-task Multimodal Learning for Disaster Situation Assessment.

Disaster Image Classification by Fusing Multimodal Social Media Data

MS4D-Net: Multitask-Based Semi-Supervised Semantic Segmentation Framework with Perturbed Dual Mean Teachers for Building Damage Assessment from High-Resolution Remote Sensing Imagery

Weakly Supervised Video Salient Object Detection via Point Supervision

Weakly Supervised Video Object Segmentation via Dual-attention Cross-branch Fusion

A Method of Efficient Synthesizing Post-disaster Remote Sensing Image with Diffusion Model and LLM

Supervision by Fusion: Towards Unsupervised Learning of Deep Salient Object Detector

Multi-modal deep learning framework for damage detection in social media posts

Towards Robust Semantic Segmentation of Accident Scenes via Multi-Source Mixed Sampling and Meta-Learning

Multi-step feature fusion for natural disaster damage assessment on satellite images

Semi-Supervised Building Extraction with Optical Flow Correction Based on Satellite Video Data in a Tsunami-Induced Disaster Scene

Cross-directional Feature Fusion Network for Building Damage Assessment from Satellite Imagery

Disaster Anomaly Detector via Deeper FCDDs for Explainable Initial Responses

Generalizable Disaster Damage Assessment via Change Detection with Vision Foundation Model