Abstract:Recently, several spatial-temporal memory-based methods have verified that storing intermediate frames and their masks as memory are helpful to segment target objects in videos. However, they mainly focus on better matching between the current frame and the memory frames without explicitly paying attention to the quality of the memory. Therefore, frames with poor segmentation masks are prone to be memorized, which leads to a segmentation mask error accumulation problem and further affect the segmentation performance. In addition, the linear increase of memory frames with the growth of frame number also limits the ability of the models to handle long videos. To this end, we propose a Quality-aware Dynamic Memory Network (QDMN) to evaluate the segmentation quality of each frame, allowing the memory bank to selectively store accurately segmented frames to prevent the error accumulation problem. Then, we combine the segmentation quality with temporal consistency to dynamically update the memory bank to improve the practicability of the models. Without any bells and whistles, our QDMN achieves new state-of-the-art performance on both DAVIS and YouTube-VOS benchmarks. Moreover, extensive experiments demonstrate that the proposed Quality Assessment Module (QAM) can be applied to memory-based methods as generic plugins and significantly improves performance. Our source code is available at https://github.com/workforai/QDMN.

Ask Me Even More: Dynamic Memory Tensor Networks (Extended Model)

Long-Term Memory Networks for Question Answering

Learning Quality-aware Dynamic Memory for Video Object Segmentation

Skip-Thought Memory Networks

Memory Augmented Deep Recurrent Neural Network for Video Question Answering

Empirical Study on Deep Learning Models for Question Answering.

Motion-Appearance Co-Memory Networks for Video Question Answering

Question-Aware Memory Network for Multi-hop Question Answering in Human-Robot Interaction

Bi-Directional Gated Memory Networks For Answer Selection

Enhancing Key-Value Memory Neural Networks for Knowledge Based Question Answering.

Answering with Memory-Augmented Networks

Visual Question Answering with Memory-Augmented Networks

A Question-Focused Multi-Factor Attention Network for Question Answering

A Memory Model for Question Answering from Streaming Data Supported by Rehearsal and Anticipation of Coreference Information

Pay More Attention - Neural Architectures for Question-Answering

A Brain-inspired Memory Transformation based Differentiable Neural Computer for Reasoning-based Question Answering

Temporality-enhanced Knowledgememory Network for Factoid Question Answering

BamnetTL: Bidirectional Attention Memory Network with Transfer Learning for Question Answering Matching

Textbook Question Answering Under Instructor Guidance with Memory Networks

Stacked dynamic memory-coattention network for answering why-questions in Arabic

Working Memory Networks: Augmenting Memory Networks with a Relational Reasoning Module