Abstract:Video captioning is a challenging task as it needs to accurately transform visual understanding into natural language description. To date, state-of-the-art methods inadequately model global-local vision representation for sentence generation, leaving plenty of room for improvement. In this work, we approach the video captioning task from a new perspective and propose a GLR framework, namely a global-local representation granularity. Our GLR demonstrates three advantages over the prior efforts. First, we propose a simple solution, which exploits extensive vision representations from different video ranges to improve linguistic expression. Second, we devise a novel global-local encoder, which encodes different video representations including long-range, short-range and local-keyframe, to produce rich semantic vocabulary for obtaining a descriptive granularity of video contents across frames. Finally, we introduce the progressive training strategy which can effectively organize feature learning to incur optimal captioning behavior. Evaluated on the MSR-VTT and MSVD dataset, we outperform recent state-of-the-art methods including a well-tuned SA-LSTM baseline by a significant margin, with shorter training schedules. Because of its simplicity and efficacy, we hope that our GLR could serve as a strong baseline for many video understanding tasks besides video captioning. Code will be available.

Local Translation Prediction with Global Sentence Representation

Translation Prediction with Source Dependency-Based Context Representation.

Learning Contextualized Sentence Representations for Document-Level Neural Machine Translation

Local System Voting Feature for Machine Translation System Combination

Syntax-Based Context Representation for Statistical Machine Translation.

Modeling Bilingual Conversational Characteristics for Neural Chat Translation

Confidence Based Bidirectional Global Context Aware Training Framework for Neural Machine Translation

Learning Bilingual Sentence Representations for Quality Estimation of Machine Translation.

Neural Machine Translation by Jointly Learning to Align and Translate

Video Captioning Using Global-Local Representation

One Type Context is Not Enough: Global Context-Aware Neural Machine Translation

Local-to-global GCN with knowledge-aware representation for distantly supervised relation extraction

Non-Autoregressive Translation with Layer-Wise Prediction and Deep Supervision

Local Byte Fusion for Neural Machine Translation

Towards Making the Most of Context in Neural Machine Translation

Context-Dependent Translation Selection Using Convolutional Neural Network

Neural Machine Translation with Dynamic Graph Convolutional Decoder

Phrase-Level Global-Local Hybrid Model for Sentence Embedding

C L ] 1 0 Ju n 20 18 Deconvolution-Based Global Decoding for Neural Machine Translation

Sentence Level Human Translation Quality Estimation with Attention-based Neural Networks

Interpretable Convolutional Neural Networks for Effective Translation Initiation Site Prediction