Abstract:Video captioning is an automated collection of natural language phrases that explains the contents in video frames. Because of the incomparable performance of deep learning in the field of computer vision and natural language processing in recent years, research in this field has been exponentially increased throughout past decades. Numerous approaches, datasets, and measurement metrics have been introduced in the literature, calling for a systematic survey to guide research efforts in this exciting new direction. Through the statistical analysis, this survey paper focuses mostly on state-of-the-art approaches, emphasizing deep learning models, assessing benchmark datasets in several parameters, and classifying the pros and cons of the various evaluation metrics based on the previous works in the deep learning field. This survey shows the most used variants of neural networks for visual and spatio-temporal feature extraction as well as language generation model. The results show that ResNet and VGG as visual feature extractor and 3D convolutional neural network as spatio-temporal feature extractor are mostly used. Besides that, Long Short Term Memory (LSTM) has been mainly used as the language model. However, nowadays, the Gated Recurrent Unit (GRU) and Transformer are slowly replacing LSTM. Regarding dataset usage, so far, MSVD and MSR-VTT are very much dominant due to be part of outstanding results among various captioning models. From 2015 to 2020, with all major datasets, some models such as, Inception-Resnet-v2 + C3D + LSTM, ResNet-101 + I3D + Transformer, ResNet-152 + ResNext-101 (R3D) + (LSTM, GAN) have achieved by far best results in video captioning. Despite rapid advancement, our survey reveals that video captioning research-work still has a lot to develop in accessing the full potential of deep learning for classifying and captioning a large number of activities, as well as creating large datasets covering diversified training video samples.

A Comprehensive Survey of Deep Learning for Image Captioning

Deep Learning Approaches on Image Captioning: A Review

Deep Learning Approaches for Image Captioning: Opportunities, Challenges and Future Potential

A comprehensive survey on image captioning: from handcrafted to deep learning-based techniques, a taxonomy and open research issues

Exploring Video Captioning Techniques: A Comprehensive Survey on Deep Learning Methods

Deep learning and knowledge graph for image/video captioning: A review of datasets, evaluation metrics, and methods

A Survey on Recent Advances in Image Captioning

Features to Text: A Comprehensive Survey of Deep Learning on Semantic Segmentation and Image Captioning.

A comprehensive review of image caption generation

Image captioning using deep learning and python

A Thorough Review on Recent Deep Learning Methodologies for Image Captioning

Deep Learning for Video Captioning: A Review

Visuals to Text: A Comprehensive Review on Automatic Image Captioning

A comprehensive literature review on image captioning methods and metrics based on deep learning technique

Supervised Deep Learning Techniques for Image Description: A Systematic Review

Domain-specific image captioning: a comprehensive review

Image Captioning using Deep Neural Architectures

Advancements in Deep Learning-Based Image Captioning

Deep Learning for Image-to-Text Generation: A Technical Overview

A Review of Deep Learning for Video Captioning

From Show to Tell: A Survey on Deep Learning-based Image Captioning