Abstract:Video prediction has emerged as a critical task in the field of computer vision. While conventional deep learning-based video prediction models excel in accuracy, they often impose significant computational overhead. Moreover, they tend to disregard crucial factors like inference latency and memory consumption, which may render them unsuitable for real-time video prediction applications. This issue is further exacerbated by the necessity to deploy many of these applications on resource-limited embedded devices. In this study, we introduce an innovative faster yet accurate video prediction (FAVP) model designed to mitigate inference latency and memory footprint, all the while maintaining predictive accuracy. Our model employs an encoder-convertor-decoder architecture with a novel multi-layer convertor. Each layer within the convertor uses consistent kernel sizes to precisely capture dimensional features, while varying kernel sizes between layers effectively capture both local and global features. This combination of uniformity within layers and diversity between layers enhances the model’s ability to grasp dynamic changes. Furthermore, we demonstrate that replacing conventional large-kernel convolutions with involutions significantly trims the model’s parameter count and inference latency, without compromising prediction accuracy. Through comprehensive experiments conducted on resource-adequate x86 platforms, utilizing KTH, TrafficBJ, Human3.6M, Moving MNIST, and KTH-Enhanced datasets, we showcase our model’s exceptional prediction performance, particularly in terms of inference latency. Additionally, we evaluate the model on resource-constrained NVIDIA Jetson Nano platform using KITTI & CalTech Pedestrian dataset. The results underscore our model’s superior inference speed and its competence in meeting real-time video prediction requirements, even under stringent resource limitations.

Fast Video Facial Expression Recognition by a Deeply Tensor-Compressed LSTM Neural Network for Mobile Devices

Fast Video Facial Expression Recognition by Deeply Tensor-Compressed LSTM Neural Network on Mobile Device.

Lightweight Deep Learning Model for Facial Expression Recognition

EmotionNet Nano: An Efficient Deep Convolutional Neural Network Design for Real-time Facial Expression Recognition

MobiFace: A Lightweight Deep Learning Face Recognition on Mobile Devices

DEEPEYE: A Compact and Accurate Video Comprehension at Terminal Devices Compressed with Quantization and Tensorization

Deep action: A mobile action recognition framework using edge offloading

Video Expression Recognition Method Based on Facial Motion Unit and Temporal Attention

Lightweight Attention Convolutional Neural Network Through Network Slimming for Robust Facial Expression Recognition

Real-Time Emotion Recognition using Deep Facial Expression Analysis on Mobile Devices

AutoFace: How to Obtain Mobile Neural Network-Based Facial Feature Extractor in Less Than 10 Minutes?

SeesawFaceNets: sparse and robust face verification model for mobile platform

CNN-based Facial Affect Analysis on Mobile Devices

Deep Facial Expression Recognition Using Transfer Learning and Fine-Tuning Techniques

Real-Time Facial Affective Computing on Mobile Devices

Facial Emotion Recognition for Mobile Devices: A Practical Review

A Real-Time and Privacy-Preserving Facial Expression Recognition System Using an AI-Powered Microcontroller

Towards Faster Yet Accurate Video Prediction for Resource-Constrained Platforms

Real-Time Micro-Expression Detection In Unlabeled Long Videos Using Optical Flow And Lstm Neural Network

A Collaborative Compression Scheme for Fast Activity Recognition on Mobile Devices Via Global Compression Ratio Decision