Abstract:Human activity recognition in videos is a challenging problem that has drawn a lot of interest, particularly when the goal requires the analysis of a large video database. AOLME project provides a collaborative learning environment for middle school students to explore mathematics, computer science, and engineering by processing digital images and videos. As part of this project, around 2200 hours of video data was collected for analysis. Because of the size of the dataset, it is hard to analyze all the videos of the dataset manually. Thus, there is a huge need for reliable computer-based methods that can detect activities of interest. My thesis is focused on the development of accurate methods for detecting and tracking objects in long videos. All the models are validated on videos from 7 different sessions, ranging from 45 minutes to 90 minutes. The keyboard detector achieved a very high average precision (AP) of 92% at 0.5 intersection over union (IoU). Furthermore, a combined system of the detector with a fast tracker KCF (159fps) was developed so that the algorithm runs significantly faster without sacrificing accuracy. For a video of 23 minutes having resolution 858X480 @ 30 fps, the detection alone runs at 4.7Xthe real-time, and the combined algorithm runs at 21Xthe real-time for an average IoU of 0.84 and 0.82, respectively. The hand detector achieved average precision (AP) of 72% at 0.5 IoU. The detection results were improved to 81% using optimal data augmentation parameters. The hand detector runs at 4.7Xthe real-time with AP of 81% at 0.5 IoU. The hand detection method was integrated with projections and clustering for accurate proposal generation. This approach reduced the number of false-positive hand detections by 80%. The overall hand detection system runs at 4Xthe real-time, capturing all the activity regions of the current collaborative group.

End-to-end bare-hand localization system for human–computer interaction: a comprehensive analysis and viable solution

Personalized Hand Modeling from Multiple Postures with Multi‐View Color Images

Real-Time Hand Locating By Monocular Vision

HandyNet: A One-stop Solution to Detect, Segment, Localize & Analyze Driver Hands

Self-Calibrated Multi-Sensor Wearable for Hand Tracking and Modeling

3D hand tracking for human computer interaction

Hand Tracking: Survey

Robust Hand Detection and Tracking Based on Monocular Vision

RGB2Hands: Real-Time Tracking of 3D Hand Interactions from Monocular RGB Video

3D Hand Pose Tracking and Estimation Using Stereo Matching.

HandOS: 3D Hand Reconstruction in One Stage

Long Term Object Detection and Tracking in Collaborative Learning Environments

A Deep Learning-Based End-to-End Composite System for Hand Detection and Gesture Recognition

Point Cloud Hand–Object Segmentation Using Multimodal Imaging with Thermal and Color Data for Safe Robotic Object Handover

Back to RGB: 3D tracking of hands and hand-object interactions based on short-baseline stereo

3D Hand Mesh Recovery from Monocular RGB in Camera Space

End-to-end Global to Local CNN Learning for Hand Pose Recovery in Depth Data

Locating human hands for real-time pose estimation from monocular video

Tracking and Reconstructing Hand Object Interactions from Point Cloud Sequences in the Wild.

A Framework Of Real Time Hand Gesture Vision Based Human-Computer Interaction

Improved mKLT and low layered HG-CNN based dynamic gesture recognition hardware system