Microarrays: Determining the Balance of Cellular Transcription

S. Harmer,S. Kay

DOI: https://doi.org/10.1105/tpc.12.5.613

2000-05-01

The Plant Cell

Abstract:The advent of high-throughput DNA sequencing and the ability to investigate global patterns of gene expression are revolutionizing biology, and plant biology is no exception. These new technologies allow biologists to systematically evaluate gene expression patterns on a genomewide scale. It is now

What problem does this paper attempt to address?

A Deep Awareness Framework for Pervasive Video Cloud

Weishan Zhang,Pengcheng Duan,Zhongwei Li,Qinghua Lu,Wenjuan Gong,Su Yang

DOI: https://doi.org/10.1109/access.2015.2497278

IF: 3.9

2015-01-01

IEEE Access

Abstract:Context-awareness for big data applications is different from that of traditional applications in that it is getting challenging to obtain the contexts from big data due to the complexity, velocity, variety, and other aspects of big data, especially big video data. The awareness of contexts in big data is more difficult, and should be more in-depth than that of classical applications. Therefore, in this paper, we propose an in-depth context-awareness framework for a pervasive video cloud in order to obtain underlying contexts in big video data. In this framework, we propose an approach that combines the historical view with the current view to obtain meaningful in-depth contexts, where deep learning techniques are used to obtain raw context data. We have conducted initial evaluations to show the effectiveness of the proposed approach in terms of performance and also the accuracy of obtaining the contexts. The evaluation results show that the proposed approach is effective for real-time context-awareness in a pervasive video cloud.
Self-Reported Wellness Profiles of Professional Australian Football Players During the Competition Phase of the Season

T. Gallo,Stuart J. Cormack,T. Gabbett,Christiane Lorenzen

DOI: https://doi.org/10.1519/JSC.0000000000001515

2016-05-25

Abstract:Abstract Gallo, TF, Cormack, SJ, Gabbett, TJ, and Lorenzen, CH. Self-reported wellness profiles of professional Australian football players during the competition phase of the season. J Strength Cond Res 31(2): 495–502, 2017—With the prevalence of customized self-report measures in high-performance sport, and the incomplete understanding of athletes' perceived wellness in response to matches and training load, the objective of this study was to explore weekly wellness profiles within the context of the competitive season of professional Australian football. Internal match load, measured through the session-rating of perceived exertion method, match-to-match microcycle, stage of the season, and training load were included in multivariate linear models to determine their effect on weekly wellness profile (n = 1,835). There was a lower weekly training load on a 6-day microcycle compared with a 7-day and 8-day microcycle. Match load had no significant impact on weekly wellness profile, while there was an interaction between microcycle and days postmatch. There was a likely moderately lower wellness Z-score 1 day postmatch for an 8-day microcycle (mean; 95% confidence interval: −1.79; −2.02 to −1.56) compared with a 6-day (−1.19; −1.30 to −1.08) and 7-day (−1.22; −1.34 to −1.09) cycle (d; 95% confidence interval: −0.82; −1.3 to −0.36, −0.78; −1.3 to −0.28, respectively). The second half of the season saw a possibly small reduction in overall wellness Z-score than the first half of the season (0.22; 0.12–0.32). Finally, training load had no effect on wellness Z-score when controlled for days postmatch, microcycle, and stage of the season. These results provide information on the status of players in response to matches and fixed conditions. Knowing when wellness Z-score returns to baseline relative to the length of the microcycle may lead practitioners to prescribe the heaviest load of the week accordingly. Furthermore, wellness “red flags” should be made relative to the microcycles and stage of the season to determine an athlete's status relative to their typical weekly profile.
Light-Sync: A low overhead synchronization algorithm for underwater acoustic networks

Davide Zennaro,B. Tomasi,Lorenzo Vangelista,Michele Zorzi

DOI: https://doi.org/10.1109/OCEANS-YEOSU.2012.6263491

2012-05-21

Oceans

Abstract:Clock synchronization is a fundamental requirement in a network of sensor nodes. Underwater sensor networks are a special kind of sensor networks characterized by a strong energy constraint; on the other hand, the low communication results in a less stringent constraint on the synchronization accuracy. In this paper, we propose Light-Sync, a new synchronization protocol for underwater sensor networks in which the best synchronization accuracy is sacrificed to the benefit of a lower communication overhead and finally to a lower consumed energy. We analyze the protocol performance in terms of skew and offset estimation accuracy and compare the estimation variance to the Cramer-Rao Lower Bound (CRLB). We study the energy consumption and synchronization error through an extensive simulation study in an underwater acoustic scenario. Simulation results show that Light-Sync is a suitable choice for synchronizing underwater sensors with an extremely low power consumption.
Video Action Understanding

Matthew Hutchinson,Vijay Gadepally

DOI: https://doi.org/10.1109/ACCESS.2021.3115476

2021-10-04

Abstract:Many believe that the successes of deep learning on image understanding problems can be replicated in the realm of video understanding. However, due to the scale and temporal nature of video, the span of video understanding problems and the set of proposed deep learning solutions is arguably wider and more diverse than those of their 2D image siblings. Finding, identifying, and predicting actions are a few of the most salient tasks in this emerging and rapidly evolving field. With a pedagogical emphasis, this tutorial introduces and systematizes fundamental topics, basic concepts, and notable examples in supervised video action understanding. Specifically, we clarify a taxonomy of action problems, catalog and highlight video datasets, describe common video data preparation methods, present the building blocks of state-of-the art deep learning model architectures, and formalize domain-specific metrics to baseline proposed solutions. This tutorial is intended to be accessible to a general computer science audience and assumes a conceptual understanding of supervised learning.

Computer Vision and Pattern Recognition,Machine Learning
AI-powered threat detection in surveillance systems: A real-time data processing framework

Emmanuel Cadet,Olajide Soji Osundare,Harrison Oke Ekpobimi,Zein Samira,Yodit Wondaferew Weldegeorgise

DOI: https://doi.org/10.53022/oarjet.2024.7.2.0057

2024-10-30

Open Access Research Journal of Engineering and Technology

Abstract:The increasing need for enhanced security has driven the adoption of AI-powered threat detection in surveillance systems. Traditional surveillance methods, reliant on manual monitoring, are often inefficient in detecting complex, evolving threats in real time. This review proposes a comprehensive real-time data processing framework for AI-powered threat detection in surveillance systems, designed to automate and optimize threat identification, classification, and response. The framework integrates AI algorithms, including machine learning and deep learning models, to analyze vast amounts of surveillance data from various sources such as video feeds, audio recordings, and sensor inputs. It utilizes techniques like object detection, facial recognition, and anomaly detection to identify potential threats, while leveraging stream processing frameworks (e.g., Apache Kafka, Apache Flink) to ensure low-latency, real-time analysis. Edge computing is incorporated to reduce network bottlenecks and enable faster decision-making closer to the data source. The framework also addresses the challenges of high data volume and velocity, as well as the need for scalable, flexible infrastructure. Security measures such as encryption, identity and access management (IAM), and compliance with data privacy regulations ensure that sensitive information is protected. The inclusion of continuous model training allows the system to adapt to emerging threats and reduce false positives and negatives. Case studies from urban environments, critical infrastructure, and law enforcement demonstrate the practical applications and effectiveness of this AI-driven approach. By integrating real-time data processing with advanced AI models, the framework provides a robust solution for improving the efficiency and accuracy of threat detection in modern surveillance systems. This research contributes to the growing field of AI-enhanced security, paving the way for future advancements in intelligent surveillance.
Artificial Intelligence of Things-assisted two-stream neural network for anomaly detection in surveillance Big Video Data

Waseem Ullah,Amin Ullah,Tanveer Hussain,Khan Muhammad,Ali Asghar Heidari,Javier Del Ser,Sung Wook Baik,Victor Hugo C. De Albuquerque

DOI: https://doi.org/10.1016/j.future.2021.10.033

IF: 7.307

2022-04-01

Future Generation Computer Systems

Abstract:In the last few years, visual sensors are deployed almost everywhere, generating a massive amount of surveillance video data in smart cities that can be inspected intelligently to recognize anomalous events. In this work, we present an efficient and robust framework to recognize anomalies from surveillance Big Video Data (BVD) using Artificial Intelligence of Things (AIoT). Smart surveillance is an important application of AIoT and we propose a two-stream neural network in this direction. The first stream comprises instant anomaly detection that is functional over resource-constrained IoT devices, whereas second phase is a two-stream deep neural network allowing for detailed anomaly analysis, suited to be deployed as a cloud computing service. Firstly, a self-pruned fine-tuned lightweight convolutional neural network (CNN) classifies the ongoing events as normal or anomalous in an AIoT environment. Upon anomaly detection, the edge device alerts the concerned departments and transmits the anomalous frames to cloud analysis center for their detailed evaluation in the second phase. The cloud analysis center resorts to the proposed two-stream network, modeled from the integration of spatiotemporal and optical flow features through the sequential frames. Fused features flow through a bi-directional long short-term memory (BD-LSTM) layer, which classifies them into their respective anomaly classes, e.g., assault and abuse. We perform extensive experiments over benchmarks built on top of the UCF-Crime and RWF-2000 datasets to test the effectiveness of our framework. We report a 9.88% and 4.01% increase in accuracy when compared to state-of-the-art methods evaluated over the aforementioned datasets.
Real-Time Surveillance with AI: A Comprehensive Approach to Security and Monitoring

Kishor Avhad,Sahil Kurhade,Vishal Jadhav,Prof. M. V. Raut

DOI: https://doi.org/10.48175/ijarsct-19681

2024-08-30

Abstract:The last ten years have been quite phenomenal for artificial intelligence and computer vision. For the past few years, it was unstoppable, with outstanding breakthroughs that moved towards advanced real-time surveillance systems-that wholesome approach to real-time surveillance through the power of AI in technological advancement for accuracy, scalability, and responsiveness. The system integrates facial recognition, object detection, and anomaly detection algorithms to give a very stable, adaptive response to the various facets of surveillance needs, such as public safety, institutional monitoring, and private security. Through deep learning models extracted from comprehensive datasets, the system is able to identify known individuals in real time and trace the movements and unusual behaviors of these people. A discussion of challenges in data privacy, computational efficiency, and false positives goes hand in hand with an in-depth analysis of system architecture, which includes thorough hardware and software configuration descriptions. The experimental results show the potential of the system in real-world environments as an alternate scalable solution for future security infrastructures.

Computer Science,Engineering
Video Question Answering for Surveillance

Iqbal Chowdhury,Kien Nguyen Thanh,Clinton fookes,Sridha Sridharan

DOI: https://doi.org/10.36227/techrxiv.12731945

2020-08-04

Abstract:There are many task in surveillance monitoring such as object detection, person identification, activity and action recognition etc. Integrating variety of surveillance task through a multimodal interactive system will benefit real-life deployment, and will also support human operators. We first introduce a dataset which is first of its kind and named as Surveillance Video Question Answering (SVideoQA) dataset. The multi-camera surveillance monitoring aspect is considered through the multimodal context of Video Question Answering (VideoQA) in the SVideoQA dataset. This paper proposes a deep learning model where VideoQA task on the SVideoQA dataset is attempted to solved in a manner where memory-driven relationship among appearance and motion aspect of the video features are captured. At each level of the relational reasoning respective attentive parts of the context of the motion and appearance features are identified forwarded through frame level and clip level relational reasoning module. Also, respective memories are updated which are again forwarded to the memory-relation module to finally predict the answer word. The proposed memory-driven multilevel relational reasoning is made compatible with the surveillance monitoring task through the incorporation of multi-camera relation module, which is able to capture and reason over the relationships among the video feeds across multiple cameras. Experimental outcome exhibits that the proposed memory-driven multilevel relational reasoning perform significantly better on the open-ended VideoQA task compared to other state-of-the art systems. The proposed method achieves an accuracy of 57\% and 57.6\% respectively for the single-camera and multi-camera task of the SVideoQA dataset.
VideoDL: Video-Based Digital Learning Framework Using AI Question Generation and Answer Assessment

Abdur Rahim Mohammad Forkan,Yong-Bin Kang,Prem Prakash Jayaraman,Hung Du,Steven Thomson,Elizabeth Kollias,Natalie Wieland

DOI: https://doi.org/10.3991/ijac.v16i1.35207

2023-03-13

International Journal of Advanced Corporate Learning (iJAC)

Abstract:Assessing learners’ understanding and competency in video-based digital learning is time-consuming and very difficult for educators, as it requires the generation of accurate and valid questions from pre-recorded learning videos. This paper demonstrates VideoDL, a video-based learning framework powered by Artificial Intelligence (AI) that supports automatic question generation and answer assessment from videos. VideoDL comprises of various AI algorithms, and an interactive web-based user interface (UI) developed using the principles of human-centred design. Our empirical evaluation using real-world videos from multiple domains demonstrates the effectiveness of VideoDL.
Towards Neuro-Symbolic Video Understanding

Minkyu Choi,Harsh Goel,Mohammad Omama,Yunhao Yang,Sahil Shah,Sandeep Chinchali

2024-07-16

Abstract:The unprecedented surge in video data production in recent years necessitates efficient tools to extract meaningful frames from videos for downstream tasks. Long-term temporal reasoning is a key desideratum for frame retrieval systems. While state-of-the-art foundation models, like VideoLLaMA and ViCLIP, are proficient in short-term semantic understanding, they surprisingly fail at long-term reasoning across frames. A key reason for this failure is that they intertwine per-frame perception and temporal reasoning into a single deep network. Hence, decoupling but co-designing semantic understanding and temporal reasoning is essential for efficient scene identification. We propose a system that leverages vision-language models for semantic understanding of individual frames but effectively reasons about the long-term evolution of events using state machines and temporal logic (TL) formulae that inherently capture memory. Our TL-based reasoning improves the F1 score of complex event identification by 9-15% compared to benchmarks that use GPT4 for reasoning on state-of-the-art self-driving datasets such as Waymo and NuScenes.

Computer Vision and Pattern Recognition,Artificial Intelligence
AI from concrete to abstract: demystifying artificial intelligence to the general public

Rubens Lacerda Queiroz,Fábio Ferrentini Sampaio,Cabral Lima,Priscila Machado Vieira Lima

DOI: https://doi.org/10.48550/arXiv.2006.04013

2020-06-07

Computers and Society

Abstract:Artificial Intelligence (AI) has been adopted in a wide range of domains. This shows the imperative need to develop means to endow common people with a minimum understanding of what AI means. Combining visual programming and WiSARD weightless artificial neural networks, this article presents a new methodology, AI from concrete to abstract (AIcon2abs), to enable general people (including children) to achieve this goal. The main strategy adopted by is to promote a demystification of artificial intelligence via practical activities related to the development of learning machines, as well as through the observation of their learning process. Thus, it is possible to provide subjects with skills that contributes to making them insightful actors in debates and decisions involving the adoption of artificial intelligence mechanisms. Currently, existing approaches to the teaching of basic AI concepts through programming treat machine intelligence as an external element/module. After being trained, that external module is coupled to the main application being developed by the learners. In the methodology herein presented, both training and classification tasks are blocks that compose the main program, just as the other programming constructs. As a beneficial side effect of AIcon2abs, the difference between a program capable of learning from data and a conventional computer program becomes more evident. In addition, the simplicity of the WiSARD weightless artificial neural network model enables easy visualization and understanding of training and classification tasks internal realization.
Real-Time Anomaly Detection in Video Streams

Fabien Poirier

2024-11-29

Abstract:This thesis is part of a CIFRE agreement between the company Othello and the LIASD laboratory. The objective is to develop an artificial intelligence system that can detect real-time dangers in a video stream. To achieve this, a novel approach combining temporal and spatial analysis has been proposed. Several avenues have been explored to improve anomaly detection by integrating object detection, human pose detection, and motion analysis. For result interpretability, techniques commonly used for image analysis, such as activation and saliency maps, have been extended to videos, and an original method has been proposed. The proposed architecture performs binary or multiclass classification depending on whether an alert or the cause needs to be identified. Numerous neural networkmodels have been tested, and three of them have been selected. You Only Looks Once (YOLO) has been used for spatial analysis, a Convolutional Recurrent Neuronal Network (CRNN) composed of VGG19 and a Gated Recurrent Unit (GRU) for temporal analysis, and a multi-layer perceptron for classification. These models handle different types of data and can be combined in parallel or in series. Although the parallel mode is faster, the serial mode is generally more reliable. For training these models, supervised learning was chosen, and two proprietary datasets were created. The first dataset focuses on objects that may play a potential role in anomalies, while the second consists of videos containing anomalies or non-anomalies. This approach allows for the processing of both continuous video streams and finite videos, providing greater flexibility in detection.

Computer Vision and Pattern Recognition,Machine Learning
Unveiling Anomaly : Empowering Video Surveillance through Intelligent Anomaly Detection

Dikshendra Sarpate,Isha Tadas,Radhesh Khaire,Mokshad Antapurkar,Amisha Sonone

DOI: https://doi.org/10.32628/ijsrset2411248

2024-04-19

Abstract:Video surveillance has become a cornerstone of security for public spaces and private property. However, the effectiveness of this approach is hampered by the limitations of manual monitoring. Human analysts face challenges such as fatigue, distraction, and the sheer volume of video data, leading to missed incidents and inefficient use of resources. This research project proposes a revolutionary solution: intelligent anomaly detection through artificial intelligence (AI). This system transcends the constraints of human observation by automatically identifying deviations from established patterns within video footage. The core concept lies in leveraging the power of AI to analyze various aspects of video data. This includes movement analysis, object recognition, and scene dynamics. Through this comprehensive approach, the system can detect anomalous events that might escape human notice – activities such as loitering, intrusions, or suspicious behavior. This project delves into the design and development of this intelligent anomaly detection system. It explores the vast potential of machine learning techniques, specifically focusing on unsupervised learning and deep learning algorithms. These algorithms play a crucial role in modeling normal behavior within video data. The system then utilizes these models to identify deviations that fall outside the established patterns. By flagging these anomalies, the system empowers security personnel to prioritize their attention on critical events. This significantly enhances overall security efficiency by allowing human analysts to focus on investigating the most relevant situations. This research project seeks to contribute significantly to the advancement of video surveillance technology. By harnessing the power of AI and machine learning, this intelligent anomaly detection system offers a promising approach to enhancing security in public spaces and private property.

English Else
Trends in Event Understanding and Caption Generation/Reconstruction in Dense Video: A Review

Ekanayake Mudiyanselage Chulabhaya Lankanatha Ekanayake,Abubakar Sulaiman Gezawa,Yunqi Lei

DOI: https://doi.org/10.32604/cmc.2024.046155

2024-01-01

Abstract:Video description generates natural language sentences that describe the subject, verb, and objects of the targeted Video. The video description has been used to help visually impaired people to understand the content. It is also playing an essential role in devolving human-robot interaction. The dense video description is more difficult when compared with simple Video captioning because of the object’s interactions and event overlapping. Deep learning is changing the shape of computer vision (CV) technologies and natural language processing (NLP). There are hundreds of deep learning models, datasets, and evaluations that can improve the gaps in current research. This article filled this gap by evaluating some state-of-the-art approaches, especially focusing on deep learning and machine learning for video caption in a dense environment. In this article, some classic techniques concerning the existing machine learning were reviewed. And provides deep learning models, a detail of benchmark datasets with their respective domains. This paper reviews various evaluation metrics, including Bilingual Evaluation Understudy (BLEU), Metric for Evaluation of Translation with Explicit Ordering (METEOR), Word Mover’s Distance (WMD), and Recall-Oriented Understudy for Gisting Evaluation (ROUGE) with their pros and cons. Finally, this article listed some future directions and proposed work for context enhancement using key scene extraction with object detection in a particular frame. Especially, how to improve the context of video description by analyzing key frames detection through morphological image analysis. Additionally, the paper discusses a novel approach involving sentence reconstruction and context improvement through key frame object detection, which incorporates the fusion of large language models for refining results. The ultimate results arise from enhancing the generated text of the proposed model by improving the predicted text and isolating objects using various keyframes. These keyframes identify dense events occurring in the video sequence.

computer science, information systems,materials science, multidisciplinary
From Lab to Field: Real-World Evaluation of an AI-Driven Smart Video Solution to Enhance Community Safety

Shanle Yao,Babak Rahimi Ardabili,Armin Danesh Pazho,Ghazal Alinezhad Noghre,Christopher Neff,Lauren Bourque,Hamed Tabkhi

2024-09-04

Abstract:This article adopts and evaluates an AI-enabled Smart Video Solution (SVS) designed to enhance safety in the real world. The system integrates with existing infrastructure camera networks, leveraging recent advancements in AI for easy adoption. Prioritizing privacy and ethical standards, pose based data is used for downstream AI tasks such as anomaly detection. Cloud-based infrastructure and mobile app are deployed, enabling real-time alerts within communities. The SVS employs innovative data representation and visualization techniques, such as the Occupancy Indicator, Statistical Anomaly Detection, Bird's Eye View, and Heatmaps, to understand pedestrian behaviors and enhance public safety. Evaluation of the SVS demonstrates its capacity to convert complex computer vision outputs into actionable insights for stakeholders, community partners, law enforcement, urban planners, and social scientists. This article presents a comprehensive real-world deployment and evaluation of the SVS, implemented in a community college environment across 16 cameras. The system integrates AI-driven visual processing, supported by statistical analysis, database management, cloud communication, and user notifications. Additionally, the article evaluates the end-to-end latency from the moment an AI algorithm detects anomalous behavior in real-time at the camera level to the time stakeholders receive a notification. The results demonstrate the system's robustness, effectively managing 16 CCTV cameras with a consistent throughput of 16.5 frames per second (FPS) over a 21-hour period and an average end-to-end latency of 26.76 seconds between anomaly detection and alert issuance.

Computer Vision and Pattern Recognition,Artificial Intelligence,Machine Learning
Image and Video Generation Using Artificial Intelligence and its Detection

Judin Thomas,Dhannya J,M Vishnu Ganesh,Arya J,Sreeraj S Nair

DOI: https://doi.org/10.46647/ijetms.2022.v06i05.094

2022-09-28

International Journal of Engineering Technology and Management Sciences

Abstract:AI (Artificial Intelligence) is intelligence shown by machines, rather than the natural intelligence showed by creatures including people. AI allows machines to learn from experiences and situations, adapt to new information sources and perform human-like errands. Now a days we have to make sure what we see is real or fake because development in AI and deep neural networks have prompted a rise in synthetic media, i.e., artificially generated or controlled photograph, video and audio content. Synthetic media today is exceptionally believable to such an extent that we can never again believe what we see or hear is pure and real.Among the various types of synthetic contents, the most concerning types are deepfakes and general adversarial networks (GANs). This new reality can have critical ramifications for network protection, duplicating, counterfeit news, and border security,fake alligations etc.

Computer Science
Automatic Generation of Descriptive Titles for Video Clips Using Deep Learning

Soheyla Amirian,Khaled Rasheed,Thiab R. Taha,Hamid R. Arabnia

DOI: https://doi.org/10.48550/arXiv.2104.03337

2021-04-07

Computer Vision and Pattern Recognition

Abstract:Over the last decade, the use of Deep Learning in many applications produced results that are comparable to and in some cases surpassing human expert performance. The application domains include diagnosing diseases, finance, agriculture, search engines, robot vision, and many others. In this paper, we are proposing an architecture that utilizes image/video captioning methods and Natural Language Processing systems to generate a title and a concise abstract for a video. Such a system can potentially be utilized in many application domains, including, the cinema industry, video search engines, security surveillance, video databases/warehouses, data centers, and others. The proposed system functions and operates as followed: it reads a video; representative image frames are identified and selected; the image frames are captioned; NLP is applied to all generated captions together with text summarization; and finally, a title and an abstract are generated for the video. All functions are performed automatically. Preliminary results are provided in this paper using publicly available datasets. This paper is not concerned about the efficiency of the system at the execution time. We hope to be able to address execution efficiency issues in our subsequent publications.
Intelligent Video Surveillance Systems

Aswini Reddithala,Jnaneswar Bammidi,Abhijith Pegallapati,Suresh Kumar Moru,Yogesh Parnandi,V. Chundi

DOI: https://doi.org/10.1109/ICCST49569.2021.9717400

2021-10-11

Abstract:In the past decades, there has been tremendous growth in security-related issues throughout the world. With such a multifold increase in demand for security, video surveillance has become a critical research area. In the most recent studies, there were many attempts to integrate image processing algorithms. It was also possible to achieve artificial intelligence capabilities using computer vision. Intelligent video surveillance systems can be described as a technique to assist security personnel by providing reliable real-time alerts, thus supporting the investigations with efficient video analysis. This paper discusses a comprehensive and systematic overview of the existing video surveillance techniques and possible enhancements.

Computer Science,Engineering
An Integrated Approach for Video Captioning and Applications

Soheyla Amirian,Thiab R. Taha,Khaled Rasheed,Hamid R. Arabnia

DOI: https://doi.org/10.48550/arXiv.2201.09153

2022-01-23

Abstract:Physical computing infrastructure, data gathering, and algorithms have recently had significant advances to extract information from images and videos. The growth has been especially outstanding in image captioning and video captioning. However, most of the advancements in video captioning still take place in short videos. In this research, we caption longer videos only by using the keyframes, which are a small subset of the total video frames. Instead of processing thousands of frames, only a few frames are processed depending on the number of keyframes. There is a trade-off between the computation of many frames and the speed of the captioning process. The approach in this research is to allow the user to specify the trade-off between execution time and accuracy. In addition, we argue that linking images, videos, and natural language offers many practical benefits and immediate practical applications. From the modeling perspective, instead of designing and staging explicit algorithms to process videos and generate captions in complex processing pipelines, our contribution lies in designing hybrid deep learning architectures to apply in long videos by captioning video keyframes. We consider the technology and the methodology that we have developed as steps toward the applications discussed in this research.

Computer Vision and Pattern Recognition,Artificial Intelligence
Understanding Contexts Inside Robot and Human Manipulation Tasks through a Vision-Language Model and Ontology System in a Video Stream

Chen Jiang,Masood Dehghan,Martin Jagersand

DOI: https://doi.org/10.48550/arXiv.2003.01163

2020-03-02

Computer Vision and Pattern Recognition

Abstract:Manipulation tasks in daily life, such as pouring water, unfold intentionally under specialized manipulation contexts. Being able to process contextual knowledge in these Activities of Daily Living (ADLs) over time can help us understand manipulation intentions, which are essential for an intelligent robot to transition smoothly between various manipulation actions. In this paper, to model the intended concepts of manipulation, we present a vision dataset under a strictly constrained knowledge domain for both robot and human manipulations, where manipulation concepts and relations are stored by an ontology system in a taxonomic manner. Furthermore, we propose a scheme to generate a combination of visual attentions and an evolving knowledge graph filled with commonsense knowledge. Our scheme works with real-world camera streams and fuses an attention-based Vision-Language model with the ontology system. The experimental results demonstrate that the proposed scheme can successfully represent the evolution of an intended object manipulation procedure for both robots and humans. The proposed scheme allows the robot to mimic human-like intentional behaviors by watching real-time videos. We aim to develop this scheme further for real-world robot intelligence in Human-Robot Interaction.

Microarrays: Determining the Balance of Cellular Transcription

A Deep Awareness Framework for Pervasive Video Cloud

Self-Reported Wellness Profiles of Professional Australian Football Players During the Competition Phase of the Season

Light-Sync: A low overhead synchronization algorithm for underwater acoustic networks

Video Action Understanding

AI-powered threat detection in surveillance systems: A real-time data processing framework

Artificial Intelligence of Things-assisted two-stream neural network for anomaly detection in surveillance Big Video Data

Real-Time Surveillance with AI: A Comprehensive Approach to Security and Monitoring

Video Question Answering for Surveillance

VideoDL: Video-Based Digital Learning Framework Using AI Question Generation and Answer Assessment

Towards Neuro-Symbolic Video Understanding

AI from concrete to abstract: demystifying artificial intelligence to the general public

Real-Time Anomaly Detection in Video Streams

Unveiling Anomaly : Empowering Video Surveillance through Intelligent Anomaly Detection

Trends in Event Understanding and Caption Generation/Reconstruction in Dense Video: A Review

From Lab to Field: Real-World Evaluation of an AI-Driven Smart Video Solution to Enhance Community Safety

Image and Video Generation Using Artificial Intelligence and its Detection

Automatic Generation of Descriptive Titles for Video Clips Using Deep Learning

Intelligent Video Surveillance Systems

An Integrated Approach for Video Captioning and Applications

Understanding Contexts Inside Robot and Human Manipulation Tasks through a Vision-Language Model and Ontology System in a Video Stream