Human locomotion in space analyzed biomechanically through a multi-link chain model.

S. Aleshinsky,V. Zatsiorsky

DOI: https://doi.org/10.1016/0021-9290(78)90002-7

IF: 2.4

Journal of Biomechanics

Abstract:

What problem does this paper attempt to address?

Towards Human-like Perception: Learning Structural Causal Model in Heterogeneous Graph

Tianqianjin Lin,Kaisong Song,Zhuoren Jiang,Yangyang Kang,Weikang Yuan,Xurui Li,Changlong Sun,Cui Huang,Xiaozhong Liu

DOI: https://doi.org/10.1016/j.ipm.2023.103600

IF: 7.466

2024-01-01

Information Processing & Management

Abstract:Heterogeneous graph neural networks have become popular in various domains. However, their generalizability and interpretability are limited due to the discrepancy between their inherent inference flows and human reasoning logic or underlying causal relationships for the learning problem. This study introduces a novel solution, HG-SCM (Heterogeneous Graph as Structural Causal Model). It can mimic the human perception and decision process through two key steps: constructing intelligible variables based on semantics derived from the graph schema and automatically learning task-level causal relationships among these variables by incorporating advanced causal discovery techniques. We compared HG-SCM to seven state-of-the-art baseline models on three real-world datasets, under three distinct and ubiquitous out-of-distribution settings. HG-SCM achieved the highest average performance rank with minimal standard deviation, substantiating its effectiveness and superiority in terms of both predictive power and generalizability. Additionally, the visualization and analysis of the auto-learned causal diagrams for the three tasks aligned well with domain knowledge and human cognition, demonstrating prominent interpretability. HG-SCM's human-like nature and its enhanced generalizability and interpretability make it a promising solution for special scenarios where transparency and trustworthiness are paramount.
Predictive event segmentation and representation with neural networks: A self-supervised model assessed by psychological experiments

Hamit Basgol,Inci Ayhan,Emre Ugur

DOI: https://doi.org/10.1016/j.cogsys.2023.101167

IF: 4.541

2023-09-01

Cognitive Systems Research

Abstract:People segment complex, ever-changing, and continuous experience into basic, stable, and discrete spatio-temporal experience units, called events. The literature on event segmentation investigates the mechanisms behind this ability. Event segmentation theory points out that people predict ongoing activities and observe prediction error signals to find event boundaries. In this study, we investigated the mechanism giving rise to this ability through a computational model and accompanying psychological experiments. Inspired by event segmentation theory and predictive processing, we introduced a self-supervised model of event segmentation. This model consists of neural networks that predict the sensory signal in the next time-step to represent different events, and a cognitive model that regulates these networks on the basis of their prediction errors. In order to verify the ability of our model in segmenting events, learning them during passive observation, and representing them in its representational space, we prepared a video of human behaviors represented by point-light displays. We compared the event segmentation behaviors of participants and our model with this video in two granularities. Using point-biserial correlation, we demonstrated that the event boundaries of our model correlated with the responses of the participants. Moreover, by approximating the representation space of participants, we showed that our model formed a similar representation space with those of participants. The result suggests that our model that tracks the prediction error signals can produce human-like event boundaries and event representations. Finally, we discuss our contribution to the literature and our understanding of how event segmentation is implemented in the brain.

psychology, experimental,neurosciences,computer science, artificial intelligence
Recognizing Cognitive Load by a Hybrid Spatio-Temporal Causal Model from Multivariate Physiological Data

Zirui Yong,Guoxin Su,Xiaohu Li,Lingyun Sun,Zejian Li,Li Liu

DOI: https://doi.org/10.1007/978-3-031-26422-1_20

2023-01-01

Abstract:Cognitive load recognition is challenging due to the inherent diversity and causality of multivariate physiological changes, with each of its instances having its own style of physiological events and their spatio-temporal causal dependencies. This leads us to define a hybrid model that employs Granger causality (GC) and Gramian angular difference fields (GADF) to discover diverse varieties of multivariate physiological events. In particular, our model introduces a GC network to explicitly characterize the unique temporal causal configurations of a particular cognitive state as a variable number of nodes and links. In addition, GADF maps are constructed to capture the inherit spatio-temporal dependency among multivariate signals in a 2D structural space. A capsule network is designed to merge these two heterogenous types of features together in a uniform way, and as a result, all local causal and spatio-temporal dependencies are globally consistent. Empirical evaluations on one benchmark dataset and two in-house datasets collected by ourselves in virtual reality learning environment suggest our model significantly outperforms the state-of-the-art approaches.
Incidence of cancer in Basrah: results of a household survey.

Riyadh Abdul-Ameer Hussain,O. Habib

DOI: https://doi.org/10.7314/APJCP.2015.16.1.163

Asian Pacific Journal of Cancer Prevention

Abstract:BACKGROUND Cancer is a major health problem at global level. It is increasingly registered in Iraq and Basrah but the epidemiological situation, though becoming better documented, is still questionable regarding the adequacy of data. OBJECTIVE The study aimed to measure the incidence of cancer in Basrah. MATERIALS AND METHODS The results presented in this paper are part of a large household survey carried out in Basrah governorate-southern Iraq over a 12 month period (January to December 2013). It involved a detailed interview with adult respondents from each and every household enrolled in the study during a three-year recall period about the incidence of cancer. A total of 6,999 households were covered yielding 40,684 persons. RESULTS The total number of new cancer cases reported over the three- year recall period (2010-2012) was 112. The average annual incidence rate of all cancers was 91.8 per 100,000 population with a higher rate for females (109.7) compared to males (74.3) The overall age standardized rate was 150.7 per 100,000. The highest incidence rate was recorded for the Southern part of the governorate (Abul-Khasib and Fao ) at 138.8 per 100,000 and the lowest was for East of Basrah (Shatt-Arab District) at 78.0 per 100,000. With respect to cancer types, the main cancers were those of breast, lung, larynx-pharynx, leukaemia, colon-rectum and urinary bladder. These six cancers accounted for 51.5% of all reported cases. Other important cancers were those of brain, bones, pancreas and liver, accounting for a further 17.9%. CONCLUSIONS The pattern of cancer in Basrah is generally similar to the pattern at the national level in terms of age, sex and topography but the incidence rate according to the present household survey is higher than any previously reported figures. Household surveys for cancer seem feasible albeit difficult and costly.
Image segmentation with cyclic load balanced parallel Fuzzy C - Means cluster analysis

Mogana Vadiveloo,R. Abdullah,M. Rajeswari,Ahmad Adel Abu-Shareha

DOI: https://doi.org/10.1109/IST.2011.5962212

2011-05-17

Abstract:This paper proposes a cyclic load balancing strategy to parallel Fuzzy C-Means cluster analysis algorithm. The problem is to minimize the total time cost and maximize the parallel processing efficiency when a subset of clusters is distributed over a set of processors cores on shared memory architecture. The parallel Fuzzy C - Means (FCM) cluster analysis algorithm is composed by two distinct parts. The two distinct parts are; first: the cluster analysis whereby using the FCM algorithm to calculate the cluster centers and second: the evaluation of the subset of clusters whereby using the cluster validity functions to produce the result of the optimal cluster. The experimental result shows that the cyclic load balanced parallel FCM cluster analysis algorithm presents good speed up especially when the size of clusters is large as compared to the parallel FCM cluster analysis algorithm.
Neural Foundations of Mental Simulation: Future Prediction of Latent Representations on Dynamic Scenes

Aran Nayebi,Rishi Rajalingham,Mehrdad Jazayeri,Guangyu Robert Yang

2023-10-25

Abstract:Humans and animals have a rich and flexible understanding of the physical world, which enables them to infer the underlying dynamical trajectories of objects and events, plausible future states, and use that to plan and anticipate the consequences of actions. However, the neural mechanisms underlying these computations are unclear. We combine a goal-driven modeling approach with dense neurophysiological data and high-throughput human behavioral readouts to directly impinge on this question. Specifically, we construct and evaluate several classes of sensory-cognitive networks to predict the future state of rich, ethologically-relevant environments, ranging from self-supervised end-to-end models with pixel-wise or object-centric objectives, to models that future predict in the latent space of purely static image-based or dynamic video-based pretrained foundation models. We find strong differentiation across these model classes in their ability to predict neural and behavioral data both within and across diverse environments. In particular, we find that neural responses are currently best predicted by models trained to predict the future state of their environment in the latent space of pretrained foundation models optimized for dynamic scenes in a self-supervised manner. Notably, models that future predict in the latent space of video foundation models that are optimized to support a diverse range of sensorimotor tasks, reasonably match both human behavioral error patterns and neural dynamics across all environmental scenarios that we were able to test. Overall, these findings suggest that the neural mechanisms and behaviors of primate mental simulation are thus far most consistent with being optimized to future predict on dynamic, reusable visual representations that are useful for Embodied AI more generally.

Artificial Intelligence,Computer Vision and Pattern Recognition,Robotics,Neurons and Cognition
Architecture of a Cortex Inspired Hierarchical Event Recaller

Valentin Puente Varona

2024-05-03

Abstract:This paper proposes a new approach to Machine Learning (ML) that focuses on unsupervised continuous context-dependent learning of complex patterns. Although the proposal is partly inspired by some of the current knowledge about the structural and functional properties of the mammalian brain, we do not claim that biological systems work in an analogous way (nor the opposite). Based on some properties of the cerebellar cortex and adjacent structures, a proposal suitable for practical problems is presented. A synthetic structure capable of identifying and predicting complex temporal series will be defined and experimentally tested. The system relies heavily on prediction to help identify and learn patterns based on previously acquired contextual knowledge. As a proof of concept, the proposed system is shown to be able to learn, identify and predict a remarkably complex temporal series such as human speech, with no prior knowledge. From raw data, without any adaptation in the core algorithm, the system is able to identify certain speech structures from a set of Spanish sentences. Unlike conventional ML, the proposal can learn with a reduced training set. Although the idea can be applied to a constrained problem, such as the detection of unknown vocabulary in a speech, it could be used in more applications, such as vision, or (by incorporating the missing biological periphery) fit into other ML techniques. Given the trivial computational primitives used, a potential hardware implementation will be remarkably frugal. Coincidentally, the proposed model not only conforms to a plausible functional framework for biological systems but may also explain many elusive cognitive phenomena.

Neural and Evolutionary Computing,Artificial Intelligence,Hardware Architecture,Machine Learning
Using Features at Multiple Temporal and Spatial Resolutions to Predict Human Behavior in Real Time

Liang Zhang,Justin Lieffers,Adarsh Pyarelal

DOI: https://doi.org/10.48550/arXiv.2211.06721

IF: 5.414

2022-11-12

Machine Learning

Abstract:When performing complex tasks, humans naturally reason at multiple temporal and spatial resolutions simultaneously. We contend that for an artificially intelligent agent to effectively model human teammates, i.e., demonstrate computational theory of mind (ToM), it should do the same. In this paper, we present an approach for integrating high and low-resolution spatial and temporal information to predict human behavior in real time and evaluate it on data collected from human subjects performing simulated urban search and rescue (USAR) missions in a Minecraft-based environment. Our model composes neural networks for high and low-resolution feature extraction with a neural network for behavior prediction, with all three networks trained simultaneously. The high-resolution extractor encodes dynamically changing goals robustly by taking as input the Manhattan distance difference between the humans' Minecraft avatars and candidate goals in the environment for the latest few actions, computed from a high-resolution gridworld representation. In contrast, the low-resolution extractor encodes participants' historical behavior using a historical state matrix computed from a low-resolution graph representation. Through supervised learning, our model acquires a robust prior for human behavior prediction, and can effectively deal with long-term observations. Our experimental results demonstrate that our method significantly improves prediction accuracy compared to approaches that only use high-resolution information.
Generative Modeling of Multimodal Multi-Human Behavior

Boris Ivanovic,Edward Schmerling,Karen Leung,Marco Pavone

DOI: https://doi.org/10.48550/arXiv.1803.02015

2018-07-26

Abstract:This work presents a methodology for modeling and predicting human behavior in settings with N humans interacting in highly multimodal scenarios (i.e. where there are many possible highly-distinct futures). A motivating example includes robots interacting with humans in crowded environments, such as self-driving cars operating alongside human-driven vehicles or human-robot collaborative bin packing in a warehouse. Our approach to model human behavior in such uncertain environments is to model humans in the scene as nodes in a graphical model, with edges encoding relationships between them. For each human, we learn a multimodal probability distribution over future actions from a dataset of multi-human interactions. Learning such distributions is made possible by recent advances in the theory of conditional variational autoencoders and deep learning approximations of probabilistic graphical models. Specifically, we learn action distributions conditioned on interaction history, neighboring human behavior, and candidate future agent behavior in order to take into account response dynamics. We demonstrate the performance of such a modeling approach in modeling basketball player trajectories, a highly multimodal, multi-human scenario which serves as a proxy for many robotic applications.

Robotics,Human-Computer Interaction
A model of cortical cognitive function using hierarchical interactions of gating matrices in internal agents coding relational representations

Michael E. Hasselmo

DOI: https://doi.org/10.48550/arXiv.1809.08203

2018-10-16

Abstract:Flexible cognition requires the ability to rapidly detect systematic functions of variables and guide future behavior based on predictions. The model described here proposes a potential framework for patterns of neural activity to detect systematic functions and relations between components of sensory input and apply them in a predictive manner. This model includes multiple internal gating agents that operate within the state space of neural activity, in analogy to external agents behaving in the external environment. The multiple internal gating agents represent patterns of neural activity that detect and gate patterns of matrix connectivity representing the relations between different neural populations. The patterns of gating matrix connectivity represent functions that can be used to predict future components of a series of sensory inputs or the relationship between different features of a static sensory stimulus. The model is applied to the prediction of dynamical trajectories, the internal relationship between features of different sensory stimuli and to the prediction of affine transformations that could be useful for solving cognitive tasks such as the Ravens progressive matrices task.

Neurons and Cognition
Modeling Dynamic Environments with Scene Graph Memory

Andrey Kurenkov,Michael Lingelbach,Tanmay Agarwal,Emily Jin,Chengshu Li,Ruohan Zhang,Li Fei-Fei,Jiajun Wu,Silvio Savarese,Roberto Martín-Martín

2023-06-13

Abstract:Embodied AI agents that search for objects in large environments such as households often need to make efficient decisions by predicting object locations based on partial information. We pose this as a new type of link prediction problem: link prediction on partially observable dynamic graphs. Our graph is a representation of a scene in which rooms and objects are nodes, and their relationships are encoded in the edges; only parts of the changing graph are known to the agent at each timestep. This partial observability poses a challenge to existing link prediction approaches, which we address. We propose a novel state representation -- Scene Graph Memory (SGM) -- with captures the agent's accumulated set of observations, as well as a neural net architecture called a Node Edge Predictor (NEP) that extracts information from the SGM to search efficiently. We evaluate our method in the Dynamic House Simulator, a new benchmark that creates diverse dynamic graphs following the semantic patterns typically seen at homes, and show that NEP can be trained to predict the locations of objects in a variety of environments with diverse object movement dynamics, outperforming baselines both in terms of new scene adaptability and overall accuracy. The codebase and more can be found at <a class="link-external link-https" href="https://www.scenegraphmemory.com" rel="external noopener nofollow">this https URL</a>.

Machine Learning,Artificial Intelligence
Understanding Human Cognition Through Computational Modeling

Janet Hui‐wen Hsiao

DOI: https://doi.org/10.1111/tops.12737

IF: 3.265

2024-05-25

Topics in Cognitive Science

Abstract:One important goal of cognitive science is to understand the mind in terms of its representational and computational capacities, where computational modeling plays an essential role in providing theoretical explanations and predictions of human behavior and mental phenomena. In my research, I have been using computational modeling, together with behavioral experiments and cognitive neuroscience methods, to investigate the information processing mechanisms underlying learning and visual cognition in terms of perceptual representation and attention strategy. In perceptual representation, I have used neural network models to understand how the split architecture in the human visual system influences visual cognition, and to examine perceptual representation development as the results of expertise. In attention strategy, I have developed the Eye Movement analysis with Hidden Markov Models method for quantifying eye movement pattern and consistency using both spatial and temporal information, which has led to novel findings across disciplines not discoverable using traditional methods. By integrating it with deep neural networks (DNN), I have developed DNN+HMM to account for eye movement strategy learning in human visual cognition. The understanding of the human mind through computational modeling also facilitates research on artificial intelligence's (AI) comparability with human cognition, which can in turn help explainable AI systems infer humans' belief on AI's operations and provide human‐centered explanations to enhance human−AI interaction and mutual understanding. Together, these demonstrate the essential role of computational modeling methods in providing theoretical accounts of the human mind as well as its interaction with its environment and AI systems.

psychology, experimental
Learning to segment self-generated from externally caused optic flow through sensorimotor mismatch circuits

Matthias Brucklacher,Giovanni Pezzulo,Francesco Mannella,Gaspare Galati,Cyriel M. A. Pennartz

DOI: https://doi.org/10.1101/2023.11.15.567170

2024-02-15

Abstract:Efficient sensory detection requires the capacity to ignore task-irrelevant information, for example when optic flow patterns created by egomotion need to be disentangled from object perception. To investigate how this is achieved in the visual system, predictive coding with sensorimotor mismatch detection is an attractive starting point. Indeed, experimental evidence for sensorimotor mismatch signals in early visual areas exists, but it is not understood how they are integrated into cortical networks that perform input segmentation and categorization. Our model advances a biologically plausible solution by extending predictive coding models with the ability to distinguish self-generated from externally caused optic flow. We first show that a simple three neuron circuit produces experience-dependent sensorimotor mismatch responses, in agreement with calcium imaging data from mice. This microcircuit is then integrated into a neural network with two generative streams. The motor-to-visual stream consists of parallel microcircuits between motor and visual areas and learns to spatially predict optic flow resulting from self-motion. The second stream bidirectionally connects a motion-selective higher visual area (mHVA) to V1, assigning a crucial role to the abundant feedback connections: the maintenance of a generative model of externally caused optic flow. In the model, area mHVA learns to segment moving objects from the background, and facilitates object categorization. Based on shared neurocomputational principles across species, the model also maps onto primate vision. Our work extends the Hebbian predictive coding to sensorimotor settings, in which the agent actively moves - and learns to predict the consequences of its own movements.

Neuroscience
An Empirical Evaluation of Neural and Neuro-symbolic Approaches to Real-time Multimodal Complex Event Detection

Liying Han,Mani B. Srivastava

2024-03-04

Abstract:Robots and autonomous systems require an understanding of complex events (CEs) from sensor data to interact with their environments and humans effectively. Traditional end-to-end neural architectures, despite processing sensor data efficiently, struggle with long-duration events due to limited context sizes and reasoning capabilities. Recent advances in neuro-symbolic methods, which integrate neural and symbolic models leveraging human knowledge, promise improved performance with less data. This study addresses the gap in understanding these approaches' effectiveness in complex event detection (CED), especially in temporal reasoning. We investigate neural and neuro-symbolic architectures' performance in a multimodal CED task, analyzing IMU and acoustic data streams to recognize CE patterns. Our methodology includes (i) end-to-end neural architectures for direct CE detection from sensor embeddings, (ii) two-stage concept-based neural models mapping sensor embeddings to atomic events (AEs) before CE detection, and (iii) a neuro-symbolic approach using a symbolic finite-state machine for CE detection from AEs. Empirically, the neuro-symbolic architecture significantly surpasses purely neural models, demonstrating superior performance in CE recognition, even with extensive training data and ample temporal context for neural approaches.

Artificial Intelligence
Exploring the Chameleon Effect of Contextual Dynamics in Temporal Knowledge Graph for Event Prediction

Xin Liu,Yi He,Wenxin Tai,Xovee Xu,Fan Zhou,Guangchun Luo

DOI: https://doi.org/10.26599/tst.2024.9010067

2024-09-14

Tsinghua Science & Technology

Abstract:The ability to forecast future events brings great benefits for society and cyberspace in many public safety domains, such as civil unrest, pandemics and crimes. The occurrences of new events are often correlated or dependent on historical and concurrent events. Many existing studies learn event-occurring processes with sequential and structural models, which, however, suffer from inefficient and inaccurate prediction problems. To better understand the event forecasting task and characterize the occurrence of new events, we exploit the human cognitive theory from the cognitive neuroscience discipline to find available cues for algorithm design and event prediction. Motivated by the dual process theory, we propose a two-stage learning scheme for event knowledge mining and prediction. First, we screen out event candidates based on historical inherent knowledge. Then we re-rank event candidates by probing into the newest relative events. Our proposed model mimics a sociological phenomenon called "the chameleon effect" and consists of a new target attentive graph collaborative learning mechanism to ensure a better understanding of sophisticated evolution patterns associated with events. In addition, self-supervised contrastive learning is employed to alleviate the over-smoothing problem that existed in graph learning while improving the model's interpretability. Experiments show the effectiveness of our approach.

computer science, information systems,engineering, electrical & electronic, software engineering
Designing Technology for Older People - The Role of Technical Self-confidence in Usability of an Inclusive Heating Control

N. Combe,D. Harrison,Hua Dong

DOI: https://doi.org/10.1007/978-3-642-39238-2_6

2013-07-21

Interacción

Abstract:
Learning, planning, and control in a monolithic neural event inference architecture

Martin V Butz,David Bilkey,Dania Humaidan,Alistair Knott,Sebastian Otte,Martin V. Butz

DOI: https://doi.org/10.1016/j.neunet.2019.05.001

IF: 7.8

2019-09-01

Neural Networks

Abstract:We introduce REPRISE, a REtrospective and PRospective Inference SchEme, which learns temporal event-predictive models of dynamical systems. REPRISE infers the unobservable contextual event state and accompanying temporal predictive models that best explain the recently encountered sensorimotor experiences retrospectively. Meanwhile, it optimizes upcoming motor activities prospectively in a goal-directed manner. Here, REPRISE is implemented by a recurrent neural network (RNN), which learns temporal forward models of the sensorimotor contingencies generated by different simulated dynamic vehicles. The RNN is augmented with contextual neurons, which enable the encoding of distinct, but related, sensorimotor dynamics as compact event codes. We show that REPRISE concurrently learns to separate and approximate the encountered sensorimotor dynamics: it analyzes sensorimotor error signals adapting both internal contextual neural activities and connection weight values. Moreover, we show that REPRISE can exploit the learned model to induce goal-directed, model-predictive control, that is, approximate active inference: Given a goal state, the system imagines a motor command sequence optimizing it with the prospective objective to minimize the distance to the goal. The RNN activities thus continuously imagine the upcoming future and reflect on the recent past, optimizing the predictive model, the hidden neural state activities, and the upcoming motor activities. As a result, event-predictive neural encodings develop, which allow the invocation of highly effective and adaptive goal-directed sensorimotor control.

computer science, artificial intelligence,neurosciences
Episodic Memory for Learning Subjective-Timescale Models

Alexey Zakharov,Matthew Crosby,Zafeirios Fountas

DOI: https://doi.org/10.48550/arXiv.2010.01430

2020-10-04

Abstract:In model-based learning, an agent's model is commonly defined over transitions between consecutive states of an environment even though planning often requires reasoning over multi-step timescales, with intermediate states either unnecessary, or worse, accumulating prediction error. In contrast, intelligent behaviour in biological organisms is characterised by the ability to plan over varying temporal scales depending on the context. Inspired by the recent works on human time perception, we devise a novel approach to learning a transition dynamics model, based on the sequences of episodic memories that define the agent's subjective timescale - over which it learns world dynamics and over which future planning is performed. We implement this in the framework of active inference and demonstrate that the resulting subjective-timescale model (STM) can systematically vary the temporal extent of its predictions while preserving the same computational efficiency. Additionally, we show that STM predictions are more likely to introduce future salient events (for example new objects coming into view), incentivising exploration of new areas of the environment. As a result, STM produces more informative action-conditioned roll-outs that assist the agent in making better decisions. We validate significant improvement in our STM agent's performance in the Animal-AI environment against a baseline system, trained using the environment's objective-timescale dynamics.

Machine Learning,Artificial Intelligence,Neurons and Cognition
Human Activity Learning and Segmentation using Partially Hidden Discriminative Models

Truyen Tran,Hung Bui,Svetha Venkatesh

DOI: https://doi.org/10.48550/arXiv.1408.3081

2014-08-06

Abstract:Learning and understanding the typical patterns in the daily activities and routines of people from low-level sensory data is an important problem in many application domains such as building smart environments, or providing intelligent assistance. Traditional approaches to this problem typically rely on supervised learning and generative models such as the hidden Markov models and its extensions. While activity data can be readily acquired from pervasive sensors, e.g. in smart environments, providing manual labels to support supervised training is often extremely expensive. In this paper, we propose a new approach based on semi-supervised training of partially hidden discriminative models such as the conditional random field (CRF) and the maximum entropy Markov model (MEMM). We show that these models allow us to incorporate both labeled and unlabeled data for learning, and at the same time, provide us with the flexibility and accuracy of the discriminative framework. Our experimental results in the video surveillance domain illustrate that these models can perform better than their generative counterpart, the partially hidden Markov model, even when a substantial amount of labels are unavailable.

Machine Learning,Computer Vision and Pattern Recognition
A Probabilistic Semi-Supervised Approach to Multi-Task Human Activity Modeling

Judith Bütepage,Hedvig Kjellström,Danica Kragic

DOI: https://doi.org/10.48550/arXiv.1809.08875

2019-03-14

Abstract:Human behavior is a continuous stochastic spatio-temporal process which is governed by semantic actions and affordances as well as latent factors. Therefore, video-based human activity modeling is concerned with a number of tasks such as inferring current and future semantic labels, predicting future continuous observations as well as imagining possible future label and feature sequences. In this paper we present a semi-supervised probabilistic deep latent variable model that can represent both discrete labels and continuous observations as well as latent dynamics over time. This allows the model to solve several tasks at once without explicit fine-tuning. We focus here on the tasks of action classification, detection, prediction and anticipation as well as motion prediction and synthesis based on 3D human activity data recorded with Kinect. We further extend the model to capture hierarchical label structure and to model the dependencies between multiple entities, such as a human and objects. Our experiments demonstrate that our principled approach to human activity modeling can be used to detect current and anticipate future semantic labels and to predict and synthesize future label and feature sequences. When comparing our model to state-of-the-art approaches, which are specifically designed for e.g. action classification, we find that our probabilistic formulation outperforms or is comparable to these task specific models.

Computer Vision and Pattern Recognition

Human locomotion in space analyzed biomechanically through a multi-link chain model.

Towards Human-like Perception: Learning Structural Causal Model in Heterogeneous Graph

Predictive event segmentation and representation with neural networks: A self-supervised model assessed by psychological experiments

Recognizing Cognitive Load by a Hybrid Spatio-Temporal Causal Model from Multivariate Physiological Data

Incidence of cancer in Basrah: results of a household survey.

Image segmentation with cyclic load balanced parallel Fuzzy C - Means cluster analysis

Neural Foundations of Mental Simulation: Future Prediction of Latent Representations on Dynamic Scenes

Architecture of a Cortex Inspired Hierarchical Event Recaller

Using Features at Multiple Temporal and Spatial Resolutions to Predict Human Behavior in Real Time

Generative Modeling of Multimodal Multi-Human Behavior

A model of cortical cognitive function using hierarchical interactions of gating matrices in internal agents coding relational representations

Modeling Dynamic Environments with Scene Graph Memory

Understanding Human Cognition Through Computational Modeling

Learning to segment self-generated from externally caused optic flow through sensorimotor mismatch circuits

An Empirical Evaluation of Neural and Neuro-symbolic Approaches to Real-time Multimodal Complex Event Detection

Exploring the Chameleon Effect of Contextual Dynamics in Temporal Knowledge Graph for Event Prediction

Designing Technology for Older People - The Role of Technical Self-confidence in Usability of an Inclusive Heating Control

Learning, planning, and control in a monolithic neural event inference architecture

Episodic Memory for Learning Subjective-Timescale Models

Human Activity Learning and Segmentation using Partially Hidden Discriminative Models

A Probabilistic Semi-Supervised Approach to Multi-Task Human Activity Modeling