AFRL-AFOSR-JP-TR-2016-0014 Bio-Inspired Human-Level Machine Learning

Byoung-Tak Zhang

Abstract:How can brain computation be so fast, flexible, and robust? What kinds of representational and organizational principles facilitate the biological brain to learn so efficiently and flexibly on the sub-second time scale and so reliably on the continuous lifetime scale? To understand these principles, we aimed to develop human-level machine learning technology that is fast, flexible, and reliable to adapt to a continuously changing, dynamic environment. Based on dynamic “neural” populations (neural assemblies), we constructed a “human-like” machine learning model and implement this model in “molecular” populations (molecular assemblies) using in vitro DNA computing. In the first year, we developed the dynamic hypernetwork models of neural populations in the sequential Bayesian framework for lifelong learning. In the second year, we extended it to the molecular dynamic hypernetwork model, and designed in vitro experimental protocols to implement online language learning from a stream of text corpus. In the third year, we demonstrated the use of molecular dynamic hypernetworks for multimodal visuo-linguistic concept learning from a long stream of video data and their extensions to high-level cognitive functions such as anagram solving problem. We expect that the bio-inspired human-level machine learning combined with molecular-computing implementation can offer an interesting, novel paradigm to address for flexible and reliable computing. Introduction: One of the main challenges in artificial intelligence is to develop human-like machine learning technology that is fast, flexible, and reliable to adapt to a continuously changing, dynamic environment. Converging neuroanatomical and neurophysiological evidence shows that the brain uses distributed, overlapping representations based on sparse population codes that are coordinated dynamically (Averbeck et al., 2006; Pouget et al., 2000; von der Malsburg et al., 2010). We hypothesize that brain computation exploits the huge degrees of freedom generated by a large number of memory units, ranging from neurotransmitters and neurons to DISTRIBUTION A: Distribution approved for public release. cell-assembly, and organized into multiscale complex networks in space and coordinated dynamically in time (Caroni, 2012; Freeman, 2000). The objective of this project is to build a learning-friendly computational model based on dynamic neural populations and implementing this model in self-assembling molecular populations using DNA computing. A key idea underlying this approach is that the plasticity of neural populations in the brain is based on molecular interactions at the physico-chemical level and, thus, molecular computational processes can naturally simulate human-like learning and memory. The molecular self-assembly mechanisms in DNA chemistry provide us a natural, physical medium for modeling dynamic “neural” populations (neural assemblies). Massively parallel mechanisms of in vitro DNA computing provide us a convenient tool for dealing with large populations, 10 molecules in a nano-mole, which is bigger than the numbers of 10 neurons and 10 synaptic connections in the human brain. In previous work, we experimentally demonstrated the feasibility of cognitive memory with DNA self-assembly. We showed that wet DNA computing can implement weighted-sum operations which are fundamental to perform pattern classification (Lim et al., 2010). Since pattern classification underlies many cognitive tasks, this work opened a new way of creating flexible cognitive memories in vitro with molecules. We also demonstrated the potential of the molecular self-assembly model to build associative language models automatically from language data to generate sentences (Lee et al., 2011). On the mathematical and computational modeling side we developed a probabilistic graphical model of sparse, random population codes called hypernetworks (Zhang, 2008). The model also applied to a visually-grounded language learning (Zhang 35 al., 2012), where cognitive memory consists of multimodal compound concepts which are encoded as hyperedges (molecular memory particles) and then assembled, dissembled, and reassembled to be adapted incrementally as the video sequences are observed. However, there were several challenges to achieving human-level learning and memory. First, the concept of population coding needed to be extended to deal with online, predictive learning in a changing environment. Second, representational formalisms and their translations between neural populations and molecular populations needed to be investigated. Third, the DNA computing and molecular learning technology needed be scaled up to make molecular computational simulation of the whole-brain scale, to make cognitive learning possible and to achieve human-level machine learning. In the first year of the project, we focused on constructing mathematical theories of dynamic neural populations. Building upon our previous work on the hypernetwork models of cognitive learning and memory (Zhang, 2012), we developed population-coded dynamic hypernetwork models of lifelong learning in a non-stationary, changing environment [1, 2, 6, 8, 9, 17]. In [9], we discussed our model from the perspectives of embodied cognition, multisensory integration, cognitive dynamics, perception-action cycle, and lifelong learning. We developed a sequential Bayesian framework for lifelong learning, built a taxonomy of lifelong-learning paradigms, and examined information-theoretic objective functions for each paradigm, with an emphasis on active learning. Also, in [7], we presented that DNA hybridization can be modeled as computing the inner product between DISTRIBUTION A: Distribution approved for public release. embedded vectors in a corresponding vector space, and proposed the algorithm performing learning of a binary classifier in this vector space. In the second year, we extended this to the molecular dynamic hypernetwork model, and designed in vitro experimental protocols to implement online language learning from a stream of text corpus [3, 4, 10, 14, 19, 20, 23]. To measure the difference between different information-encoded sequences, we introduced the symmetric internal loops of double stranded DNA, and which were used to recognize similar or different patterns. Through a series of training processes which is simply storing the given training data in different microtubes in each class of hypernetwork, we observed that the accuracy of sentence classification tasks increased on the corpus of TV show dialogue and our molecular learning was able to generalize the training sentences. In the third year, we demonstrated the use of molecular dynamic hypernetworks for multimodal visuo-linguistic concept learning from a long stream of video data. Motivated by the cognitive developmental process of children constructing the visually grounded concepts from multimodal stimuli (Meltzoff, 1990), we proposed a hierarchical model of automatically constructing visual-linguistic knowledge by dynamically learning concepts represented with vision and language from videos [8, 12, 15, 16, 22]. We developed a stochastic method for graph construction, i.e. a graph Monte Carlo algorithm, and our model learns the concepts by the algorithm while observing new videos, thus robustly tracing concept drift and continuously accumulating new conceptual knowledge. Using a series of approximately 200 episodes of educational cartoon videos we examined the emergence and evolution of the concept hierarchies as the video stories unfold. Through the experiment, we observed that the number of visual and linguistic nodes tends to increase, because the concepts continuously develop while observing the videos. Also, we presented a molecular computational model for human anagram solving to show the potential of application to high-level cognitive functions [5, 11, 13, 18, 21]. Our major contribution is to propose the molecular assembly model of cognitive memory and learning which can be used as a tool for simulating cognitive dynamics involved with multisensory cue integration, grounded concept learning, and interaction of vision and language. We believe that the bio-inspired human-level machine learning combined with molecular-computing implementation can offer an interesting, novel paradigm to address for flexible and reliable computing. We also expect that the cognitive memory architectures and their learning algorithms contribute to revolutionize the AI technology to be used in lifelong learning, self-organizing, sensorimotor systems. DISTRIBUTION A: Distribution approved for public release. [1 Year] The Dynamic Hypernetwork Models of Neural Populations Experiments: In the first year, we constructed a dynamic Bayesian inference framework and examined information-theoretic objective functions for lifelong learning [9]. In lifelong learning, training data are observed sequentially as learning unfolds and not kept for iterative reuse. The learning is proceeded in an online and incremental manner over an extended period in a changing environment. This requires incremental transfer of knowledge acquired from previous learning to future learning, which can be formulated as a Bayesian inference. We applied a sequential Bayesian framework for lifelong learning to build taxonomy of lifelong-learning paradigms, and examine information-theoretic objective functions for each paradigm (Figure 1). Figure 1. Lifelong learning with action-perception-learning cycle [9] Results and Discussion: We distinguished three paradigms of lifelong-learning: learning with passive and continual observations, learning with actions (but without reward feedbacks), and active learning with explicit rewards. For each of the paradigm we examined the objective functions of the lifelong learning styles: prediction errors and predictive information, empowerment which measures how much influence an agent has on its environment, and the