The Computational Theory of Intelligence: Data Aggregation

Daniel Kovach

DOI: https://doi.org/10.4236/ijmnta.2014.34016

2014-12-24

Abstract:In this paper, we will expound upon the concepts proffered in [1], where we proposed an information theoretic approach to intelligence in the computational sense. We will examine data and meme aggregation, and study the effect of limited resources on the resulting meme amplitudes.

Artificial Intelligence

What problem does this paper attempt to address?

The Computational Theory of Intelligence: Information Entropy

Daniel Kovach

DOI: https://doi.org/10.4236/ijmnta.2014.34020

2014-12-24

Abstract:This paper presents an information theoretic approach to the concept of intelligence in the computational sense. We introduce a probabilistic framework from which computational intelligence is shown to be an entropy minimizing process at the local level. Using this new scheme, we develop a simple data driven clustering example and discuss its applications.

Artificial Intelligence,Machine Learning
Intelligence as Computation

Oliver Brock

DOI: https://doi.org/10.48550/arXiv.2405.16604

2024-05-26

Abstract:This paper proposes a specific conceptualization of intelligence as computation. This conceptualization is intended to provide a unified view for all disciplines of intelligence research. Already, it unifies several conceptualizations currently under investigation, including physical, neural, embodied, morphological, and mechanical intelligences. To achieve this, the proposed conceptualization explains the differences among existing views by different computational paradigms, such as digital, analog, mechanical, or morphological computation. Viewing intelligence as a composition of computations from different paradigms, the challenges posed by previous conceptualizations are resolved. Intelligence is hypothesized as a multi-paradigmatic computation relying on specific computational principles. These principles distinguish intelligence from other, non-intelligent computations. The proposed conceptualization implies a multi-disciplinary research agenda that is intended to lead to unified science of intelligence.

Artificial Intelligence,Robotics
Computational principles of intelligence: learning and reasoning with neural networks

Abel Torres Montoya

DOI: https://doi.org/10.48550/arXiv.2012.09477

2020-12-17

Abstract:Despite significant achievements and current interest in machine learning and artificial intelligence, the quest for a theory of intelligence, allowing general and efficient problem solving, has done little progress. This work tries to contribute in this direction by proposing a novel framework of intelligence based on three principles. First, the generative and mirroring nature of learned representations of inputs. Second, a grounded, intrinsically motivated and iterative process for learning, problem solving and imagination. Third, an ad hoc tuning of the reasoning mechanism over causal compositional representations using inhibition rules. Together, those principles create a systems approach offering interpretability, continuous learning, common sense and more. This framework is being developed from the following perspectives: as a general problem solving method, as a human oriented tool and finally, as model of information processing in the brain.

Artificial Intelligence,Machine Learning
The Information-theoretic and Algorithmic Approach to Human, Animal and Artificial Cognition

Nicolas Gauvrit,Hector Zenil,Jesper Tegnér

DOI: https://doi.org/10.48550/arXiv.1501.04242

2015-12-24

Abstract:We survey concepts at the frontier of research connecting artificial, animal and human cognition to computation and information processing---from the Turing test to Searle's Chinese Room argument, from Integrated Information Theory to computational and algorithmic complexity. We start by arguing that passing the Turing test is a trivial computational problem and that its pragmatic difficulty sheds light on the computational nature of the human mind more than it does on the challenge of artificial intelligence. We then review our proposed algorithmic information-theoretic measures for quantifying and characterizing cognition in various forms. These are capable of accounting for known biases in human behavior, thus vindicating a computational algorithmic view of cognition as first suggested by Turing, but this time rooted in the concept of algorithmic probability, which in turn is based on computational universality while being independent of computational model, and which has the virtue of being predictive and testable as a model theory of cognitive behavior.

Artificial Intelligence
Intelligent encoding and economical communication in the visual stream

Andras Lorincz

DOI: https://doi.org/10.48550/arXiv.q-bio/0403022

2004-03-16

Neurons and Cognition

Abstract:The theory of computational complexity is used to underpin a recent model of neocortical sensory processing. We argue that encoding into reconstruction networks is appealing for communicating agents using Hebbian learning and working on hard combinatorial problems, which are easy to verify. Computational definition of the concept of intelligence is provided. Simulations illustrate the idea.
Abstract Intelligence

Yingxu Wang,Lotfi A. Zadeh,Bernard Widrow,Newton Howard,Francoise Beaufays,George Baciu,D. Frank Hsu,Guiming Luo,Fumio Mizoguchi,Shushma Patel,Victor Raskin,Shusaku Tsumoto,Wei,Du Zhang

DOI: https://doi.org/10.4018/ijcini.2017010101

IF: 0.9

2017-01-01

International Journal of Cognitive Informatics and Natural Intelligence

Abstract:Basic studies in denotational mathematics and mathematical engineering have led to the theory of abstract intelligence aI, which is a set of mathematical models of natural and computational intelligence in cognitive informatics CI and cognitive computing CC. Abstract intelligence triggers the recent breakthroughs in cognitive systems such as cognitive computers, cognitive robots, cognitive neural networks, and cognitive learning. This paper reports a set of position statements presented in the plenary panel Part II of IEEE ICCI*CC'16 on Cognitive Informatics and Cognitive Computing at Stanford University. The summary is contributed by invited panelists who are part of the world's renowned scholars in the transdisciplinary field of CI and CC.
The Computational Principles of Learning Ability

Hao Wu

DOI: https://doi.org/10.48550/arXiv.1509.08891

2015-09-23

Abstract:It has been quite a long time since AI researchers in the field of computer science stop talking about simulating human intelligence or trying to explain how brain works. Recently, represented by deep learning techniques, the field of machine learning is experiencing unprecedented prosperity and some applications with near human-level performance bring researchers confidence to imply that their approaches are the promising candidate for understanding the mechanism of human brain. However apart from several ancient philological criteria and some imaginary black box tests (Turing test, Chinese room) there is no computational level explanation, definition or criteria about intelligence or any of its components. Base on the common sense that learning ability is one critical component of intelligence and inspect from the viewpoint of mapping relations, this paper presents two laws which explains what is the "learning ability" as we familiar with and under what conditions a mapping relation can be acknowledged as "Learning Model".

Artificial Intelligence
You want to survive the data deluge: Be careful, Computational Intelligence will not serve you as a rescue boat

Emanuel Diamant

DOI: https://doi.org/10.48550/arXiv.1607.05810

IF: 14.4

2016-07-20

Artificial Intelligence

Abstract:We are at the dawn of a new era, where advances in computer power, broadband communication and digital sensor technologies have led to an unprecedented flood of data inundating our surrounding. It is generally believed that means such as Computational Intelligence will help to outlive these tough times. However, these hopes are improperly high. Computational Intelligence is a surprising composition of two mutually exclusive and contradicting constituents that could be coupled only if you disregard and neglect their controversies: "Computational" implies reliance on data processing and "Intelligence" implies reliance on information processing. Only those who are indifferent to data-information discrepancy can believe that such a combination can be viable. We do not believe in miracles, so we will try to share with you our reservations.
Bits from Biology for Computational Intelligence

Michael Wibral,Joseph T. Lizier,Viola Priesemann

DOI: https://doi.org/10.3389/frobt.2015.00005

2014-12-01

Abstract:Computational intelligence is broadly defined as biologically-inspired computing. Usually, inspiration is drawn from neural systems. This article shows how to analyze neural systems using information theory to obtain constraints that help identify the algorithms run by such systems and the information they represent. Algorithms and representations identified information-theoretically may then guide the design of biologically inspired computing systems (BICS). The material covered includes the necessary introduction to information theory and the estimation of information theoretic quantities from neural data. We then show how to analyze the information encoded in a system about its environment, and also discuss recent methodological developments on the question of how much information each agent carries about the environment either uniquely, or redundantly or synergistically together with others. Last, we introduce the framework of local information dynamics, where information processing is decomposed into component processes of information storage, transfer, and modification -- locally in space and time. We close by discussing example applications of these measures to neural data and other complex systems.

Neurons and Cognition,Information Theory,Data Analysis, Statistics and Probability
Advances in Artificial Intelligence: Are you sure, we are on the right track?

Emanuel Diamant

DOI: https://doi.org/10.48550/arXiv.1502.04791

2015-02-17

Abstract:Over the past decade, AI has made a remarkable progress. It is agreed that this is due to the recently revived Deep Learning technology. Deep Learning enables to process large amounts of data using simplified neuron networks that simulate the way in which the brain works. However, there is a different point of view, which posits that the brain is processing information, not data. This unresolved duality hampered AI progress for years. In this paper, I propose a notion of Integrated information that hopefully will resolve the problem. I consider integrated information as a coupling between two separate entities - physical information (that implies data processing) and semantic information (that provides physical information interpretation). In this regard, intelligence becomes a product of information processing. Extending further this line of thinking, it can be said that information processing does not require more a human brain for its implementation. Indeed, bacteria and amoebas exhibit intelligent behavior without any sign of a brain. That dramatically removes the need for AI systems to emulate the human brain complexity! The paper tries to explore this shift in AI systems design philosophy.

Artificial Intelligence,Neurons and Cognition
Intelligence in Artificial Intelligence

Shoumen Palit Austin Datta

DOI: https://doi.org/10.48550/arXiv.1610.07862

2016-10-26

Abstract:The elusive quest for intelligence in artificial intelligence prompts us to consider that instituting human-level intelligence in systems may be (still) in the realm of utopia. In about a quarter century, we have witnessed the winter of AI (1990) being transformed and transported to the zenith of tabloid fodder about AI (2015). The discussion at hand is about the elements that constitute the canonical idea of intelligence. The delivery of intelligence as a pay-per-use-service, popping out of an app or from a shrink-wrapped software defined point solution, is in contrast to the bio-inspired view of intelligence as an outcome, perhaps formed from a tapestry of events, cross-pollinated by instances, each with its own microcosm of experiences and learning, which may not be discrete all-or-none functions but continuous, over space and time. The enterprise world may not require, aspire or desire such an engaged solution to improve its services for enabling digital transformation through the deployment of digital twins, for example. One might ask whether the "work-flow on steroids" version of decision support may suffice for intelligence? Are we harking back to the era of rule based expert systems? The image conjured by the publicity machines offers deep solutions with human-level AI and preposterous claims about capturing the "brain in a box" by 2020. Even emulating insects may be difficult in terms of real progress. Perhaps we can try to focus on worms (Caenorhabditis elegans) which may be better suited for what business needs to quench its thirst for so-called intelligence in AI.

Artificial Intelligence
Data Fusion: Theory, Methods, and Applications

Marek Gagolewski

DOI: https://doi.org/10.5281/zenodo.6960306

2022-08-02

Abstract:A proper fusion of complex data is of interest to many researchers in diverse fields, including computational statistics, computational geometry, bioinformatics, machine learning, pattern recognition, quality management, engineering, statistics, finance, economics, etc. It plays a crucial role in: synthetic description of data processes or whole domains, creation of rule bases for approximate reasoning tasks, reaching consensus and selection of the optimal strategy in decision support systems, imputation of missing values, data deduplication and consolidation, record linkage across heterogeneous databases, and clustering. This open-access research monograph integrates the spread-out results from different domains using the methodology of the well-established classical aggregation framework, introduces researchers and practitioners to Aggregation 2.0, as well as points out the challenges and interesting directions for further research.

Databases
Machine Learning and Information Theory Concepts towards an AI Mathematician

Yoshua Bengio,Nikolay Malkin

DOI: https://doi.org/10.1090/bull/1839

2024-05-25

Bulletin of the American Mathematical Society

Abstract:Yoshua Bengio and Nikolay Malkin Bull. Amer. Math. Soc. 61 (), 457-469. , references and article information

mathematics
Computational Intelligence for Big Social Data Analysis [guest Editorial].

Erik Cambria,Newton Howard,Yunqing Xia,Tat-Seng Chua

DOI: https://doi.org/10.1109/mci.2016.2572481

IF: 9.809

2016-01-01

IEEE Computational Intelligence Magazine

Abstract:The articles in this special section focus on computational intelligence for big social data analytics. In the eras of social connectedness and social colonization, people are becoming increasingly enthusiastic about interacting, sharing, and collaborating through online collaborative media. In recent years, this collective intelligence has spread to many different areas, with particular focus on fields related to everyday life such as commerce, tourism, education, and health, causing the size of the Social Web to expand exponentially. The distillation of knowledge from such a large amount of unstructured information, however, is an extremely difficult task, as the contents of today's Web are perfectly suitable for human consumption, but remain hardly understandable to machines. Big social data analysis grows out of this need and combines multiple disciplines such as social network analysis, multimedia management, social media analytics, trend discovery, and opinion mining.
Information Decomposition Diagrams Applied beyond Shannon Entropy: A Generalization of Hu's Theorem

Leon Lang,Pierre Baudot,Rick Quax,Patrick Forré

2024-09-23

Abstract:In information theory, one major goal is to find useful functions that summarize the amount of information contained in the interaction of several random variables. Specifically, one can ask how the classical Shannon entropy, mutual information, and higher interaction information relate to each other. This is answered by Hu's theorem, which is widely known in the form of information diagrams: it relates shapes in a Venn diagram to information functions, thus establishing a bridge from set theory to information theory. In this work, we view random variables together with the joint operation as a monoid that acts by conditioning on information functions, and entropy as a function satisfying the chain rule of information. This abstract viewpoint allows to prove a generalization of Hu's theorem. It applies to Shannon and Tsallis entropy, (Tsallis) Kullback-Leibler Divergence, cross-entropy, Kolmogorov complexity, submodular information functions, and the generalization error in machine learning. Our result implies for Chaitin's Kolmogorov complexity that the interaction complexities of all degrees are in expectation close to Shannon interaction information. For well-behaved probability distributions on increasing sequence lengths, this shows that the per-bit expected interaction complexity and information asymptotically coincide, thus showing a strong bridge between algorithmic and classical information theory.

Information Theory
Information Theory for Agents in Artificial Intelligence, Psychology, and Economics

Michael S Harré,Michael S. Harré

DOI: https://doi.org/10.3390/e23030310

IF: 2.738

2021-03-06

Entropy

Abstract:This review looks at some of the central relationships between artificial intelligence, psychology, and economics through the lens of information theory, specifically focusing on formal models of decision-theory. In doing so we look at a particular approach that each field has adopted and how information theory has informed the development of the ideas of each field. A key theme is expected utility theory, its connection to information theory, the Bayesian approach to decision-making and forms of (bounded) rationality. What emerges from this review is a broadly unified formal perspective derived from three very different starting points that reflect the unique principles of each field. Each of the three approaches reviewed can, in principle at least, be implemented in a computational model in such a way that, with sufficient computational power, they could be compared with human abilities in complex tasks. However, a central critique that can be applied to all three approaches was first put forward by Savage in The Foundations of Statistics and recently brought to the fore by the economist Binmore: Bayesian approaches to decision-making work in what Savage called ‘small worlds’ but cannot work in ‘large worlds’. This point, in various different guises, is central to some of the current debates about the power of artificial intelligence and its relationship to human-like learning and decision-making. Recent work on artificial intelligence has gone some way to bridging this gap but significant questions remain to be answered in all three fields in order to make progress in producing realistic models of human decision-making in the real world in which we live in.

physics, multidisciplinary
A Constructive Model for Collective Intelligence

Wei Zhang,Hong Mei

DOI: https://doi.org/10.1093/nsr/nwaa092

2020-01-01

Abstract:Summary Natural phenomena of collective intelligence (CI) occurring in physical space show a potential approach to effective large-scale human collaboration in cyberspace. Based on existing explanatory understanding of CI, this perspective proposes a constructive model for building artificial CI systems, i.e., problem-oriented CI phenomena with AI-powered information integration and feedback.
Divergence, Entropy, Information: An Opinionated Introduction to Information Theory

Philip Chodrow

DOI: https://doi.org/10.48550/arXiv.1708.07459

2019-03-24

Abstract:Information theory is a mathematical theory of learning with deep connections with topics as diverse as artificial intelligence, statistical physics, and biological evolution. Many primers on information theory paint a broad picture with relatively little mathematical sophistication, while many others develop specific application areas in detail. In contrast, these informal notes aim to outline some elements of the information-theoretic "way of thinking," by cutting a rapid and interesting path through some of the theory's foundational concepts and results. They are aimed at practicing systems scientists who are interested in exploring potential connections between information theory and their own fields. The main mathematical prerequisite for the notes is comfort with elementary probability, including sample spaces, conditioning, and expectations. We take the Kullback-Leibler divergence as our most basic concept, and then proceed to develop the entropy and mutual information. We discuss some of the main results, including the Chernoff bounds as a characterization of the divergence; Gibbs' Theorem; and the Data Processing Inequality. A recurring theme is that the definitions of information theory support natural theorems that sound ``obvious'' when translated into English. More pithily, ``information theory makes common sense precise.'' Since the focus of the notes is not primarily on technical details, proofs are provided only where the relevant techniques are illustrative of broader themes. Otherwise, proofs and intriguing tangents are referenced in liberally-sprinkled footnotes. The notes close with a highly nonexhaustive list of references to resources and other perspectives on the field.

Information Theory,Statistics Theory,Data Analysis, Statistics and Probability
Information Compression, Intelligence, Computing, and Mathematics

J. Gerard Wolff

DOI: https://doi.org/10.48550/arXiv.1310.8599

2015-07-13

Abstract:This paper presents evidence for the idea that much of artificial intelligence, human perception and cognition, mainstream computing, and mathematics, may be understood as compression of information via the matching and unification of patterns. This is the basis for the "SP theory of intelligence", outlined in the paper and fully described elsewhere. Relevant evidence may be seen: in empirical support for the SP theory; in some advantages of information compression (IC) in terms of biology and engineering; in our use of shorthands and ordinary words in language; in how we merge successive views of any one thing; in visual recognition; in binocular vision; in visual adaptation; in how we learn lexical and grammatical structures in language; and in perceptual constancies. IC via the matching and unification of patterns may be seen in both computing and mathematics: in IC via equations; in the matching and unification of names; in the reduction or removal of redundancy from unary numbers; in the workings of Post's Canonical System and the transition function in the Universal Turing Machine; in the way computers retrieve information from memory; in systems like Prolog; and in the query-by-example technique for information retrieval. The chunking-with-codes technique for IC may be seen in the use of named functions to avoid repetition of computer code. The schema-plus-correction technique may be seen in functions with parameters and in the use of classes in object-oriented programming. And the run-length coding technique may be seen in multiplication, in division, and in several other devices in mathematics and computing. The SP theory resolves the apparent paradox of "decompression by compression". And computing and cognition as IC is compatible with the uses of redundancy in such things as backup copies to safeguard data and understanding speech in a noisy environment.

Artificial Intelligence
Compression, The Fermi Paradox and Artificial Super-Intelligence

Michael Timothy Bennett

DOI: https://doi.org/10.1007/978-3-030-93758-4_5

2021-10-05

Abstract:The following briefly discusses possible difficulties in communication with and control of an AGI (artificial general intelligence), building upon an explanation of The Fermi Paradox and preceding work on symbol emergence and artificial general intelligence. The latter suggests that to infer what someone means, an agent constructs a rationale for the observed behaviour of others. Communication then requires two agents labour under similar compulsions and have similar experiences (construct similar solutions to similar tasks). Any non-human intelligence may construct solutions such that any rationale for their behaviour (and thus the meaning of their signals) is outside the scope of what a human is inclined to notice or comprehend. Further, the more compressed a signal, the closer it will appear to random noise. Another intelligence may possess the ability to compress information to the extent that, to us, their signals would appear indistinguishable from noise (an explanation for The Fermi Paradox). To facilitate predictive accuracy an AGI would tend to more compressed representations of the world, making any rationale for their behaviour more difficult to comprehend for the same reason. Communication with and control of an AGI may subsequently necessitate not only human-like compulsions and experiences, but imposed cognitive impairment.

Artificial Intelligence

The Computational Theory of Intelligence: Data Aggregation

The Computational Theory of Intelligence: Information Entropy

Intelligence as Computation

Computational principles of intelligence: learning and reasoning with neural networks

The Information-theoretic and Algorithmic Approach to Human, Animal and Artificial Cognition

Intelligent encoding and economical communication in the visual stream

Abstract Intelligence

The Computational Principles of Learning Ability

You want to survive the data deluge: Be careful, Computational Intelligence will not serve you as a rescue boat

Bits from Biology for Computational Intelligence

Advances in Artificial Intelligence: Are you sure, we are on the right track?

Intelligence in Artificial Intelligence

Data Fusion: Theory, Methods, and Applications

Machine Learning and Information Theory Concepts towards an AI Mathematician

Computational Intelligence for Big Social Data Analysis [guest Editorial].

Information Decomposition Diagrams Applied beyond Shannon Entropy: A Generalization of Hu's Theorem

Information Theory for Agents in Artificial Intelligence, Psychology, and Economics

A Constructive Model for Collective Intelligence

Divergence, Entropy, Information: An Opinionated Introduction to Information Theory

Information Compression, Intelligence, Computing, and Mathematics

Compression, The Fermi Paradox and Artificial Super-Intelligence