Program Synthesis using Inductive Logic Programming for the Abstraction and Reasoning Corpus

Filipe Marinho Rocha,Inês Dutra,Vítor Santos Costa
2024-05-10
Abstract:The Abstraction and Reasoning Corpus (ARC) is a general artificial intelligence benchmark that is currently unsolvable by any Machine Learning method, including Large Language Models (LLMs). It demands strong generalization and reasoning capabilities which are known to be weaknesses of Neural Network based systems. In this work, we propose a Program Synthesis system that uses Inductive Logic Programming (ILP), a branch of Symbolic AI, to solve ARC. We have manually defined a simple Domain Specific Language (DSL) that corresponds to a small set of object-centric abstractions relevant to ARC. This is the Background Knowledge used by ILP to create Logic Programs that provide reasoning capabilities to our system. The full system is capable of generalize to unseen tasks, since ILP can create Logic Program(s) from few examples, in the case of ARC: pairs of Input-Output grids examples for each task. These Logic Programs are able to generate Objects present in the Output grid and the combination of these can form a complete program that transforms an Input grid into an Output grid. We randomly chose some tasks from ARC that dont require more than the small number of the Object primitives we implemented and show that given only these, our system can solve tasks that require each, such different reasoning.
Machine Learning,Artificial Intelligence,Programming Languages
What problem does this paper attempt to address?
### Problems the Paper Aims to Solve This paper aims to address the Abstraction and Reasoning Corpus (ARC) challenge, an artificial intelligence benchmark that no current machine learning methods (including large language models) can solve. Specifically: 1. **Characteristics of the ARC Challenge**: - ARC requires strong generalization and reasoning abilities, which are lacking in neural network-based systems. - ARC is a general artificial intelligence benchmark designed to assess whether AI systems can emulate human-like general intelligence. 2. **Proposed Method**: - The authors propose a program synthesis system that uses Inductive Logic Programming (ILP) to solve ARC tasks. - They define a simple Domain Specific Language (DSL) to describe object abstractions related to ARC. - ILP uses this background knowledge to create logical programs, endowing the system with reasoning capabilities. - The system can generate logical programs from a small number of examples, transforming input grids into output grids. 3. **Experimental Results**: - The system was successfully applied to 5 randomly selected ARC tasks, demonstrating its effectiveness across different types of reasoning. In summary, the main goal of this paper is to present a new approach to solving the ARC challenge through inductive logic programming and object-centered abstraction, thereby enhancing the generalization and reasoning capabilities of AI systems.