Abstract:We show how complexity theory can be introduced in machine learning to help bring together apparently disparate areas of current research. We show that this model-driven approach may require less training data and can potentially be more generalizable as it shows greater resilience to random attacks. In an algorithmic space the order of its element is given by its algorithmic probability, which arises naturally from computable processes. We investigate the shape of a discrete algorithmic space when performing regression or classification using a loss function parametrized by algorithmic complexity, demonstrating that the property of differentiation is not required to achieve results similar to those obtained using differentiable programming approaches such as deep learning. In doing so we use examples which enable the two approaches to be compared (small, given the computational power required for estimations of algorithmic complexity). We find and report that 1) machine learning can successfully be performed on a non-smooth surface using algorithmic complexity; 2) that solutions can be found using an algorithmic-probability classifier, establishing a bridge between a fundamentally discrete theory of computability and a fundamentally continuous mathematical theory of optimization methods; 3) a formulation of an algorithmically directed search technique in non-smooth manifolds can be defined and conducted; 4) exploitation techniques and numerical methods for algorithmic search to navigate these discrete non-differentiable spaces can be performed; in application of the (a) identification of generative rules from data observations; (b) solutions to image classification problems more resilient against pixel attacks compared to neural networks; (c) identification of equation parameters from a small data-set in the presence of noise in continuous ODE system problem, (d) classification of Boolean NK networks by (1) network topology, (2) underlying Boolean function, and (3) number of incoming edges.

Active learning meets fractal decision boundaries: a cautionary tale from the Sitnikov three-body problem

Can machine learning really solve the three-body problem?

Machine Learning for predicting chaotic systems

Active Learning Algorithm for Computational Physics

Numerical Studies of Statistical Management Decisions in Conditions of Stochastic Chaos

Active Learning in Physics: From 101, to Progress, and Perspective

Influence of database noises to machine learning for spatiotemporal chaos

Innovative Deep Learning Strategies for Chaotic Data Classification: A Multi-Algorithm Comparison in the Presence of Noise

On the Limitations of Simulating Active Learning

Emergent Instabilities in Algorithmic Feedback Loops

Stabilizing machine learning prediction of dynamics: Novel noise-inspired regularization tested with reservoir computing

Isles of regularity in a sea of chaos amid the gravitational three-body problem

Speeding Up Quantum Few-Body Calculation with Active Learning

Predicting Chaotic System Behavior using Machine Learning Techniques

On the Fragility of Active Learners for Text Classification

Controlling nonlinear dynamical systems into arbitrary states using machine learning

Algorithmic Probability-Guided Machine Learning on Non-Differentiable Spaces

Active Learning based on Data Uncertainty and Model Sensitivity

Machine learning for decision-making under uncertainty

Frugal Reinforcement-based Active Learning