Abstract:Summary form only given. Real-world problems are often too large and complex for a single monolithic system to solve. In practice, the divide-and-conquer strategy has often been used to decompose a large and complex problem into smaller tractable sub-problems and then solve them. However, good decomposition of large and complex problems requires experienced human experts and rich prior domain knowledge, which are usually unavailable for real-world problems. This paper explores some of our research efforts towards an adaptive approach to divide-and-conquer in the design of machine learning systems, e.g., evolutionary and neural learning systems. The basic idea is to move away from designing a single monolithic system that would solve a large and complex problem, and to employ a population of simpler sub-systems that will cooperatively solve the problem. In such populations based systems, each sub-system will be simpler and easier to learn than the monolithic system. The integrated system based on the whole population can generalise better than any single subsystems in the population. In particular, by evolving and training a team of specialists from random initial conditions, we were able to "decompose" a large and complex problem into simpler ones and solve them without human intervention (X. Yao et al., 1996). Two major approaches is described. One uses the population structure in evolutionary algorithms, where individuals in a population are evolved into species (i.e., specialists for solving sub-problems) (P. J. Darwen and X. Yao, 1997). The other uses neural network ensembles in which individual neural networks learn to differentiate from and cooperate with each other (Liu and Yao, 1999; and Liu et al., 2000). A constructive algorithm for designing ensembles as well as individual neural networks will be introduced by MM Islam et al. (2003).

Multi-network Evolutionary Systems and Automatic Decomposition of Complex Problems

Co-Evolutionary Modular Neural Networks for Automatic Problem Decomposition

CMNN: Coupled Modular Neural Network.

Evolving Modular Neural Networks Which Generalise Well

Automatic Feature Extraction Based Structure Decomposition Method for Multi-Classification

Automatic Divide-And-Conquer Using Populations and Ensembles

Evolutionary Multitasking Via Artificial Neural Networks.

A Parallel and Modular Multi-Sieving Neural Network Architecture with Multiple Control Networks

Multi-Objective Neural Evolutionary Algorithm for Combinatorial Optimization Problems

Task Decomposition Based on Class Relations: A Modular Neural Network Architecture for Pattern Classification

An Online Self-Organizing Modular Neural Network for Nonlinear System Modeling

Evolutionary Multitasking Multilayer Network Reconstruction

Neural Network Module Decomposition and Recomposition

Million module neural systems evolution

Evolving Artificial Neural Network Ensembles

An Adaptive Evolutionary Modular Neural Network with Intermodule Connections

Task Decomposition and Module Combination Based on Class Relations: a Modular Neural Network for Pattern Classification

Evolving Deep Parallel Neural Networks for Multi-Task Learning

An Evolutionary Multiobjective Framework for Complex Network Reconstruction Using Community Structure

Designing Neural Networks Based on Structure Decomposition