Abstract:Ensembling is one approach that improves the performance of a neural network by combining a number of independent neural networks, usually by either averaging or summing up their individual outputs. We modify this ensembling approach by training the sub-networks concurrently instead of independently. This concurrent training of sub-networks leads them to cooperate with each other, and we refer to them as "cooperative ensemble". Meanwhile, the mixture-of-experts approach improves a neural network performance by dividing up a given dataset to its sub-networks. It then uses a gating network that assigns a specialization to each of its sub-networks called "experts". We improve on these aforementioned ways for combining a group of neural networks by using a k-Winners-Take-All (kWTA) activation function, that acts as the combination method for the outputs of each sub-network in the ensemble. We refer to this proposed model as "kWTA ensemble neural networks" (kWTA-ENN). With the kWTA activation function, the losing neurons of the sub-networks are inhibited while the winning neurons are retained. This results in sub-networks having some form of specialization but also sharing knowledge with one another. We compare our approach with the cooperative ensemble and mixture-of-experts, where we used a feed-forward neural network with one hidden layer having 100 neurons as the sub-network architecture. Our approach yields a better performance compared to the baseline models, reaching the following test accuracies on benchmark datasets: 98.34% on MNIST, 88.06% on Fashion-MNIST, 91.56% on KMNIST, and 95.97% on WDBC.

Multi-robot competitive tracking based on k-WTA neural network with one single neuron

Distributed Dynamic Task Allocation for Moving Target Tracking of Networked Mobile Robots Using $k$-WTA Network

Multi-modal 3D Human Tracking for Robots in Complex Environment with Siamese Point-Video Transformer

Exploit the Connectivity: Multi-Object Tracking with TrackletNet

Cooperative Tracking Control for Nonlinear MASs under Event-Triggered Communication.

k-Winners-Take-All Ensemble Neural Network

Multi-Sensors Data Tracking Fusion Based on a Neural Network Filter

A new recurrent neural network for solving convex quadratic programming problems with an application to the κ-winners-take-all problem

Adaptive NN-based finite-time tracking control for wheeled mobile robots with time-varying full state constraints

PNAS-MOT: Multi-Modal Object Tracking with Pareto Neural Architecture Search

Task-space tracking for networked heterogeneous robotic systems via adaptive neural fixed-time control

Multi-Robot Active Mapping via Neural Bipartite Graph Matching

Dynamic Task Allocation in Multi-Robot Coordination for Moving Target Tracking: A Distributed Approach

Neural-Network-Based Adaptive Tracking Control for Nonlinear Multiagent Systems: The Observer Case

Data-driven Predictive Tracking Control based on Koopman Operators

Manoeuvre Target Tracking in Wireless Sensor Networks Using Convolutional Bi-Directional Long Short-Term Memory Neural Networks and Extended Kalman Filtering

Solving the K-Winners-take-all Problem and the Oligopoly Cournot-Nash Equilibrium Problem Using the General Projection Neural Networks.

An Improved Dual Neural Network for Solving A Class of Quadratic Programming Problems and Its K-Winners-Take-All Application

The framework of multi-target tracking based on neural network and motion model prediction

MM-Tracker: Visual Tracking with A Multi-Task Model Integrating Detection and Differentiating Feature Extraction