Abstract:The alternating direction method of multipliers (ADMM) has been extensively used in a wide variety of different applications. When large datasets with high-dimensional variables are considered, subproblems arising from the ADMM must be inexactly solved even though they may theoretically have closed-form solutions. Such a scenario immediately poses mathematical ambiguities such as how these subproblems should be accurately solved and whether the convergence can still be guaranteed. Although the ADMM is well known, it seems that these topics should be deeply investigated. In this paper, we study the mathematics of how to implement the ADMM for a large dataset scenarios. More specifically, we attempt to focus on the convex programming case where there is a quadratic function with extremely high-dimensional variables in the objective function of the model; thereby there is a huge-scale system for linear equations needing to be solved at each iteration of the ADMM. It is revealed that there is no need, indeed it is impossible, to exactly solve this linear system, and we attempt to propose an adjustable inexactness criterion to automatically and inexactly solve this linear system. We further attempt to identify the safe-guard number for the internally nested iterations that can sufficiently ensure this inexactness criterion if the linear system would be solved by a standard numerical linear algebra solver. The convergence, together with the worst-case convergence rate measured by the iteration complexity, is rigorously established for the ADMM with inexactly solved subproblems. Some numerical experiments for large datasets of the least absolute shrinkage and selection operator containing millions of variables are reported to show the efficiency of the mentioned inaccurate implementation of the ADMM.

A Maximally Split and Adaptive Relaxed Alternating Direction Method of Multipliers for Regularized Extreme Learning Machines

A Maximally Split and Relaxed ADMM for Regularized Extreme Learning Machines

Extreme Learning Machine with Multiple Kernels

Multiple-kernel-learning-based Extreme Learning Machine for Classification Design

A Novel Relaxed ADMM with Highly Parallel Implementation for Extreme Learning Machine.

An accelerated optimization algorithm for the elastic-net extreme learning machine

Multi-class AdaBoost ELM

A Batch Inheritance Extreme Learning Machine Algorithm Based on Regular Optimization

Distributed Extreme Learning Machine with Alternating Direction Method of Multiplier

An Adaptive Low Computational Cost Alternating Direction Method of Multiplier for RELM Large-Scale Distributed Optimization

A robust regularized extreme learning machine for regression problems based on self-adaptive accelerated extra-gradient algorithm

A Fast Training Algorithm for Extreme Learning Machine Based on Matrix Decomposition.

Kernel Extreme Learning Machine Based on Alternating Direction Multiplier Method of Binary Splitting Operator

Optimizing Extreme Learning Machine Via Generalized Hebbian Learning and Intrinsic Plasticity Learning

Linearized alternating direction method of multipliers for elastic-net support vector machines

A Multiobjective Optimization-Based Sparse Extreme Learning Machine Algorithm

A Modified Residual Extreme Learning Machine Algorithm and Its Application

Elastic Extreme Learning Machine For Big Data Classification

A Heterogeneous AdaBoost Ensemble Based Extreme Learning Machines for Imbalanced Data

Implementing the Alternating Direction Method of Multipliers for Big Datasets: A Case Study of Least Absolute Shrinkage and Selection Operator

PR-ELM: Parallel regularized extreme learning machine based on cluster.