Abstract:The quick evolution and widespread applicability of machine learning and artificial intelligence have fundamentally shaped and transcended modern life. Three key players stand behind such a ubiquitous emergence: big data, growing computing power, and improved algorithms. The need for distributed storage and processing arises from this ``data deluge'' that can flood any powerful machine, from the dispersively available data that are prohibitively costly to transfer to a central unit for further processing, and from the prevalent Internet-of-Things (IoT) devices that require real-time response as well as respect of privacy. Modern machine learning algorithms built to exploit such huge amounts of data are often computationally ``hungry'' and their ``appetite'' for computing power increases rapidly at a pace unmatched by the development of computing hardware. All these considerations justify the pressing need for distributed optimization algorithms that are scalable yet flexible to adapt to various configurations of networked computing nodes. To cope with these challenges, the present thesis first introduces a novel ADMM based approach (termed hybrid ADMM) for efficient decentralized optimization. By modeling the underlying communication patterns as hypergraphs, it provides a unifying framework that subsumes both centralized and fully decentralized counterparts as special cases, and allows nodes to communicate in centralized and decentralized ways at the same time. Leveraging the expressiveness of hypergraph models, a technique termed ``in-network acceleration'' is introduced enabling ``almost free'' performance gain by exploiting local graph topology. To account for heterogeneity of nodes and edges, a diagonal scaling based approach is proposed to tackle weighted updates, where proper edge weights are identified through solving a preconditioning problem. By assigning larger weights to critical edges, the proposed algorithm achieves higher efficiency and becomes more robust to perturbations. Finally, to boost the efficiency of the whole system to its full potential, an asynchronous method is introduced to mitigate the straggler problem so that nodes with different processing power can run at full speed. Convergence analysis for proposed algorithms is provided which reveals the connection between convergence rate and spectral properties of the communication graph. Numerical tests of several common tasks on different graphs are carried out to demonstrate the effectiveness of proposed algorithms.

Game Theory for Big Data Processing: Multileader Multifollower Game-Based ADMM

Multi-leader Multi-Follower Game-Based ADMM for Big Data Processing

Bridge the Gap Between ADMM and Stackelberg Game: Incentive Mechanism Design for Big Data Networks.

Target-Value-Competition-Based Multi-Agent Deep Reinforcement Learning Algorithm for Distributed Nonconvex Economic Dispatch

Game Theoretic Approaches to Massive Data Processing in Wireless Networks

A game theoretical formulation for distributed optimization problems

Hierarchical game theoretical distributed adaptive control for large scale multi‐group multi‐agent system

A New Game Model for Distributed Optimization Problems with Directed Communication Topologies

Delay-Optimal Computation Offloading in Large-Scale Multi-Access Edge Computing Using Mean Field Game

Distributed ADMM for Time-Varying Communication Networks

Distributed Stackelberg Equilibrium Seeking for Networked Multi-Leader Multi-Follower Games with A Clustered Information Structure

Bridging the Gap Between Big Data and Game Theory: A General Hierarchical Pricing Framework

Efficient Methods for Decentralized Optimization over Graphs

Online Parallel Multi-Task Relationship Learning via Alternating Direction Method of Multipliers

Two-block vs. Multi-block ADMM: An empirical evaluation of convergence

A Multi-Leader Multi-Follower Stackelberg Game for Coalition-Based UAV MEC Networks

Coded Stochastic ADMM for Decentralized Consensus Optimization With Edge Computing

Adaptive dynamic programming-based hierarchical decision-making of non-affine systems

Improved Hierarchical ADMM for Nonconvex Cooperative Distributed Model Predictive Control

Bio-Inspired Dynamic Collective Choice in Large-Population Systems: A Robust Mean-Field Game Perspective

Optimal Leader-Following Consensus Control of Multi-Agent Systems: A Neural Network Based Graphical Game Approach