Distributed Learning in Wireless Sensor Networks

Ananthram Swami,Qing Zhao,Yao-Win Hong,Lang Tong

2007-01-01

Abstract:This chapter contains sections titled: Introduction Classical Learning Distributed Learning in Wireless Sensor Networks Distributed Learning in WSNs with a Fusion Center Distributed Learning in Ad‐hoc WSNs with In‐network Processing Conclusion Bibliography

What problem does this paper attempt to address?

Distributed Storage and Parallel Processing in Large-Scale Wireless Sensor Networks.

Yuexuan Wang,Yongcai Wang

DOI: https://doi.org/10.3233/978-1-60750-803-8-288

2010-01-01

Abstract:A Large-scale Wireless Sensor Network (LWSN). such as an environment monitoring system deployed in a city, could yield data on the order of petabytes each year. Storage and computation of such vast quantities of data pose difficult challenges to the LWSN, particularly because sensors are highly constrained by their scarce resources. Distributed storage and parallel processing are solutions that deal with the massive amount of data by utilizing the collective computational power of the large number of sensors, all while keeping inter-node communication minimized to save energy. In this chapter, we conduct a survey on the state-of-the-art of distributed storage and parallel processing in LWSNs. We will focus on the LWSN scenario in which vast amounts of data are collected prior to intensive computation on that data. We argue that current research results in this direction fall into three categories: 1) hierarchical system architecture to support parallel and distributed computation; 2) distributed data aggregation and storage strategies; 3) parallel processing, scheduling and programming methods. We highlight some important results for each of these categories and discuss existing problems and future directions.
Learn to Sense: a Meta-learning Based Distributed Sensing Algorithm in Wireless Sensor Networks

Hui Wu,Zhaoyang Zhang,Chunxu Jiao

DOI: https://doi.org/10.1109/wcsp.2018.8555926

2018-01-01

Abstract:In the era of big data, it has been much more challenging to sense and reconstruct a scalar field for wireless sensor network (WSN). Both academia and industry show an urgent need for cost-efficient field estimation algorithm. To address this issue, this paper appeals to a newly proposed communication computation integrated framework. On the basis of reinforcement learning and meta-learning, a distributed two-layer learning and sensing algorithm, which adaptively determines the most informative sensing location, is presented. It significantly reduces the communication overhead and lays a good foundation for future efficient information acquisition system. Numerical results show that the proposed two-layer algorithm brings a remarkable improvement in acquisition efficiency compared to conventional ones. In the meantime, it shows strong robustness to time-varying field functions and task changes.
Distributed Learning for Wireless Communications: Methods, Applications and Challenges

Liangxin Qian,Ping Yang,Ming Xiao,Marco Di Renzo,Jun Li,Zhu Han,Qin Yi,Jiarong Zhao,Octavia Dobre

DOI: https://doi.org/10.1109/jstsp.2022.3156756

IF: 7.695

2022-01-01

IEEE Journal of Selected Topics in Signal Processing

Abstract:With its privacy-preserving and decentralized features, distributed learning plays an irreplaceable role in the era of wireless networks with a plethora of smart terminals, an explosion of information volume and increasingly sensitive data privacy issues. There is a tremendous increase in the number of scholars investigating how distributed learning can be employed to emerging wireless network paradigms in the physical layer, media access control layer and network layer. Nonetheless, research on distributed learning for wireless communications is still in its infancy. In this paper, we review the contemporary technical applications of distributed learning for wireless communications. We first introduce the typical frameworks and algorithms for distributed learning. Examples of applications of distributed learning frameworks in emerging wireless network paradigms are then provided. Finally, main research directions and challenges of distributed learning for wireless communications are discussed.

engineering, electrical & electronic
Learn to Sense: A Meta-Learning-Based Sensing and Fusion Framework for Wireless Sensor Networks

Hui Wu,Zhaoyang Zhang,Chunxu Jiao,Chunguang Li,Tony Q. S. Quek

DOI: https://doi.org/10.1109/jiot.2019.2919225

IF: 10.6

2019-01-01

IEEE Internet of Things Journal

Abstract:Wireless sensor networks (WSNs) act as the backbone of Internet of Things (IoT) technology. In WSN, field sensing and fusion are the most commonly seen problems, which involve collecting and processing of a huge volume of spatial samples in an unknown field to reconstruct the field or extract its features. One of the major concerns is how to reduce the communication overhead and data redundancy with prescribed fusion accuracy. In this paper, an integrated communication and computation framework based on meta-learning is proposed to enable adaptive field sensing and reconstruction. It consists of a stochastic-gradient-descent (SGD)-based base-learner used for the field model prediction aiming to minimize the average prediction error, and a reinforcement meta-learner aiming to optimize the sensing decision by simultaneously rewarding the error reduction with samples obtained so far and penalizing the corresponding communication cost. An adaptive sensing algorithm based on the above two-layer meta-learning framework is presented. It actively determines the next most informative sensing location, and thus considerably reduces the spatial samples and yields superior performance and robustness compared with conventional schemes. The convergence behavior of the proposed algorithm is also comprehensively analyzed and simulated. The results reveal that the proposed field sensing algorithm significantly improves the convergence rate.
Distributed Learning in Wireless Networks: Recent Progress and Future Challenges

Mingzhe Chen,Deniz Gündüz,Kaibin Huang,Walid Saad,Mehdi Bennis,Aneta Vulgarakis Feljan,H. Vincent Poor,Deniz Gunduz

DOI: https://doi.org/10.1109/jsac.2021.3118346

IF: 16.4

2021-12-01

IEEE Journal on Selected Areas in Communications

Abstract:The next-generation of wireless networks will enable many machine learning (ML) tools and applications to efficiently analyze various types of data collected by edge devices for inference, autonomy, and decision making purposes. However, due to resource constraints, delay limitations, and privacy challenges, edge devices cannot offload their entire collected datasets to a cloud server for centrally training their ML models or inference purposes. To overcome these challenges, distributed learning and inference techniques have been proposed as a means to enable edge devices to collaboratively train ML models without raw data exchanges, thus reducing the communication overhead and latency as well as improving data privacy. However, deploying distributed learning over wireless networks faces several challenges including the uncertain wireless environment (e.g., dynamic channel and interference), limited wireless resources (e.g., transmit power and radio spectrum), and hardware resources (e.g., computational power). This paper provides a comprehensive study of how distributed learning can be efficiently and effectively deployed over wireless edge networks. We present a detailed overview of several emerging distributed learning paradigms, including federated learning, federated distillation, distributed inference, and multi-agent reinforcement learning. For each learning framework, we first introduce the motivation for deploying it over wireless networks. Then, we present a detailed literature review on the use of communication techniques for its efficient deployment. We then introduce an illustrative example to show how to optimize wireless networks to improve its performance. Finally, we introduce future research opportunities. In a nutshell, this paper provides a holistic set of guidelines on how to deploy a broad range of distributed learning frameworks over real-world wireless communication networks.
Decentralized learning for wireless communications and networking

Georgios B. Giannakis,Qing Ling,Gonzalo Mateos,Ioannis D. Schizas,Hao Zhu

DOI: https://doi.org/10.48550/arXiv.1503.08855

2015-03-31

Abstract:This chapter deals with decentralized learning algorithms for in-network processing of graph-valued data. A generic learning problem is formulated and recast into a separable form, which is iteratively minimized using the alternating-direction method of multipliers (ADMM) so as to gain the desired degree of parallelization. Without exchanging elements from the distributed training sets and keeping inter-node communications at affordable levels, the local (per-node) learners consent to the desired quantity inferred globally, meaning the one obtained if the entire training data set were centrally available. Impact of the decentralized learning framework to contemporary wireless communications and networking tasks is illustrated through case studies including target tracking using wireless sensor networks, unveiling Internet traffic anomalies, power system state estimation, as well as spectrum cartography for wireless cognitive radio networks.

Optimization and Control,Information Theory,Machine Learning,Multiagent Systems,Systems and Control
Distributed Learning for Time-varying Networks: A Scalable Design

Jian Wang,Yourui Huangfu,Rong Li,Yiqun Ge,Jun Wang

DOI: https://doi.org/10.1109/GCWkshps52748.2021.9682123

2021-01-01

Abstract:The wireless network is undergoing a trend from "connection of things" to "connection of intelligence". With data spread over the communication networks and computing capability enhanced on the devices, distributed learning becomes a hot topic in both industrial and academic communities. Many frameworks, such as federated learning and federated distillation, have been proposed. However, few of the...
Distributed Semi-Supervised Metric Learning.

Pengcheng Shen,Xin Du,Chunguang Li

DOI: https://doi.org/10.1109/access.2016.2632158

IF: 3.9

2016-01-01

IEEE Access

Abstract:Over the last decade, many pairwise-constraint-based metric learning algorithms have been developed to automatically learn application-specific metrics from data under similarity/dissimilarity data-pair constraints (weak labels). Nevertheless, these existing methods are designed for the centralized learning case, in which all the data and constraints are supposed to be gathered together in one source, and the algorithms utilize the whole data and constraints information during the learning process. However, in many real applications, large amounts of data (constraints) are dispersedly generated/stored in geographically distributed nodes over networks. Thus, it might be impractical to centralize the whole data information to one fusion node. Besides, in such cases, it is often hard to have every data pair labeled due to the huge data-pair amounts, resulting in numerous unlabeled data pairs. Given these situations, in this paper, we propose two types, namely, a diffusion type and an alternating-direction-method-of-multipliers type, of distributed semi-supervised metric learning frameworks, which make use of both labeled and unlabeled data pairs. The proposed frameworks can be easily used to extend centralized metric learning methods of different objective functions to distributed cases. In particular, we apply our frameworks on a well-behaved centralized semi-supervised metric learning method called SERAPH and yield two new distributed semi-supervised metric learning algorithms. Our simulation results show that the metrics learned by the proposed distributed algorithms are very close to that of the corresponding centralized method in most cases.
Distributed Machine Learning for Wireless Communication Networks: Techniques, Architectures, and Applications

Shuyan Hu,Xiaojing Chen,Wei Ni,Ekram Hossain,Xin Wang

DOI: https://doi.org/10.1109/comst.2021.3086014

2021-01-01

Abstract:Distributed machine learning (DML) techniques, such as federated learning, partitioned learning, and distributed reinforcement learning, have been increasingly applied to wireless communications. This is due to improved capabilities of terminal devices, explosively growing data volume, congestion in the radio interfaces, and increasing concern of data privacy. The unique features of wireless systems, such as large scale, geographically dispersed deployment, user mobility, and the massive amount of data, give rise to new challenges in the design of DML techniques. There is a clear gap in the existing literature that the DML techniques are yet to be systematically reviewed for their applicability to wireless systems. This survey bridges the gap by providing a contemporary and comprehensive survey of DML techniques with a focus on wireless networks. Specifically, we review the latest applications of DML in power control, spectrum management, user association, and edge cloud computing. The optimality, accuracy, convergence rate, computation cost, and communication overhead of DML are analyzed. We also discuss the potential adversarial attacks faced by DML applications, and describe state-of-the-art countermeasures to preserve privacy and security. Last but not least, we point out a number of key issues yet to be addressed, and collate potentially interesting and challenging topics for future research.

computer science, information systems,telecommunications
Diffusion Based Distributed Detection in Wireless Sensor Network

Anand Singh

2015-06-01

Abstract:Distributed wireless sensor networks find various remote sensing purposes like battleground monitoring, target localization, environmental monitoring, accurate cultivation, mobile communication and medicinal applications. Due to a wide variety of applications of wireless data, suitable design and implementation of data detection become the modern field of study and research. The distribution of the nodes in the network provides a spatial diversity, which includes the temporal dimension for the purpose of increase the robustness of the ongoing tasks and enhance the probability of data and event detection. In this area, we study the distributed network that contain the collection of a node connected to each other in the distributed manner. The node connected to each other is called neighbor node. In the problem of distributed detection of data, nodes have to decide based on the binary hypotheses of the measured data. In this detection problem we find the fully distributed and adaptive approach where all the node have to make own real time decision by cooperating with their immediate neighbour only and for this implementation no central processing node is required.For this distributed detection, we used diffusion based strategies Diffusion least mean square (DLMS) and Diffusion recursive least mean square(RLS) to find out distributed estimation of the parameter of interest. Distributed detection suitable in the wireless sensor network due to their robustness to node and link failure as compare to centralized scheme,,scalability and ability to save power and communication resources.The algorithm utilized is adaptive and track the variation in the active hypotheses.After the use for detection we analyze the performance of the proposed algorithm in term of probability og detection and probability of false alarm and find out the simulation result. We use some nonlinear techniques(huber loss , bi-square ) to reduce the effect of impulsive interference on the systems.

Computer Science,Engineering,Environmental Science
Distributed visual-target-surveillance system in wireless sensor networks

Xue Wang,Sheng Wang,Daowei Bi

DOI: https://doi.org/10.1109/TSMCB.2009.2013196

Abstract:A wireless sensor network (WSN) is a powerful unattended distributed measurement system, which is widely used in target surveillance because of its outstanding performance in distributed sensing and signal processing. This paper introduces a multiview visual-target-surveillance system in WSN, which can autonomously implement target classification and tracking with collaborative online learning and localization. The proposed system is a hybrid system of single-node and multinode fusion. It is constructed on a peer-to-peer (P2P)-based computing paradigm and consists of some simple but feasible methods for target detection and feature extraction. Importantly, a support-vector-machine-based semisupervised learning method is used to achieve online classifier learning with only unlabeled samples. To reduce the energy consumption and increase the accuracy, a novel progressive data-fusion paradigm is proposed for online learning and localization, where a feasible routing method is adopted to implement information transmission with the tradeoff between performance and cost. Experiment results verify that the proposed surveillance system is an effective, energy-efficient, and robust system for real-world application. Furthermore, the P2P-based progressive data-fusion paradigm can improve the energy efficiency and robustness of target surveillance.
Distributed Storage for Wireless Sensor Networks Based on Wavelet Hierarchy

Hui DONG,Jian-gang LU,You-xian SUN

DOI: https://doi.org/10.3969/j.issn.1004-1699.2007.06.037

2007-01-01

Abstract:The Wireless Sensor Network (WSN) is mainly used to gather data information from the monitored environments. However, due to the strict constrains on individual sensor node's resource, distributed storage and query must be taken. This paper proposed a novel distributed approach based on wavelet hierarchy, which uses wavelet compression techniques to construct a Spatial Architecture Tree and distribute the sensing data in various nodes. The simulation results showed that this approach perform well in database management; a) simplify the lifting scheme of traditional wavelet to eliminates extra transmission and computation, reduce energy costs significantly; b) exploit the inherent correlations that exist in or between sensor readings to store data efficiently; c) support highly efficient spatio-temporal query utilizing wavelet multi-resolution analysis and spatial self-similarity of Spatial Architecture Tree.
Distributed Active Learning.

Pengcheng Shen,Chunguang Li,Zhaoyang Zhang

DOI: https://doi.org/10.1109/access.2016.2572198

IF: 3.9

2016-01-01

IEEE Access

Abstract:Active learning aims at obtaining high-accuracy models with as a few labeled data as possible, by iteratively and elaborately selecting most valuable data to query labels during the learning process, thereby the cost of labeling data can be reduced. Most previous active learning approaches consider the situation of centralized processing, where all the unlabeled data are supposed to be gathered together in one place. Due to the development of distributed applications, distributed processing has attracted a lot of interests given the situation that data are distributed at different nodes over network. In this paper, we focus on the issue of distributed active learning (DAL) for the classification problem. We propose a fully decentralized active learning approach, which consists of two parts, namely, a distributed sample selection strategy and a distributed classification algorithm. The former helps nodes to cooperatively select data based on uncertainty, diversity, and representativeness of data. Due to the introducing of a randomized preselection method in the strategy, we can achieve diversity of the selected data without any information exchange among nodes. The latter helps each node to train its local multi-class classification model in a global sense without transmitting original data among nodes. We demonstrate the effectiveness of the proposed DAL approach on several real data sets. Simulation results show that the proposed approach can significantly reduce the number of labeled data needed for obtaining a high-accuracy classifier in distributed case.
Distributed Extreme Learning Machine for Nonlinear Learning over Network

Songyan Huang,Chunguang Li

DOI: https://doi.org/10.3390/e17020818

IF: 2.738

2015-01-01

Entropy

Abstract:Distributed data collection and analysis over a network are ubiquitous, especially over a wireless sensor network (WSN). To our knowledge, the data model used in most of the distributed algorithms is linear. However, in real applications, the linearity of systems is not always guaranteed. In nonlinear cases, the single hidden layer feedforward neural network (SLFN) with radial basis function (RBF) hidden neurons has the ability to approximate any continuous functions and, thus, may be used as the nonlinear learning system. However, confined by the communication cost, using the distributed version of the conventional algorithms to train the neural network directly is usually prohibited. Fortunately, based on the theorems provided in the extreme learning machine (ELM) literature, we only need to compute the output weights of the SLFN. Computing the output weights itself is a linear learning problem, although the input-output mapping of the overall SLFN is still nonlinear. Using the distributed algorithm to cooperatively compute the output weights of the SLFN, we obtain a distributed extreme learning machine (dELM) for nonlinear learning in this paper. This dELM is applied to the regression problem and classification problem to demonstrate its effectiveness and advantages.
Federated Learning over Wireless Fading Channels

Mohammad Mohammadi Amiri,Deniz Gunduz

DOI: https://doi.org/10.48550/arXiv.1907.09769

2020-02-11

Abstract:We study federated machine learning at the wireless network edge, where limited power wireless devices, each with its own dataset, build a joint model with the help of a remote parameter server (PS). We consider a bandwidth-limited fading multiple access channel (MAC) from the wireless devices to the PS, and propose various techniques to implement distributed stochastic gradient descent (DSGD). We first propose a digital DSGD (D-DSGD) scheme, in which one device is selected opportunistically for transmission at each iteration based on the channel conditions; the scheduled device quantizes its gradient estimate to a finite number of bits imposed by the channel condition, and transmits these bits to the PS in a reliable manner. Next, motivated by the additive nature of the wireless MAC, we propose a novel analog communication scheme, referred to as the compressed analog DSGD (CA-DSGD), where the devices first sparsify their gradient estimates while accumulating error, and project the resultant sparse vector into a low-dimensional vector for bandwidth reduction. Numerical results show that D-DSGD outperforms other digital approaches in the literature; however, in general the proposed CA-DSGD algorithm converges faster than the D-DSGD scheme and other schemes in the literature, and reaches a higher level of accuracy. We have observed that the gap between the analog and digital schemes increases when the datasets of devices are not independent and identically distributed (i.i.d.). Furthermore, the performance of the CA-DSGD scheme is shown to be robust against imperfect channel state information (CSI) at the devices. Overall these results show clear advantages for the proposed analog over-the-air DSGD scheme, which suggests that learning and communication algorithms should be designed jointly to achieve the best end-to-end performance in machine learning applications at the wireless edge.

Information Theory,Distributed, Parallel, and Cluster Computing,Machine Learning
Distributed Problem Solving in Sensor Networks

Weixiong Zhang,Zhidong Deng,Guandong Wang,Lars Wittenburg,Zhao Xing

DOI: https://doi.org/10.1145/544862.544971

2002-01-01

Abstract:The difficulties in developing large-scale, distributed sensor networks are discussed and our recent experience in developing and analyzing distributed problem solving methods for applications in sensor networks is overviewed.
Self-Organized Low-Power IoT Networks: A Distributed Learning Approach

Amin Azari,Cicek Cavdar

DOI: https://doi.org/10.1109/glocom.2018.8647894

2018-12-01

Abstract:Enabling large-scale energy-efficient Internet-of-things (IoT) connectivity is an essential step towards realization of networked society. While legacy wide-area wireless systems are highly dependent on network-side coordination, the level of consumed energy in signaling, as well as the expected increase in the number of IoT devices, makes such centralized approaches infeasible in future. Here, we address this problem by self-coordination for IoT networks through learning from past communications. To this end, we first study low-complexity distributed learning approaches applicable in IoT communications. Then, we present a learning solution to adapt communication parameters of devices to the environment for maximizing energy efficiency and reliability in data transmissions. Furthermore, leveraging tools from stochastic geometry, we evaluate the performance of proposed distributed learning solution against the centralized coordination. Finally, we analyze the interplay amongst energy efficiency, reliability of communications against noise and interference over data channel, and reliability against adversarial interference over data and feedback channels. The simulation results indicate that compared to the state of the art approaches, both energy efficiency and reliability in IoT communications could be significantly improved using the proposed learning approach. These promising results, which are achieved using lightweight learning, make our solution favorable in many low-cost low-power IoT applications.
Wireless Sensor Networks: an Information Processing Approach

Feng Zhao,Leonidas Guibas

2004-01-01

Abstract:Ch 1 Intro. Ch 2 Canonical Problem: Localization and Tracking Ch 3 Networking Sensor Networks Ch 4 Synchronization and Localization Ch 5 Sensor Tasking and Control Ch 6 Sensor Network Database Ch 7 Sensor Network Platforms and Tools Ch 8 Application and Future Direction
Consistency in Models for Communication Constrained Distributed Learning

J. B. Predd,S. R. Kulkarni,H. V. Poor

DOI: https://doi.org/10.1007/978-3-540-27819-1_31

2004-01-01

Abstract:Motivated by sensor networks and other distributed settings, several models for distributed learning are presented. The models differ from classical works in statistical pattern recognition by allocating observations of an i.i.d. sampling process amongst members of a network of learning agents. The agents are limited in their ability to communicate to a fusion center; the amount of information available for classification or regression is constrained. For several simple communication models, questions of universal consistency are addressed; i.e., the asymptotics of several agent decision rules and fusion rules are considered in both binary classification and regression frameworks. These models resemble distributed environments and introduce new questions regarding universal consistency. Insofar as these models offer a useful picture of distributed scenarios, this paper considers whether the guarantees provided by Stone’s Theorem in centralized environments hold in distributed settings.
Federated Learning for Distributed Spectrum Sensing in NextG Communication Networks

Yi Shi,Yalin E. Sagduyu,Tugba Erpek

DOI: https://doi.org/10.48550/arXiv.2204.03027

2022-04-06

Networking and Internet Architecture

Abstract:NextG networks are intended to provide the flexibility of sharing the spectrum with incumbent users and support various spectrum monitoring tasks such as anomaly detection, fault diagnostics, user equipment identification, and authentication. A network of wireless sensors is needed to monitor the spectrum for signal transmissions of interest over a large deployment area. Each sensor receives signals under a specific channel condition depending on its location and trains an individual model of a deep neural network (DNN) accordingly to classify signals. To improve the accuracy, individual sensors may exchange sensing data or sensor results with each other or with a fusion center (such as in cooperative spectrum sensing). In this paper, distributed federated learning over a multi-hop wireless network is considered to collectively train a DNN for signal identification. In distributed federated learning, each sensor broadcasts its trained model to its neighbors, collects the DNN models from its neighbors, and aggregates them to initialize its own model for the next round of training. Without exchanging any spectrum data, this process is repeated over time such that a common DNN is built across the network while preserving the privacy associated with signals collected at different locations. Signal classification accuracy and convergence time are evaluated for different network topologies (including line, star, ring, grid, and random networks) and packet loss events. Then, the reduction of communication overhead and energy consumption is considered with random participation of sensors in model updates. The results show the feasibility of extending cooperative spectrum sensing over a general multi-hop wireless network through federated learning and indicate its robustness to wireless network effects, thereby sustaining high accuracy with low communication overhead and energy consumption.

Distributed Learning in Wireless Sensor Networks

Distributed Storage and Parallel Processing in Large-Scale Wireless Sensor Networks.

Learn to Sense: a Meta-learning Based Distributed Sensing Algorithm in Wireless Sensor Networks

Distributed Learning for Wireless Communications: Methods, Applications and Challenges

Learn to Sense: A Meta-Learning-Based Sensing and Fusion Framework for Wireless Sensor Networks

Distributed Learning in Wireless Networks: Recent Progress and Future Challenges

Decentralized learning for wireless communications and networking

Distributed Learning for Time-varying Networks: A Scalable Design

Distributed Semi-Supervised Metric Learning.

Distributed Machine Learning for Wireless Communication Networks: Techniques, Architectures, and Applications

Diffusion Based Distributed Detection in Wireless Sensor Network

Distributed visual-target-surveillance system in wireless sensor networks

Distributed Storage for Wireless Sensor Networks Based on Wavelet Hierarchy

Distributed Active Learning.

Distributed Extreme Learning Machine for Nonlinear Learning over Network

Federated Learning over Wireless Fading Channels

Distributed Problem Solving in Sensor Networks

Self-Organized Low-Power IoT Networks: A Distributed Learning Approach

Wireless Sensor Networks: an Information Processing Approach

Consistency in Models for Communication Constrained Distributed Learning

Federated Learning for Distributed Spectrum Sensing in NextG Communication Networks