Membrane association with multiple calcium ions: vitamin-K-dependent proteins, annexins and pentraxins.

G. Nelsestuen,B. Ostrowski

DOI: https://doi.org/10.1016/S0959-440X(99)80060-8

IF: 7.786

1999-08-01

Current Opinion in Structural Biology

Abstract:

What problem does this paper attempt to address?

Penalized Reinforcement Learning-Based Energy-Efficient UAV-RIS Assisted Maritime Uplink Communications Against Jamming

Kailong Lin,Helin Yang,Mengting Zheng,Liang Xiao,Chongwen Huang,Dusit Niyato

DOI: https://doi.org/10.1109/tvt.2024.3406896

2024-01-01

Abstract:This paper proposes an aerial reconfigurable intelligent surface (RIS)-assisted maritime communication network (MCN) against jamming, where a hybrid RIS mounted on an unmanned aerial vehicle (UAV) is deployed to adjust and amplify signals as a communication relay. This paper aims to maximize the system energy efficiency (EE) considering jamming noise and quality of service (QoS) constraints of maritime users (MUs) by jointly optimizing resource scheduling of hybrid UAV-RIS including phase shift factor, amplitude coefficient, active RIS ratio, and transmit power of MUs. Due to dynamic and unknown environments, we design a novel approach based on penalized deep reinforcement learning (DRL) to solve the optimization problem, in which a penalized point policy difference authentic boundary proximal policy optimization (P3D-ABPPO) approach is proposed to enhance the learning capacity and system EE performance. Simulation results demonstrate that our proposed P3D-ABPPO-based hybrid UAV-RIS resource scheduling approach can significantly improve the system EE compared with other traditional DRL approaches. For example, the proposed P3D-ABPPO approach achieves system EE improvements of 10.38% compared with the PPO approach.
DDQN with Prioritized Experience Replay-Based Optimized Geographical Routing Protocol of Considering Link Stability and Energy Prediction for UANET

Yanan Zhang,Hongbing Qiu

DOI: https://doi.org/10.3390/s22135020

IF: 3.9

2022-07-04

Sensors

Abstract:Unmanned aerial vehicles (UAVs) are important equipment for efficiently executing search and rescue missions in disaster or air-crash scenarios. Each node can communicate with the others by a routing protocol in UAV ad hoc networks (UANETs). However, UAV routing protocols are faced with the challenges of high mobility and limited node energy, which hugely lead to unstable link and sparse network topology due to premature node death. Eventually, this severely affects network performance. In order to solve these problems, we proposed the deep-reinforcement-learning-based geographical routing protocol of considering link stability and energy prediction (DSEGR) for UANETs. First of all, we came up with the link stability evaluation indicator and utilized the autoregressive integrated moving average (ARIMA) model to predict the residual energy of neighbor nodes. Then, the packet forward process was modeled as a Markov Decision Process, and according to a deep double Q network with prioritized experience replay to learn the routing-decision process. Meanwhile, a reward function was designed to obtain a better convergence rate, and the analytic hierarchy process (AHP) was used to analyze the weights of the considered factors in the reward function. Finally, to verify the effectiveness of DSEGR, we conducted simulation experiments to analyze network performance. The simulation results demonstrate that our proposed routing protocol remarkably outperforms others in packet delivery ratio and has a faster convergence rate.

engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation
Multi-Objective Optimized GPSR Intelligent Routing Protocol for UAV Clusters

Hao Chen,Fan Luo,Jianguo Zhou,Yanming Dong

DOI: https://doi.org/10.3390/math12172672

IF: 2.4

2024-08-31

Mathematics

Abstract:Unmanned aerial vehicle (UAV) clusters offer significant potential in civil, military, and commercial fields due to their flexibility and cooperative capabilities. However, characteristics such as dynamic topology and limited energy storage bring challenges to the design of routing protocols for UAV networks. This study leverages the Deep Double Q-Learning Network (DDQN) algorithm to optimize the traditional Greedy Perimeter Stateless Routing (GPSR) protocol, resulting in a multi-objective optimized GPSR routing protocol (DDQN-MTGPSR). By constructing a multi-objective routing optimization model through cross-layer data fusion, the proposed approach aims to enhance UAV network communication performance comprehensively. In addition, this study develops the above DDQN-MTGPSR intelligent routing algorithm based on the NS-3 platform and uses an artificial intelligence framework. In order to verify the effectiveness of the DDQN-MTGPSR algorithm, it is simulated and compared with the traditional ad hoc routing protocols, and the experimental results show that compared with the GPSR protocol, the DDQN-MTGPSR has achieved significant optimization in the key metrics such as the average end-to-end delay, packet delivery rate, node average residual energy variance and percentage of node average residual energy. In high dynamic scenarios, the above indicators were optimized by 20.05%, 12.72%, 0.47%, and 50.15%, respectively, while optimizing 36.31%, 26.26%, 8.709%, and 69.3% in large-scale scenarios, respectively.

mathematics
A Q-Learning-Based Hierarchical Routing Protocol with Unequal Clustering for Underwater Acoustic Sensor Networks

Yufan Yuan,Meiyan Liu,Xiaoxiao Zhuo,Yan Wei,Xingbin Tu,Fengzhong Qu

DOI: https://doi.org/10.1109/jsen.2022.3232614

IF: 4.3

2023-01-01

IEEE Sensors Journal

Abstract:Underwater acoustic sensor networks (UASNs) have emerged as a viable networking approach due to their numerous aquatic applications in recent years. As a vital component of UASNs, routing protocols are essential for ensuring reliable data transmissions and extending the longevity of UASNs. Recently, several clustering-based routing protocols have been proposed to reduce energy consumption and overcome the resource constraints of deployed sensor nodes. However, they rarely consider the hot-spots' problem and the sink node isolation problem in the multihop underwater sensor networks. In this article, we propose a Q-learning-based hierarchical routing protocol with unequal clustering (QHUC) for determining an effective data forwarding path to extend the lifespan of UASNs. First, a hierarchical network structure is constructed for initialization. Then, a combination of unequal clustering and the Q-learning algorithm is applied to the hierarchical structure to disperse the remaining energy more evenly throughout the network. With the use of the Q-learning algorithm, a global optimal CH and next-hop can be determined better than a greedy one. In addition, the Q value that guarantees the optimal routing decisions can be computed without incurring any additional costs by combining the Q-learning algorithmwith clustering. The simulation results showthat the QHUC can achieve efficient routing and prolong the network lifetime significantly.
UBER: UAV-Based Energy-Efficient Reconfigurable Routing Scheme for Smart Wireless Livestock Sensor Network

Mohammed A Alanezi,Abdulazeez F Salami,Yusuf A Sha'aban,Houssem R E H Bouchekara,Mohammad S Shahriar,Mohammed Khodja,Mostafa K Smail

DOI: https://doi.org/10.3390/s22166158

2022-08-17

Abstract:This paper addresses coverage loss and rapid energy depletion issues for wireless livestock sensor networks by proposing a UAV-based energy-efficient reconfigurable routing (UBER) scheme for smart wireless livestock sensor networking applications. This routing scheme relies on a dynamic residual energy thresholding strategy, robust cluster-to-UAV link formation, and UAV-assisted network coverage and recovery mechanism. The performance of UBER was evaluated using low, normal and high UAV altitude scenarios. Performance metrics employed for this analysis are network stability (NST), load balancing ratio (LBR), and topology fluctuation effect ratio (TFER). Obtained results demonstrated that operating with a UAV altitude of 230 m yields gains of 31.58%, 61.67%, and 75.57% for NST, LBR, and TFER, respectively. A comparative performance evaluation of UBER was carried out with respect to hybrid heterogeneous routing (HYBRID) and mobile sink using directional virtual coordinate routing (MS-DVCR). The performance indicators employed for this comparative analysis are energy consumption (ENC), network coverage (COV), received packets (RPK), SN failures detected (SNFD), route failures detected (RFD), routing overhead (ROH), and end-to-end delay (ETE). With regard to the best-obtained results, UBER recorded performance gains of 46.48%, 47.33%, 15.68%, 19.78%, 46.44%, 29.38%, and 58.56% over HYBRID and MS-DVCR in terms of ENC, COV, RPK, SNFD, RFD, ROH, and ETE, respectively. The results obtained demonstrated that the UBER scheme is highly efficient with competitive performance against the benchmarked CBR schemes.
Performance Evaluation of Standard and Modified OLSR Protocols for Uncoordinated UAV Ad-Hoc Networks in Search and Rescue Environments

Ali H. Wheeb,Rosdiadee Nordin,Dimitris Kanellopoulos,Asma' Abu Samah

DOI: https://doi.org/10.3390/electronics12061334

IF: 2.9

2023-03-12

Electronics

Abstract:Widespread usage of unmanned aerial vehicles (UAVs) in new and emerging applications needs dynamic and adaptive networking. The development of routing protocols for UAV ad hoc networks faces numerous issues because of the unique characteristics of UAVs, such as rapid mobility, frequent changes in network topology, and limited energy consumption. The Optimized Link State Routing (OLSR) protocol seems to be a promising solution as it offers improved delay performance. It is expected that OLSR will satisfy the strict demands of real-time UAV applications such as "search and rescue" (SAR) missions as it involves the most recent update of routing information. The classical OLSR routing protocol and its enhanced versions, D-OLSR, ML-OLSR, and P-OLSR, use different techniques to make an appropriate decision for routing packets. These routing techniques consider the quality of a wireless link, type of antenna, load, and mobility-aware mechanism to select the best UAV to send the message to the destination. This study evaluates and examines the performance of the original and modified OLSR routing protocols in UAV ad hoc networks for three SAR scenarios: (1) increasing mobility, (2) increasing scalability, and (3) increasing the allowed space of UAVs. It analyzes and validates the performance of the four OLSR-based routing protocols. It determines the best OSLR routing protocol by taking into account the packet delivery ratio, latency, energy consumption, and throughput. The four routing protocols and the SAR scenarios were simulated using NS-3.32. Based on the simulation results, ML-OLSR outperforms OLSR, D-OLSR, and P-OLSR in the considered measures.

engineering, electrical & electronic,computer science, information systems,physics, applied
Improved Q-learning based Multi-hop Routing for UAV-Assisted Communication

N P Sharvari,Dibakar Das,Jyotsna Bapat,Debabrata Das

2024-08-17

Abstract:Designing effective Unmanned Aerial Vehicle(UAV)-assisted routing protocols is challenging due to changing topology, limited battery capacity, and the dynamic nature of communication environments. Current protocols prioritize optimizing individual network parameters, overlooking the necessity for a nuanced approach in scenarios with intermittent connectivity, fluctuating signal strength, and varying network densities, ultimately failing to address aerial network requirements comprehensively. This paper proposes a novel, Improved Q-learning-based Multi-hop Routing (IQMR) algorithm for optimal UAV-assisted communication systems. Using Q(\lambda) learning for routing decisions, IQMR substantially enhances energy efficiency and network data throughput. IQMR improves system resilience by prioritizing reliable connectivity and inter-UAV collision avoidance while integrating real-time network status information, all in the absence of predefined UAV path planning, thus ensuring dynamic adaptability to evolving network conditions. The results validate IQMR's adaptability to changing system conditions and superiority over the current techniques. IQMR showcases 36.35\% and 32.05\% improvements in energy efficiency and data throughput over the existing methods.

Networking and Internet Architecture,Machine Learning
Reinforcement Learning Placement Algorithm for Optimization of UAV Network in Wireless Communication

Sahar Baghdady,Seyed Masoud Mirrezaei,Rashid Mirzavand

DOI: https://doi.org/10.1109/access.2024.3374384

IF: 3.9

2024-03-19

IEEE Access

Abstract:Unmanned aerial vehicles (UAVs) or drones have attracted much attention in wireless communication networks because of their agility, unique flexibility, low cost of implementation, and the high strength of the line-of-sight (LoS) channel. They are widely used in different scenarios. In many environments with complex geographical conditions or in situations where areas are affected by natural disasters, UAVs can be used as base stations (BSs) for downlink ground users. The article proposes a communication system using multiple UAV-mounted BSs to improve coverage rate and minimize the number of required UAVs. The problem is formulated as a mixed-integer programming problem with constraints on the quality of service (QoS) and serviceability of each UAV. A three-step method is developed to solve the problem, which includes deriving the maximum service radius of UAVs using the Karush-Kuhn-Tucker (KKT) method, minimizing the number of required UAVs using reinforcement learning (RL) algorithm, and designing the three-dimensional (3D) position and frequency band of each UAV to increase signal power and reduce interference. The simulation results show that the RLP algorithm outperforms other algorithms in terms of coverage rate, user clustering, increased signal, reduced interference, and processing time required to find the optimal solution.

computer science, information systems,telecommunications,engineering, electrical & electronic
A Robust Routing Protocol in Cognitive Unmanned Aerial Vehicular Networks

Anatte Rozario,Ehasan Ahmed,Nafees Mansoor

DOI: https://doi.org/10.3390/s24196334

IF: 3.9

2024-10-01

Sensors

Abstract:The adoption of UAVs in defence and civilian sectors necessitates robust communication networks. This paper presents a routing protocol for Cognitive Radio Unmanned Aerial Vehicles (CR-UAVs) in Flying Ad-hoc Networks (FANETs). The protocol is engineered to optimize route selection by considering crucial parameters such as distance, speed, link quality, and energy consumption. A standout feature is the introduction of the Central Node Resolution Factor (CNRF), which enhances routing decisions. Leveraging the Received Signal Strength Indicator (RSSI) enables accurate distance estimation, crucial for effective routing. Moreover, predictive algorithms are integrated to tackle the challenges posed by high mobility scenarios. Security measures include the identification of malicious nodes, while the protocol ensures resilience by managing multiple routes. Furthermore, it addresses route maintenance and handles link failures efficiently, cluster formation, and re-clustering with joining and leaving new nodes along with the predictive algorithm. Simulation results showcase the protocol's self-comparison under different packet sizes, particularly in terms of end-to-end delay, throughput, packet delivery ratio, and normalized routing load. However, superior performance compared to existing methods, particularly in terms of throughput and packet transmission delay, underscoring its potential for widespread adoption in both defence and civilian UAV applications.

engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation
Efficient UAVs Deployment and Resource Allocation in UAV-Relay Assisted Public Safety Networks for Video Transmission

Naveed Khan,Ayaz Ahmad,Abdul Wakeel,Zeeshan Kaleem,Bushra Rashid,Waqas Khalid

DOI: https://doi.org/10.1109/access.2024.3350138

IF: 3.9

2024-01-01

IEEE Access

Abstract:Wireless communication highly depends on the cellular ground base station (GBS). A failure of the cellular GBS, fully or partially, during natural or man-made disasters creates a communication gap in the disaster-affected areas. In such situations, public safety communication (PSC) can significantly save the national infrastructure, property, and lives. Throughout emergencies, the PSC can provide mission-critical communication and video transmission services in the affected area. Unmanned aerial vehicles (UAVs) as flying base stations (UAV-BSs) are particularly suitable for PSC services as they are flexible, mobile, and easily deployable. This manuscript considers a multi-UAV-assisted PSC network with an observational UAV receiving videos from the affected area’s ground users (AGUs) and transmitting them to the nearby GBS via a relay UAV. The objective of the proposed study is to maximize the average utility of the video streams generated by the AGUs upon reaching the GBS. This is achieved by optimizing the positions of the observational and relay UAVs, as well as the distribution of communication resources, such as bandwidth, and transmit power, while satisfying the system-designed constraints, such as transmission rate, rate outage probability, transmit power budget, and available bandwidth. To this end, a joint UAVs placement and resource allocation problem is mathematically formulated. The proposed problem poses a significant challenge for a solution. Considering the block coordinate descent and successive convex approximation techniques, an efficient iterative algorithm is proposed. Finally, simulation results are provided which show that our proposed approach outperforms the existing methods.

computer science, information systems,telecommunications,engineering, electrical & electronic
Integrated clustering and routing design and triangle path optimization for UAV-assisted wireless sensor networks

Shao Liwei,Qian Liping,Wu Mengru,Wu Yuan

DOI: https://doi.org/10.23919/jcc.fa.2023-0495.202404

2024-04-27

China Communications

Abstract:With the development of the Internet of Things (IoT), it requires better performance from wireless sensor networks (WSNs), such as larger coverage, longer lifetime, and lower latency. However, a large amount of data generated from monitoring and long-distance transmission places a heavy burden on sensor nodes with the limited battery power. For this, we investigate an unmanned aerial vehicles assisted mobile wireless sensor network (UAV-assisted WSN) to prolong the network lifetime in this paper. Specifically, we use UAVs to assist the WSN in collecting data. In the current UAV-assisted WSN, the clustering and routing schemes are determined sequentially. However, such a separate consideration might not maximize the lifetime of the whole WSN due to the mutual coupling of clustering and routing. To efficiently prolong the lifetime of the WSN, we propose an integrated clustering and routing scheme that jointly optimizes the clustering and routing together. In the whole network space, it is intractable to efficiently obtain the optimal integrated clustering and routing scheme. Therefore, we propose the Monte-Las search strategy based on Monte Carlo and Las Vegas ideas, which can generate the chain matrix to guide the algorithm to find the solution faster. Unnecessary point-to-point collection leads to long collection paths, so a triangle optimization strategy is then proposed that finds a compromise path to shorten the collection path based on the geometric distribution and energy of sensor nodes. To avoid the coverage hole caused by the death of sensor nodes, the deployment of mobile sensor nodes and the preventive mechanism design are indispensable. An emergency data transmission mechanism is further proposed to reduce the latency of collecting the latency-sensitive data due to the absence of UAVs. Compared with the existing schemes, the proposed scheme can prolong the lifetime of the UAV-assisted WSN at least by 360%, and shorten the collection path of UAVs by 56.24%.

telecommunications
3D UAV Trajectory Design for Fair and Energy-Efficient Communication: A Deep Reinforcement Learning Technique

Shahid Rasool,Irfan Ullah,Abid Ali,Ishtiaq Ahmad

2023-01-27

Abstract:In different situations, like disaster communication and network connectivity for rural locations, unmanned aerial vehicles (UAVs) could indeed be utilized as airborne base stations to improve both the functionality and coverage of communication networks. Ground users can employ mobile UAVs to establish communication channels and deliver packages. UAVs, on the other hand, have restricted transmission capabilities and fuel supplies. They can't always cover the full region or continue to fly for a long time, especially in a huge territory. Controlling a swarm of UAVs to yield a relatively long communication coverage while maintaining connectivity and limiting energy usage is so difficult. We use modern deep reinforcement learning (DRL) for UAV connectivity to provide an innovative and extremely energy-efficient DRL-based algorithm. The proposed method: 1) enhances novel energy efficiency while taking into account communications throughput, energy consumption, fairness, and connectivity; 2) evaluates the environment and its dynamics; and 3) makes judgments using strong deep neural networks. For performance evaluation, we have performed comprehensive simulations. In terms of energy consumption and fairness, simulation results show that the DRL-based algorithm consistently outperforms two commonly used baseline techniques.

Networking and Internet Architecture,Systems and Control
Energy efficient gateway based routing with maximized node coverage in a UAV assisted wireless sensor network

Bilal Ahmad,Masroor Ahmed,Nadeem Anjum,Masood Ur Rehman,Naeem Ramzan

DOI: https://doi.org/10.1371/journal.pone.0295615

IF: 3.7

2023-12-27

PLoS ONE

Abstract:Ad-hoc wireless sensor networks face challenges of optimized node deployment for maximizing coverage and efficiently routing data to control centers in post disaster events. These challenges impact the outcome for extending the lifetime of wireless sensor networks. This study presents a uav assisted reactive zone based EHGR (energy efficient hierarchical gateway routing protocol) that is deployed in a situation where the natural calamity has caused communication and infrastructure damage to a major portion of the sensor network. EHGR is a hybrid multi layer routing protocol for large heterogeneous sensor nodes (smart nodes, basic nodes, user handheld devices etc.) EHGR is tailored to meet two important concerns for a disaster hit wsn ie. optimized deployment and energy efficient routing. The first part of EGHR focuses on maximized coverage during node deployments. Maximized coverage is an important aspect to be considered during the event of disaster since most of the nodes loose coverage and are detached from the wireless sensor network. The first part of EHGR uses state of the art game theory approach to build a model that maximizes the coverage of nodes during the deployment phase from all participating entities i.e. nodes and uavs. Rather than fixing the cluster head as is the case in traditional cluster-based approaches EHGR uses the energy centroid nodes. Energy centroid nodes evolve on the basis of aggregated energy of the zone. This approach is superior to simply electing cluster head nodes on the basis of some probability function. The nodes that fail to achieve any successful outcome from the game theory matching model fail to get any association. These nodes will use multi hop d2d relay approach to reach the energy centroid nodes. Gateway relay nodes used with the game theory approach during the deployment of the uav assisted wsn improves the overall coverage by 25% against traditional leach based hierarchical approaches. Once the optimum deployment phase is completed the routing phase is initiated. Aggregated data is sent by the energy centroid nodes from the ECN nodes to the servicing micro controller enabled un manned aerial vehicles. The routing process places partial burden of zone formation and data transmission to the control center for each phase on the servicing uavs. Energy centroid nodes engage only in the data aggregation process and transmission of data to servicing uav. Servicing-uavs reduce energy dissipated of the entire zone which result in gradual decrease of energy for the zone thus increasing the network lifetime. Node deployment phase and the routing phase of EHGR utilize the computations provide by the mirco controller enabled unmanned aerial vehicles such that the computationally intensive calculations are offloaded to the servicing uav. Experiment results indicate an increase in the first dead node report, half dead node report, and last dead node report. Network lifetime is extended to approximately 1800 rounds which is an increase by ratio of 100% against the traditional leach approach and increase by 50% percent against the latest approaches as highlighted in the literature.
An Innovative Clustering Hierarchical Protocol for Data Collection from Remote Wireless Sensor Networks Based Internet of Things Applications

Syed Luqman Shah,Ziaul Haq Abbas,Ghulam Abbas,Fazal Muhammad,Aseel Hussien,Thar Baker

DOI: https://doi.org/10.3390/s23125728

2023-06-19

Abstract:Recently, unmanned aerial vehicles (UAVs) have emerged as a viable solution for data collection from remote Internet of Things (IoT) applications. However, the successful implementation in this regard necessitates the development of a reliable and energy-efficient routing protocol. This paper proposes a reliable and an energy-efficient UAV-assisted clustering hierarchical (EEUCH) protocol designed for remote wireless sensor networks (WSNs) based IoT applications. The proposed EEUCH routing protocol facilitates UAVs to collect data from ground sensor nodes (SNs) that are equipped with wake-up radios (WuRs) and deployed remotely from the base station (BS) in the field of interest (FoI). During each round of the EEUCH protocol, the UAVs arrive at the predefined hovering positions at the FoI, perform clear channel assignment, and broadcast wake-up calls (WuCs) to the SNs. Upon receiving the WuCs by the SNs' wake-up receivers, the SNs perform carrier sense multiple access/collision avoidance before sending joining requests to ensure reliability and cluster-memberships with the particular UAV whose WuC is received. The cluster-member SNs turn on their main radios (MRs) for data packet transmission. The UAV assigns time division multiple access (TDMA) slots to each of its cluster-member SNs whose joining request is received. Each SN must send the data packets in its assigned TDMA slot. When data packets are successfully received by the UAV, it sends acknowledgments to the SNs, after which the SNs turn off their MRs, completing a single round of the protocol. The proposed EEUCH routing protocol with WuR eliminates the issue of cluster overlapping, improves the overall performance, and increases network stability time by a factor of 8.7. It also improves energy efficiency by a factor of 12.55, resulting in a longer network lifespan compared to Low Energy Adaptive Clustering Hierarchy (LEACH) protocol. Moreover, EEUCH collects 5.05 times more data from the FoI than LEACH. These results are based on simulations in which the EEUCH protocol outperformed the existing six benchmark routing protocols proposed for homogeneous, two-tier, and three-tier heterogeneous WSNs.
Traffic and Obstacle-aware UAV Positioning in Urban Environments Using Reinforcement Learning

Kamran Shafafi,Manuel Ricardo,Rui Campos

2024-08-08

Abstract:Unmanned Aerial Vehicles (UAVs) are suited as cost-effective and adaptable platforms for carrying Wi-Fi Access Points (APs) and cellular Base Stations (BSs). Implementing aerial networks in disaster management scenarios and crowded areas can effectively enhance Quality of Service (QoS). In such environments, maintaining Line-of-Sight (LoS), especially at higher frequencies, is crucial for ensuring reliable communication networks with high capacity, particularly in environments with obstacles. The main contribution of this paper is a traffic- and obstacle-aware UAV positioning algorithm named Reinforcement Learning-based Traffic and Obstacle-aware Positioning Algorithm (RLTOPA), for such environments. RLTOPA determines the optimal position of the UAV by considering the positions of ground users, the coordinates of obstacles, and the traffic demands of users. This positioning aims to maximize QoS in terms of throughput by ensuring optimal LoS between ground users and the UAV. The network performance of the proposed solution, characterized in terms of mean delay and throughput, was evaluated using the ns- 3 simulator. The results show up to 95% improvement in aggregate throughput and 71% in delay without compromising fairness.

Networking and Internet Architecture,Signal Processing
Reinforcement Learning based Energy-Efficient Fast Routing for FANETs

Jieling Li,Liang Xiao,Xuchen Qi,Zefang Lv,Qiaoxin Chen,Yong-Jin Liu

DOI: https://doi.org/10.1109/TCOMM.2024.3409561

IF: 6.166

2024-01-01

IEEE Transactions on Communications

Abstract:Reinforcement learning (RL) based flying ad-hoc network (FANET) routing enables unmanned aerial vehicles (UAVs) to choose the next-hop to increase the packet delivery ratio, but the routing latency and energy consumption have to be further reduced over inaccurate feedback for large-scale networks. In this paper, we propose an RL based energy-efficient fast routing for each UAV to choose the forwarding decision and the power. Based on the state consisting of the battery level, channel conditions and forwarding decisions of the one-hop neighbors, the routing policy is chosen to enhance the utility as the weighted sum of the delivery success indicator, the latency and the energy consumption. The number of the latency violations and the learning parameters shared among the one-hop neighbors are exploited in the update of the routing policy distribution following the latency constraint with the reduced energy consumption. The deep neural networks address the state quantization error of the latency and the channel gain for UAVs with high mobility under large-scale networks. The performance bound regarding the end-to-end latency and the energy consumption is derived in terms of network topology and channel gain based on the packet forwarding game. The performance gain over the benchmark is provided via both simulation and experimental results.
Deep Reinforcement Learning for UAV Routing in the Presence of Multiple Charging Stations

Mingfeng Fan,Yaoxin Wu,Tianjun Liao,Zhiguang Cao,Hongliang Guo,Guillaume Sartoretti,Guohua Wu

DOI: https://doi.org/10.1109/tvt.2022.3232607

IF: 6.8

2022-01-01

IEEE Transactions on Vehicular Technology

Abstract:Deploying Unmanned Aerial Vehicles (UAVs) for traffic monitoring has been a hotspot given their flexibility and broader view. However, a UAV is usually constrained by battery capacity due to limited payload. On the other hand, the development of wireless charging technology has allowed UAVs to replenish energy from charging stations.In this paper, we study a UAV routing problem in the presence of multiple charging stations (URPMCS) with the objective of minimizing the total distance traveled by the UAV during traffic monitoring. We present a deep reinforcement learning based method, where a multi-head heterogeneous attention mechanism is designed to facilitate learning a policy that automatically and sequentially constructs the route, while taking the energy consumption into account. In our method, two types of attentions are leveraged to learn the relations between monitoring targets and charging station nodes, adopting an encoder-decoder-like policy network. Moreover, we also employ a curriculum learning strategy to enhance generalization to different numbers of charging stations. Computational results show that our method outperforms conventional algorithms with higher solution quality (except for exact methods such as Gurobi) and shorter runtime in general, and also exhibits strong generalized performance on problem instances with different distributions and sizes.

telecommunications,engineering, electrical & electronic,transportation science & technology
Multi-UAV Multi-RIS QoS-Aware Aerial Communication Systems using DRL and PSO

Marwan Dhuheir,Aiman Erbad,Ala Al-Fuqaha,Mohsen Guizani

2024-06-17

Abstract:Recently, Unmanned Aerial Vehicles (UAVs) have attracted the attention of researchers in academia and industry for providing wireless services to ground users in diverse scenarios like festivals, large sporting events, natural and man-made disasters due to their advantages in terms of versatility and maneuverability. However, the limited resources of UAVs (e.g., energy budget and different service requirements) can pose challenges for adopting UAVs for such applications. Our system model considers a UAV swarm that navigates an area, providing wireless communication to ground users with RIS support to improve the coverage of the UAVs. In this work, we introduce an optimization model with the aim of maximizing the throughput and UAVs coverage through optimal path planning of UAVs and multi-RIS phase configurations. The formulated optimization is challenging to solve using standard linear programming techniques, limiting its applicability in real-time decision-making. Therefore, we introduce a two-step solution using deep reinforcement learning and particle swarm optimization. We conduct extensive simulations and compare our approach to two competitive solutions presented in the recent literature. Our simulation results demonstrate that our adopted approach is 20 \% better than the brute-force approach and 30\% better than the baseline solution in terms of QoS.

Signal Processing,Machine Learning
Resource Allocation in UAV-Assisted Networks: A Clustering-Aided Reinforcement Learning Approach

Shiyang Zhou,Yufan Cheng,Xia Lei,Qihang Peng,Jun Wang,Shaoqian Li

DOI: https://doi.org/10.1109/tvt.2022.3189552

IF: 6.8

2022-11-15

IEEE Transactions on Vehicular Technology

Abstract:As an aerial base station, unmanned aerial vehicle (UAV) has been considered as a promising technology to assist future wireless communications due to its flexible, swift and low cost features, where resource allocation is the basis for ensuring energy-efficient UAV-assisted networks. This paper formulates a joint optimization problem of user association, UAV trajectory design and power control to maximize the channel capacity among all ground users at a limited power level in a downlink transmission. To tackle the mixed-integer non-linear programming problem, this paper proposes a clustering-aided reinforcement learning approach consisting of three consecutive stages. Firstly, modified expectation-maximization unsupervised learning algorithm is investigated to cluster the ground users, which reduces the dimensions and hence, the association complexity is reduced as well. Then, Kuhn-Munkres algorithm is incorporated for user association, which associates a UAV with the ground users via matching to the cluster, and assigns the UAVs to the centroid of the matching cluster for pre-placement, with the aim of speeding up the convergence of the following deep reinforcement learning algorithm. Finally, a multi-agent twin delayed deep deterministic (MATD3) policy gradient is proposed to solve the non-convex sub-problem, which determines the transmit power and designs the fine-tuned trajectory of UAVs. By incorporating low-bias value estimation technique, the reward of the proposed MATD3 algorithm is improved. Simulation results have demonstrated that our proposed approach achieves higher reward as well as converging faster than existing reinforcement algorithms. Besides, the clustering-aided reinforcement learning has lower computational complexity than the benchmark schemes.

telecommunications,engineering, electrical & electronic,transportation science & technology

Membrane association with multiple calcium ions: vitamin-K-dependent proteins, annexins and pentraxins.

Penalized Reinforcement Learning-Based Energy-Efficient UAV-RIS Assisted Maritime Uplink Communications Against Jamming

DDQN with Prioritized Experience Replay-Based Optimized Geographical Routing Protocol of Considering Link Stability and Energy Prediction for UANET

Multi-Objective Optimized GPSR Intelligent Routing Protocol for UAV Clusters

A Q-Learning-Based Hierarchical Routing Protocol with Unequal Clustering for Underwater Acoustic Sensor Networks

UBER: UAV-Based Energy-Efficient Reconfigurable Routing Scheme for Smart Wireless Livestock Sensor Network

Performance Evaluation of Standard and Modified OLSR Protocols for Uncoordinated UAV Ad-Hoc Networks in Search and Rescue Environments

Improved Q-learning based Multi-hop Routing for UAV-Assisted Communication

Reinforcement Learning Placement Algorithm for Optimization of UAV Network in Wireless Communication

A Robust Routing Protocol in Cognitive Unmanned Aerial Vehicular Networks

Efficient UAVs Deployment and Resource Allocation in UAV-Relay Assisted Public Safety Networks for Video Transmission

Integrated clustering and routing design and triangle path optimization for UAV-assisted wireless sensor networks

3D UAV Trajectory Design for Fair and Energy-Efficient Communication: A Deep Reinforcement Learning Technique

Energy efficient gateway based routing with maximized node coverage in a UAV assisted wireless sensor network

An Innovative Clustering Hierarchical Protocol for Data Collection from Remote Wireless Sensor Networks Based Internet of Things Applications

Traffic and Obstacle-aware UAV Positioning in Urban Environments Using Reinforcement Learning

Reinforcement Learning based Energy-Efficient Fast Routing for FANETs

Deep Reinforcement Learning for UAV Routing in the Presence of Multiple Charging Stations

Multi-UAV Multi-RIS QoS-Aware Aerial Communication Systems using DRL and PSO

Resource Allocation in UAV-Assisted Networks: A Clustering-Aided Reinforcement Learning Approach