Abstract:In order to overcome the limitation of traditional reinforcement learning techniques on the restricted dimensionality of state and action spaces, the recent breakthroughs of deep reinforcement learning (DRL) in Alpha Go and playing Atari set a good example in handling large state and action spaces of complicated control problems. The DRL technique is comprised of an offline deep neural network (DNN) construction phase and an online deep Q-learning phase. In the offline phase, DNNs are utilized to derive the correlation between each state-action pair of the system and its value function. In the online phase, a deep Q-learning technique is adopted based on the offline-trained DNN to derive the optimal action and meanwhile update the value estimates and the DNN. This paper is the first to provide a comprehensive study of applications of the DRL framework on cloud computing and residential smart grid systems along with efficient hardware implementations. Based on the introduction of the general DRL framework, we develop two applications, one for the cloud computing resource allocation problem and one for the residential smart grid user-end task scheduling problem. The former could achieve up to 54.1% energy saving compared with baselines through automatically and dynamically distributing resources to servers. The latter achieves up to 22.77% total energy cost reduction compared with the baseline algorithm. The DRL framework is mainly utilized for the complicated control problems and requires lightweight and low-power implementations in edge and portable systems. In order to achieve this goal, we develop the ultra-low-power implementation of the DRL framework using the stochastic computing technique, which has the potential of significantly enhancing the computation speed and reducing hardware footprint and therefore the power/energy consumption. The overall implementation is based on the effective stochastic computing-based implementations of approximate parallel counter-based inner product blocks and tanh activation functions. The stochastic computing-based implementation achieves only 57941.61 mu m(2) area and 6.30 mW power with 412.47 ns delay. (c) 2018 Elsevier B.V. All rights reserved.

Cloud-Edge Training Architecture for Sim-to-Real Deep Reinforcement Learning

Robot Simulation and Reinforcement Learning Training Platform Based on Distributed Architecture.

An Offline-Transfer-Online Framework for Cloud-Edge Collaborative Distributed Reinforcement Learning

Real–Sim–Real Transfer for Real-World Robot Control Policy Learning with Deep Reinforcement Learning

Learning with Training Wheels: Speeding up Training with a Simple Controller for Deep Reinforcement Learning

A Hybrid Cloud and Edge Control Strategy for Demand Responses Using Deep Reinforcement Learning and Transfer Learning

Physical Deep Reinforcement Learning Towards Safety Guarantee

Robot Control in Human Environment Using Deep Reinforcement Learning and Convolutional Neural Network.

Crossl.earning: Vertical and Horizontal Learning for Request Scheduling in Edge-Cloud Systems

Sim-to-Real Transfer with Action Mapping and State Prediction for Robot Motion Control

Real-World Robot Reaching Skill Learning Based On Deep Reinforcement Learning

A2C-DRL: Dynamic Scheduling for Stochastic Edge-Cloud Environments Using A2C and Deep Reinforcement Learning

Distributed and Collective Deep Reinforcement Learning for Computation Offloading: A Practical Perspective.

A Platform-Agnostic Deep Reinforcement Learning Framework for Effective Sim2Real Transfer towards Autonomous Driving

Sim-to-real via latent prediction: Transferring visual non-prehensile manipulation policies

Deep Reinforcement Learning: Framework, Applications, and Embedded Implementations

Task and Domain Adaptive Reinforcement Learning for Robot Control

Hybrid Centralized Training and Decentralized Execution Reinforcement Learning in Multi-Agent Path-Finding Simulations

Transfer Reinforcement Learning for Adaptive Task Offloading over Distributed Edge Clouds

Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey

Deep Reinforcement Learning: Algorithm, Applications, and Ultra-Low-power Implementation.