Cover: International Journal of Intelligent Systems, Volume 36 Issue 3 March 2021

Yang Guan,Shengbo Eben Li,Jingliang Duan,Jie Li,Yangang Ren,Qi Sun,Bo Cheng

DOI: https://doi.org/10.1002/int.22385

IF: 8.993

2021-01-01

International Journal of Intelligent Systems

Abstract:Cover Caption : The cover image is based on the Research Article Direct and indirect reinforcement learning by Yang Guan et al., https://doi.org/10.1002/int.22466 .

What problem does this paper attempt to address?

Front Cover: International Journal of Intelligent Systems, Volume 37 Issue 11 November 2022

Xianjia Meng,Yong Yang,Ximeng Liu,Nan Jiang

DOI: https://doi.org/10.1002/int.23089

IF: 8.993

2022-09-26

International Journal of Intelligent Systems

Abstract:Cover Caption: The cover image is based on the Research Article Active forgetting via influence estimation for neural networks by Xianjia Meng et al., https://doi.org/10.1002/int.22981.

computer science, artificial intelligence
Cover: International Journal of Intelligent Systems, Volume 37 Issue 5 May 2022

Zhijing Wan,Xin Xu,Zheng Wang,Toshihiko Yamasaki,Xiaolong Zhang,Ruimin Hu

DOI: https://doi.org/10.1002/int.22887

IF: 8.993

2022-03-25

International Journal of Intelligent Systems

Abstract:Cover Caption: The cover image is based on the Research Article Efficient virtual data search for annotationfree vehicle reidentification by Zhijing Wan et al., https://doi.org/10.1002/int.22829.

computer science, artificial intelligence
Cover Image, Volume 39, Issue 5

DOI: https://doi.org/10.1111/mice.13182

IF: 10.066

2024-02-25

Computer-Aided Civil and Infrastructure Engineering

Abstract:On the cover: The cover image is based on the Research Article A probabilistic deep reinforcement learning approach for optimal monitoring of a building adjacent to deep excavation by Yue Pan et al., https://doi.org/10.1111/mice.13021.

transportation science & technology,construction & building technology,computer science, interdisciplinary applications,engineering, civil
Cover Image, Volume 39, Number 7, October 2022

Rao Li,Cheng Zhou,Quanli Dou,Bin Hu

DOI: https://doi.org/10.1002/rob.22115

IF: 6.385

2022-08-22

Journal of Field Robotics

Abstract:The cover image is based on the Research Article Complete coverage path planning and performance factor analysis for autonomous bulldozer by Rao Li et al., https://doi.org/10.1002/rob.22085.

robotics
Introduction to the Focused Section on New Trends in Modelling and Simulation for Intelligent Robotics

Wang, Jin,Chen, Silu,Bai, Kun,He, Bin,Li, Howard,Zheng, Jinchuan

DOI: https://doi.org/10.1007/s41315-021-00212-0

IF: 1.7

2021-01-01

International Journal of Intelligent Robotics and Applications

Abstract:Advanced functional materials, drives and transmissions, sensors and mechanisms provide solid foundations for developing intelligent robots, for which modelling and simulation tools have contributed greatly to these developments, and facilitated deployments and applications of intelligent robots. Motivated by rapid developments of sensor fusion, intelligent control and machine learning tools and their widely availability at low cost, this special issue aims at disseminating new modelling and simulation methods for developing intelligent robotic systems and their applications. With the emerging applications to robotics, this focused section competitively selects the 10 research papers covering a spectrum of modeling of novel single robots, human-robot interactive systems, and even robotic automation process, which includes modeling of biped and wearable robots, parallel mechanisms, autonomous robots, and robot-environment interaction, as well as modeling with robot visions.
Cover Image, Volume 39, Issue 6

DOI: https://doi.org/10.1111/mice.13189

IF: 10.066

2024-03-01

Computer-Aided Civil and Infrastructure Engineering

Abstract:On the cover: The cover image is based on the Research Article A deep learning framework based on improved self‐supervised learning for ground‐penetrating radar tunnel lining inspection by Jian Huang et al., https://doi.org/10.1111/mice.13042.

transportation science & technology,construction & building technology,computer science, interdisciplinary applications,engineering, civil
AI Online Filters to Real World Image Recognition

Hai Xiao,Jin Shang,Mengyuan Huang

DOI: https://doi.org/10.48550/arXiv.2002.08242

2020-02-11

Abstract:Deep artificial neural networks, trained with labeled data sets are widely used in numerous vision and robotics applications today. In terms of AI, these are called reflex models, referring to the fact that they do not self-evolve or actively adapt to environmental changes. As demand for intelligent robot control expands to many high level tasks, reinforcement learning and state based models play an increasingly important role. Herein, in computer vision and robotics domain, we study a novel approach to add reinforcement controls onto the image recognition reflex models to attain better overall performance, specifically to a wider environment range beyond what is expected of the task reflex models. Follow a common infrastructure with environment sensing and AI based modeling of self-adaptive agents, we implement multiple types of AI control agents. To the end, we provide comparative results of these agents with baseline, and an insightful analysis of their benefit to improve overall image recognition performance in real world.

Computer Vision and Pattern Recognition,Artificial Intelligence
An Intelligent Social Learning-based Optimization Strategy for Black-box Robotic Control with Reinforcement Learning

Xubo Yang,Jian Gao,Ting Wang,Yaozhen He

DOI: https://doi.org/10.48550/arXiv.2311.06576

IF: 14.4

2023-11-11

Artificial Intelligence

Abstract:Implementing intelligent control of robots is a difficult task, especially when dealing with complex black-box systems, because of the lack of visibility and understanding of how these robots work internally. This paper proposes an Intelligent Social Learning (ISL) algorithm to enable intelligent control of black-box robotic systems. Inspired by mutual learning among individuals in human social groups, ISL includes learning, imitation, and self-study styles. Individuals in the learning style use the Levy flight search strategy to learn from the best performer and form the closest relationships. In the imitation style, individuals mimic the best performer with a second-level rapport by employing a random perturbation strategy. In the self-study style, individuals learn independently using a normal distribution sampling method while maintaining a distant relationship with the best performer. Individuals in the population are regarded as autonomous intelligent agents in each style. Neural networks perform strategic actions in three styles to interact with the environment and the robot and iteratively optimize the network policy. Overall, ISL builds on the principles of intelligent optimization, incorporating ideas from reinforcement learning, and possesses strong search capabilities, fast computation speed, fewer hyperparameters, and insensitivity to sparse rewards. The proposed ISL algorithm is compared with four state-of-the-art methods on six continuous control benchmark cases in MuJoCo to verify its effectiveness and advantages. Furthermore, ISL is adopted in the simulation and experimental grasping tasks of the UR3 robot for validations, and satisfactory solutions are yielded.
Predictive Control of Voltage Source Inverter: an Online Reinforcement Learning Solution

Xing Liu,Lin Qiu,Youtong Fang,Kui Wang,Yongdong Li,Jose Rodriguez

DOI: https://doi.org/10.1109/tie.2023.3303626

IF: 7.7

2024-01-01

IEEE Transactions on Industrial Electronics

Abstract:The focus of this article is to introduce the concept of an online reinforcement learning (RL) solution and to propose a novel finite control-set model predictive control framework subject to system uncertainties, which possesses the excellent applicative potential for power converter systems with unknown perturbations. In this framework, the control task is performed by incorporating an adaptive neural network approximation-based RL and neural predictor-based predictive current control solution. To be more precise, a critic neural network is responsible for learning a strategic utility function online, and an actor network is developed to derive control behaviors by approximating the unknown model dynamics and optimizing the learned utility function obtained from the critic network. Compared to previous works, it not only attenuates the inherent issues of system uncertainties and unknown disturbances, but also provides a flexible framework and allows the enhancement of control property. Furthermore, by deploying the Lyapunov approach, it shows that all signals in the closed-loop system are uniformly ultimately bounded. Finally, numerical simulation and experiments validate our theoretical findings.
A Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to Imitation

Lei Tai,Jingwei Zhang,Ming Liu,Joschka Boedecker,Wolfram Burgard

DOI: https://doi.org/10.48550/arXiv.1612.07139

IF: 3.7

2016-12-21

Robotics

Abstract:Deep learning techniques have been widely applied, achieving state-of-the-art results in various fields of study. This survey focuses on deep learning solutions that target learning control policies for robotics applications. We carry out our discussions on the two main paradigms for learning control with deep networks: deep reinforcement learning and imitation learning. For deep reinforcement learning (DRL), we begin from traditional reinforcement learning algorithms, showing how they are extended to the deep context and effective mechanisms that could be added on top of the DRL algorithms. We then introduce representative works that utilize DRL to solve navigation and manipulation tasks in robotics. We continue our discussion on methods addressing the challenge of the reality gap for transferring DRL policies trained in simulation to real-world scenarios, and summarize robotics simulation platforms for conducting DRL research. For imitation leaning, we go through its three main categories, behavior cloning, inverse reinforcement learning and generative adversarial imitation learning, by introducing their formulations and their corresponding robotics applications. Finally, we discuss the open challenges and research frontiers.
Cover Image, Volume 38, Issue 7

DOI: https://doi.org/10.1111/mice.13006

IF: 10.066

2023-04-03

Computer-Aided Civil and Infrastructure Engineering

Abstract:On the cover: The cover image is based on the Research Article Building height calculation for an urban area based on street view images and deep learning by Zhen Xu et al., https://doi.org/10.1111/mice.12930.

transportation science & technology,construction & building technology,computer science, interdisciplinary applications,engineering, civil
Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning

Lukas Brunke,Melissa Greeff,Adam W. Hall,Zhaocong Yuan,Siqi Zhou,Jacopo Panerati,Angela P. Schoellig

DOI: https://doi.org/10.1146/annurev-control-042920-020211

2022-01-26

Annual Review of Control, Robotics, and Autonomous Systems

Abstract:The last half decade has seen a steep rise in the number of contributions on safe learning methods for real-world robotic deployments from both the control and reinforcement learning communities. This article provides a concise but holistic review of the recent advances made in using machine learning to achieve safe decision-making under uncertainties, with a focus on unifying the language and frameworks used in control theory and reinforcement learning research. It includes learning-based control approaches that safely improve performance by learning the uncertain dynamics, reinforcement learning approaches that encourage safety or robustness, and methods that can formally certify the safety of a learned control policy. As data- and learning-based robot control methods continue to gain traction, researchers must understand when and how to best leverage them in real-world scenarios where safety is imperative, such as when operating in close proximity to humans. We highlight some of the open challenges that will drive the field of robot learning in the coming years, and emphasize the need for realistic physics-based benchmarks to facilitate fair comparisons between control and reinforcement learning approaches. Expected final online publication date for the Annual Review of Control, Robotics, and Autonomous Systems, Volume 5 is May 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.
International Journal of Neural Systems: Introduction

Jun Wang,Zhang Yi,Bao-Liang Lu,Hujun Yin

IF: 6.325

2007-01-01

International Journal of Neural Systems

Abstract:Wang, J, Yi, Z, Lu, BL & Yin, H 2007, 'International Journal of Neural Systems: Introduction', International Journal of Neural Systems, vol. 17, no. 6 … Wang, J., Yi, Z., Lu, BL., & Yin, H. (2007). International Journal of Neural Systems: Introduction. International Journal of Neural Systems, 17(6) … Wang J, Yi Z, Lu BL, Yin H. International Journal of Neural Systems: Introduction. International Journal of Neural Systems. 2007 Dec;17(6) … Wang, Jun ; Yi, Zhang ; Lu, Bao-Liang ; Yin, Hujun. / International Journal of Neural Systems : Introduction. In: International Journal of Neural Systems. 2007 ; Vol. 17, No. 6.
A Linear Programming-Based Reinforcement Learning Mechanism for Incomplete-Information Games

Baosen Yang,Changbing Tang,Yang Liu,Guanghui Wen,Guanrong Chen

DOI: https://doi.org/10.1109/jas.2024.124464

2024-10-12

IEEE/CAA Journal of Automatica Sinica

Abstract:Dear Editor, Recently, with the development of artificial intelligence, game intelligence decision-making has attracted more and more attention. In particular, incomplete-information games (IIG) have gradually become a new research focus, where players make decisions without sufficient information, such as the opponent's strategies or preferences. In this case, a selfish player can only make reactive decisions based on the changes in environment and state. Thus, blind decisions by players may drift them away from the path of reward maximization, and may even hinder the health of the IIG environment. Therefore, it is necessary to design an effective mechanism to optimize decision-making for IIG players.

automation & control systems
A Comprehensive Survey on Inverse Constrained Reinforcement Learning: Definitions, Progress and Challenges

Guiliang Liu,Sheng Xu,Shicheng Liu,Ashish Gaurav,Sriram Ganapathi Subramanian,Pascal Poupart

2024-09-22

Abstract:Inverse Constrained Reinforcement Learning (ICRL) is the task of inferring the implicit constraints followed by expert agents from their demonstration data. As an emerging research topic, ICRL has received considerable attention in recent years. This article presents a categorical survey of the latest advances in ICRL. It serves as a comprehensive reference for machine learning researchers and practitioners, as well as starters seeking to comprehend the definitions, advancements, and important challenges in ICRL. We begin by formally defining the problem and outlining the algorithmic framework that facilitates constraint inference across various scenarios. These include deterministic or stochastic environments, environments with limited demonstrations, and multiple agents. For each context, we illustrate the critical challenges and introduce a series of fundamental methods to tackle these issues. This survey encompasses discrete, virtual, and realistic environments for evaluating ICRL agents. We also delve into the most pertinent applications of ICRL, such as autonomous driving, robot control, and sports analytics. To stimulate continuing research, we conclude the survey with a discussion of key unresolved questions in ICRL that can effectively foster a bridge between theoretical understanding and practical industrial applications.

Machine Learning,Artificial Intelligence
Towards Control-Centric Representations in Reinforcement Learning from Images

Chen Liu,Hongyu Zang,Xin Li,Yong Heng,Yifei Wang,Zhen Fang,Yisen Wang,Mingzhong Wang

DOI: https://doi.org/10.48550/arxiv.2310.16655

2023-01-01

Abstract:Image-based Reinforcement Learning is a practical yet challenging task. A major hurdle lies in extracting control-centric representations while disregarding irrelevant information. While approaches that follow the bisimulation principle exhibit the potential in learning state representations to address this issue, they still grapple with the limited expressive capacity of latent dynamics and the inadaptability to sparse reward environments. To address these limitations, we introduce ReBis, which aims to capture control-centric information by integrating reward-free control information alongside reward-specific knowledge. ReBis utilizes a transformer architecture to implicitly model the dynamics and incorporates block-wise masking to eliminate spatiotemporal redundancy. Moreover, ReBis combines bisimulation-based loss with asymmetric reconstruction loss to prevent feature collapse in environments with sparse rewards. Empirical studies on two large benchmarks, including Atari games and DeepMind Control Suit, demonstrate that ReBis has superior performance compared to existing methods, proving its effectiveness.
A Survey of Imitation Learning: Algorithms, Recent Developments, and Challenges

Maryam Zare,Parham M Kebria,Abbas Khosravi,Saeid Nahavandi

DOI: https://doi.org/10.1109/TCYB.2024.3395626

2024-07-18

Abstract:In recent years, the development of robotics and artificial intelligence (AI) systems has been nothing short of remarkable. As these systems continue to evolve, they are being utilized in increasingly complex and unstructured environments, such as autonomous driving, aerial robotics, and natural language processing. As a consequence, programming their behaviors manually or defining their behavior through the reward functions as done in reinforcement learning (RL) has become exceedingly difficult. This is because such environments require a high degree of flexibility and adaptability, making it challenging to specify an optimal set of rules or reward signals that can account for all the possible situations. In such environments, learning from an expert's behavior through imitation is often more appealing. This is where imitation learning (IL) comes into play -a process where desired behavior is learned by imitating an expert's behavior, which is provided through demonstrations.This article aims to provide an introduction to IL and an overview of its underlying assumptions and approaches. It also offers a detailed description of recent advances and emerging areas of research in the field. Additionally, this article discusses how researchers have addressed common challenges associated with IL and provides potential directions for future research. Overall, the goal of this article is to provide a comprehensive guide to the growing field of IL in robotics and AI.
Inside Cover: Discovering Electrochemistry with an Electrochemistry‐Informed Neural Network (ECINN)

Haotian Chen,Minjun Yang,Bedřich Smetana,Vlastimil Novák,Vlastimil Matějka,Richard G. Compton

DOI: https://doi.org/10.1002/anie.202402855

2024-02-16

Angewandte Chemie International Edition

Abstract:A festive electrochemistry‐informed neural network (ECINN) is demonstrated by Richard G. Compton et al. (e202315937) in their Communication. The ECINN has three subnets, each made of three layers of “Chinese lantern” neurons, embedding diverse electrochemical knowledge including electrochemical kinetics to mass transport to solve experimental voltammograms and hence parameter estimation. Decorated with auspicious clouds and China′s earliest dragon from Neolithic Hongshan Culture, the cover celebrates the Year of the Dragon.

chemistry, multidisciplinary
Enhancing Offline Coverage Path Planning with Deep Reinforcement Learning

Dian-xi Shi,Shaowu Yang,Yaoning Lian,Songchang Jin,Yuxi Zheng

DOI: https://doi.org/10.1109/MLCCIM60412.2023.00056

2023-07-25

Abstract:Coverage path planning is a critical challenge in various domains, including drones, robots, and automation devices. Traditional methods heavily rely on manually crafted heuristic rules, which struggle to adapt to complex environments and task requirements. This paper proposes a deep reinforcement learning-based algorithm for coverage path planning to achieve automated and intelligent solutions. Three significant contributions are presented. Firstly, curriculum learning is integrated to adjust the training difficulty, facilitating rapid convergence in deep reinforcement learning. Secondly, a coverage heat map is introduced, providing a visual representation of target areas and guiding the convolutional neural network to focus on relevant features, enhancing coverage performance. Lastly, a more reasonable reward function is designed, promoting desirable behavior and facilitating effective exploration and exploitation during learning. Experimental evaluations demonstrate the versatility of the training framework, successfully extending its application to larger and more complex environments. The proposed algorithm is thoroughly tested and exhibits stable and efficient coverage in maps with varying initial positions and random obstacles.

Computer Science,Engineering
Intelligent Decision-Making and Human Language Communication Based on Deep Reinforcement Learning in a Wargame Environment

Yuxiang Sun,Bo Yuan,Qi Xiang,Jiawei Zhou,Jiahui Yu,Di Dai,Xianzhong Zhou

DOI: https://doi.org/10.1109/thms.2022.3225867

2023-01-01

IEEE Transactions on Human-Machine Systems

Abstract:The application of artificial intelligence (AI) in games has been significantly developed and attracted much attention over the past few years. This article not only leverages the reinforcement learning multiagent deep deterministic policy gradient algorithm to realize the dynamic decision-making of game AI but also creatively incorporates deep learning and natural language processing technologies in the wargame field to transform game context situation maps into textual suggestions in wargame confrontation. In this article, we effectively integrate reinforcement learning technologies, deep learning technologies, and natural language processing technologies to generalize the semantic text output at state-of-the-art accuracy, which plays an important role in human understanding of game AI behavior. The experimental results are promising and can be used to verify the feasibility, accuracy, and performance of our proposed model in extensive simulations against benchmarking methods.

Cover: International Journal of Intelligent Systems, Volume 36 Issue 3 March 2021

Front Cover: International Journal of Intelligent Systems, Volume 37 Issue 11 November 2022

Cover: International Journal of Intelligent Systems, Volume 37 Issue 5 May 2022

Cover Image, Volume 39, Issue 5

Cover Image, Volume 39, Number 7, October 2022

Introduction to the Focused Section on New Trends in Modelling and Simulation for Intelligent Robotics

Cover Image, Volume 39, Issue 6

AI Online Filters to Real World Image Recognition

An Intelligent Social Learning-based Optimization Strategy for Black-box Robotic Control with Reinforcement Learning

Predictive Control of Voltage Source Inverter: an Online Reinforcement Learning Solution

A Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to Imitation

Cover Image, Volume 38, Issue 7

Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning

International Journal of Neural Systems: Introduction

A Linear Programming-Based Reinforcement Learning Mechanism for Incomplete-Information Games

A Comprehensive Survey on Inverse Constrained Reinforcement Learning: Definitions, Progress and Challenges

Towards Control-Centric Representations in Reinforcement Learning from Images

A Survey of Imitation Learning: Algorithms, Recent Developments, and Challenges

Inside Cover: Discovering Electrochemistry with an Electrochemistry‐Informed Neural Network (ECINN)

Enhancing Offline Coverage Path Planning with Deep Reinforcement Learning

Intelligent Decision-Making and Human Language Communication Based on Deep Reinforcement Learning in a Wargame Environment