Layered learning of soccer robot based on artificial neural network

XueDong Han,Bingrong Hong

2001-01-01

Abstract:The paper discusses the application of artificial neural network for MIROSOT, and introduces a layered model of BP network of soccer robot for learning basic behavior and cooperative behavior. From experimental results show that the model is effective.

What problem does this paper attempt to address?

An Intelligent Decision-Making Subsystem For Autonomous Soccer Robot

Yuehai Wang,Juan Qiao,Guangyou Wang

2008-01-01

Abstract:Decision-making subsystem is the most important part for the soccer robot that influences the result the robot soccer competition significantly. A three-layered decision-making mechanism was first proposed for the single autonomous robot. And then, based on this structure, cooperation module that negotiates among the robots of entire team was introduced. With variable learning modules were included in different layers, the proposed decision-making subsystem shows good learning capacity and, therefore, can adapt it to the evolution of the environment easily.
Robotic velocity generation using neural network

Lingyun Ye,Rong Xiong

2001-01-01

Abstract:The fast-paced nature of robotic soccer necessitates real-time sensing coupled with quick decision-making and behaving. The robot must have high response-rate, exact motion ability, and must robust enough to confront interfere during drastic match. But during the match, we find that the robot usually do not act exactly as the commands from host computer. We analyze the reason and present a method that uses BP neural network to output robotic velocity directly instead of conventional path-plan strategy, to reduce the error between actual motion and ideal plan.
Towards a cooperative soccer team in RoboCup

Jun Guan,DiJun Luo,Da Ming Liang,Rong Xiong

2004-01-01

Abstract:The key skills and algorithms of soccer simulation team ZJUBase are proposed. To improve the decision-making ability in complex, noisy and dynamic environment such as in soccer games, the layered decision-making architecture is applied and the methods and key points of maintaining the world model and the skills of players are presented. The team formation, roles and the novel idea of cooperative teammates set (CTS) are also introduced to reduce the complexity of the problem-solving work. Through the CTS of each player and on-line greedily searching, the players will cooperate to win the game.
Design of Kinect-Based Human Robot Interaction Systems for A Robocup Middle Size League Soccer Robot

Zhenli Lu,Kai Tian,Huigang Xu,Jun Liu,Limin Mao,Shujun Liu,Changkao Shan,Bin Li

DOI: https://doi.org/10.1109/cyber.2015.7288231

2015-01-01

Abstract:In order to launch a soccer game between human and robots, one of the crucial problems for robots is to effectively identify human and their actions. In this paper, based on human skeleton tracking information from Kinect, GR (Gesture Recognition), FVGR (Front View Gait Recognition) and NNVGR (Non-Normative View Gait Recognition) are carried out to develop the HRI (Human Robot Interaction) systems for a middle size league soccer robot. This method is used to identify the human body language and obtain information of body movements, so that the HRI system can generate different command to control the movements of the middle size league soccer robot. The performances of the proposed HRI systems are verified by different experiments. The experimental results show that this method can quickly and safely identify human and their movements. This research can provide a practical solution for Mid-Sized League competition between human and robots in the future.
Vision-Action Semantic Associative Learning Based on Spiking Neural Networks for Cognitive Robot.

Jiaxin Li,Dengju Li,Runhao Jiang,Rong Xiao,Huajin Tang,Kay Chen Tan

DOI: https://doi.org/10.1109/mci.2022.3199623

IF: 9.809

2022-01-01

IEEE Computational Intelligence Magazine

Abstract:Establishing cognitive environments help cognitive robots understand human actions, languages and observed objects. In this paper, a cognitive robotic model based on a novel spiking bidirectional associative memory (BAM) method is presented to establish a cognitive environment. The spiking BAM network uses a supervised spike-based learning rule to learn the relationship between the semantic information of vision and action, where a new coding scheme is proposed to encode and decode vision and action semantic information. Vision semantic information is obtained by the deeply salient shape detection (DSSD) method. Action semantic information is obtained by a spiking neural network (SNN) with the proposed coding method. Experimental results demonstrate that the spiking BAM achieves a good convergence ability and relatively high recall accuracy. Specially, for the cognitive development of the robot, this system can be used to improve the robot’s ability to infer human intentions in its natural interactions with humans.
Deep Q-Network for AI Soccer

Curie Kim,Yewon Hwang,Jong-Hwan Kim

DOI: https://doi.org/10.48550/arXiv.2209.09491

2022-09-21

Abstract:Reinforcement learning has shown an outstanding performance in the applications of games, particularly in Atari games as well as Go. Based on these successful examples, we attempt to apply one of the well-known reinforcement learning algorithms, Deep Q-Network, to the AI Soccer game. AI Soccer is a 5:5 robot soccer game where each participant develops an algorithm that controls five robots in a team to defeat the opponent participant. Deep Q-Network is designed to implement our original rewards, the state space, and the action space to train each agent so that it can take proper actions in different situations during the game. Our algorithm was able to successfully train the agents, and its performance was preliminarily proven through the mini-competition against 10 teams wishing to take part in the AI Soccer international competition. The competition was organized by the AI World Cup committee, in conjunction with the WCG 2019 Xi'an AI Masters. With our algorithm, we got the achievement of advancing to the round of 16 in this international competition with 130 teams from 39 countries.

Artificial Intelligence
Coordination and Machine Learning in Multi-Robot Systems: Applications in Robotic Soccer

Luis Paulo Reis

DOI: https://doi.org/10.48550/arXiv.2312.16273

2023-12-26

Abstract:This paper presents the concepts of Artificial Intelligence, Multi-Agent-Systems, Coordination, Intelligent Robotics and Deep Reinforcement Learning. Emphasis is given on and how AI and DRL, may be efficiently used to create efficient robot skills and coordinated robotic teams, capable of performing very complex actions and tasks, such as playing a game of soccer. The paper also presents the concept of robotic soccer and the vision and structure of the RoboCup initiative with emphasis on the Humanoid Simulation 3D league and the new challenges this competition, poses. The final topics presented at the paper are based on the research developed/coordinated by the author throughout the last 22 years in the context of the FCPortugal project. The paper presents a short description of the coordination methodologies developed, such as: Strategy, Tactics, Formations, Setplays, and Coaching Languages and the use of Machine Learning to optimize the use of this concepts. The topics presented also include novel stochastic search algorithms for black box optimization and their use in the optimization of omnidirectional walking skills, robotic multi-agent learning and the creation of a humanoid kick with controlled distance. Finally, new applications using variations of the Proximal Policy Optimization algorithm and advanced modelling for robot and multi-robot learning are briefly explained with emphasis for our new humanoid sprinting and running skills and an amazing humanoid robot soccer dribbling skill. FCPortugal project enabled us to publish more than 100 papers and win several competitions in different leagues and many scientific awards at RoboCup. In total, our team won more than 40 awards in international competitions including a clear victory at the Simulation 3D League at RoboCup 2022 competition, scoring 84 goals and conceding only 2.

Robotics
Learning Competition In Robot Soccer Game Based On An Adapted Neuro-Fuzzy Inference System

Li Shi,Chen Jiang,Ye Zhen,Sun Zengqi

DOI: https://doi.org/10.1109/ISIC.2001.971507

2001-01-01

Abstract:RoboCup (Robot World Cup Tournament) is a worldwide popular research domain in recent years. The main dealing of it includes complex system behavior of multiple autonomous agents. Because of the complexity of system, how to describe cooperation and competition between agents becomes a great challenge in RoboCup Simulation Game. On one hand, the rich experience of human soccer player is of great service to the robot players. On the other hand, the difference between simulation game and real game make it a must to fit the transcendental knowledge into the new environment.Commonly used reinforcement learning is weak in utilizing transcendental knowledge, thus is limited in complex Multi-agent System (MAS) learning problems. This thesis puts forward a supervised learning method on the basis of Adapted Neuro-Fuzzy Inference System (ANFIS) to mapping the competition among the robots. This method can build an ANFIS according to experts' knowledge, and with data obtained in the simulation environment; it is able to adjust parameters of the system's antecedents and consequents in a self-adapt way. So it can establish a correct map to describe the competition among the robots.We use this method to describe the antagonization between the shooter and goalie, and successfully apply it in the RoboCup Simulation Game to build the champion team in RoboCup 2000 of China.
MARLadona -- Towards Cooperative Team Play Using Multi-Agent Reinforcement Learning

Zichong Li,Filip Bjelonic,Victor Klemm,Marco Hutter

2024-10-06

Abstract:Robot soccer, in its full complexity, poses an unsolved research challenge. Current solutions heavily rely on engineered heuristic strategies, which lack robustness and adaptability. Deep reinforcement learning has gained significant traction in various complex robotics tasks such as locomotion, manipulation, and competitive games (e.g., AlphaZero, OpenAI Five), making it a promising solution to the robot soccer problem. This paper introduces MARLadona. A decentralized multi-agent reinforcement learning (MARL) training pipeline capable of producing agents with sophisticated team play behavior, bridging the shortcomings of heuristic methods. Further, we created an open-source multi-agent soccer environment based on Isaac Gym. Utilizing our MARL framework and a modified a global entity encoder as our core architecture, our approach achieves a 66.8% win rate against HELIOS agent, which employs a state-of-the-art heuristic strategy. Furthermore, we provided an in-depth analysis of the policy behavior and interpreted the agent's intention using the critic network.

Multiagent Systems
Building Integrated Mobile Robots for Soccer Competition

Hong Li,Bingrong,Shu,Guang

2001-01-01

Abstract:Robot soccer competition provides an excellent opportunity for robotics research. We have built a soccer robot system to participate in internal and oversea matches. Firstly, we propose a new learning control scheme adaptive PID learning controller. It means to overcome the drawbacks of the conventional PID type control methods. Secondly, we introduce our vision recognition algorithm. It remarkably increases the speed of recognition. Finally, we refer the communication system. We adopt bulletin board system to prevent communication confusion.
Pose Analysis of Humanoid Robot Imitation Process Based on Improved MLP Network

Shuhuan Wen,Yang Liu,Leibo Zheng,Fuchun Sun,Bin Fang

DOI: https://doi.org/10.1109/WRC-SARA.2019.8931970

2019-01-01

Abstract:This paper proposes a system, by which humanoid robots could imitate human movements to learn human behaviors. Considering the security problem during humanoid robot imitation process, a method of stability identification based on improved Multi-Layer Perception (MLP) is proposed. Firstly, the positions of the human joints are collected by a Kinect. Secondly, the human joints information is transformed into robot angles information by the inverse kinematics equation. Finally, the stability of the robot is identified by the deep neural network, which has the robot angles as its input. Simulations and experimental results demonstrate that compared with the Back Propagation neural network (BPnn), RBF neural network (RBFnn), Support Vector Machine (SVM) and Extreme Learning Machine (ELM), MLP neural network has the best identification accuracy. And the filters we used can overcome the problem of speed pulse, to some extent, and achieved an ideal imitation result.
Prediction-based Hierarchical Reinforcement Learning for Robot Soccer

Zongyuan Zhang,Tianyang Duan,Zekai Sun,Xiuxian Guan,Junming Wang,Hongbin Liang,Yong Cui,Heming Cui

DOI: https://doi.org/10.1109/iccc62479.2024.10681747

2024-01-01

Abstract:Robot soccer as a complex mixed cooperative-competitive task presents many challenges to multi-agent reinforcement learning (MARL), such as assigning long-term credits and effective exploration in high-dimensional and continuous state-action spaces. We propose Prediction-based Hierarchical Reinforcement Learning (P-HRL) for robot soccer. P-HRL consists of a coach for soccer tactics and a robot controller for robot motion control. To comprehensively evaluate the performance of P-HRL, we design various key performance indicators for robot soccer such as ball possession rate. Experimental results demonstrate that P-HRL has a better performance than the baseline MATD3, with 52% win rate, 22% draw rate, and 26% loss rate.
A neural network based on biological vision learning and its application on robot

Ying Gao,Xiaodan Lu,Liming Zhang

DOI: https://doi.org/10.1007/11427469_31

2005-01-01

Abstract:This paper proposes a neural network called “Hierarchical Overlapping Sensory Mapping (HOSM)”, motivated by the structure of receptive fields in biological vision. To extract the features from these receptive fields, a method called Candid covariance-free Incremental Principal Component Analysis (CCIPCA) is used to automatically develop a set of orthogonal filters. An application of HOSM on a robot with eyes shows that the HOSM algorithm can pay attention to different targets and get its cognition for different environments in real time.
ROBO: Robust, Fully Neural Object Detection for Robot Soccer

Marton Szemenyei,Vladimir Estivill-Castro

DOI: https://doi.org/10.48550/arXiv.1910.10949

2019-10-24

Abstract:Deep Learning has become exceptionally popular in the last few years due to its success in computer vision and other fields of AI. However, deep neural networks are computationally expensive, which limits their application in low power embedded systems, such as mobile robots. In this paper, an efficient neural network architecture is proposed for the problem of detecting relevant objects in robot soccer environments. The ROBO model's increase in efficiency is achieved by exploiting the peculiarities of the environment. Compared to the state-of-the-art Tiny YOLO model, the proposed network provides approximately 35 times decrease in run time, while achieving superior average precision, although at the cost of slightly worse localization accuracy.

Computer Vision and Pattern Recognition,Neural and Evolutionary Computing,Robotics
Two-stage training algorithm for AI robot soccer

Taeyoung Kim,Luiz Felipe Vecchietti,Kyujin Choi,Sanem Sariel,Dongsoo Har

DOI: https://doi.org/10.48550/arXiv.2104.05931

2021-04-13

Abstract:In multi-agent reinforcement learning, the cooperative learning behavior of agents is very important. In the field of heterogeneous multi-agent reinforcement learning, cooperative behavior among different types of agents in a group is pursued. Learning a joint-action set during centralized training is an attractive way to obtain such cooperative behavior, however, this method brings limited learning performance with heterogeneous agents. To improve the learning performance of heterogeneous agents during centralized training, two-stage heterogeneous centralized training which allows the training of multiple roles of heterogeneous agents is proposed. During training, two training processes are conducted in a series. One of the two stages is to attempt training each agent according to its role, aiming at the maximization of individual role rewards. The other is for training the agents as a whole to make them learn cooperative behaviors while attempting to maximize shared collective rewards, e.g., team rewards. Because these two training processes are conducted in a series in every timestep, agents can learn how to maximize role rewards and team rewards simultaneously. The proposed method is applied to 5 versus 5 AI robot soccer for validation. Simulation results show that the proposed method can train the robots of the robot soccer team effectively, achieving higher role rewards and higher team rewards as compared to other approaches that can be used to solve problems of training cooperative multi-agent.

Artificial Intelligence
Multi-Stage Decision-Making Skill Learning for Soccer Robot

Zhike Chen,Zhiye He,Haozhe Du,Chengrui Han,Yunkai Wang,Zexi Chen,Rong Xiong

DOI: https://doi.org/10.1109/rcar52367.2021.9517586

2021-07-15

Abstract:The application scenarios of robots are becoming more and more complex, leading to higher and higher demand on skills for robots. This paper proposes a method to learn complex skill by decomposing it into basic skills, then train them respectively as well as combination into a whole, where reinforcement learning are used both in basic skill learning and in the integration. The method proposed is validated on the RoboCup small soccer robot platform via the skill of chasing and shooting ball and tested in both simulation environment and real world. It is verified our method has achieved a higher success rate comparing with traditional methods. Code is available here.
Study and application of reinforcement learning based on DAI in cooperative strategy of robot soccer

Qi Guo,ZHANG Da-zhi,YANG Yong-tian,郭琦,张达志,杨永田

2009-01-01

Abstract:A dynamic cooperation model of multi-agent is established by combining reinforcement learning with distributed artificial intelligence (DAI), in which the concept of individual optimization loses its meaning because of the dependence of repayment on each agent itself and the choice of other agents. Utilizing the idea of DAI, the intellectual unit of each robot and the change of task and environment, each agent can make decisions independently and finish various complicated tasks by communication and reciprocation between each other. The method is superior to other reinforcement learning methods commonly used in the multi-agent system. It can improve the convergence velocity of reinforcement learning, decrease requirements of computer memory, and enhance the capability of computing and logical ratiocinating for agent. The result of a simulated robot soccer match proves that the proposed cooperative strategy is valid.
Application of Deep Learning Technology in Strength Training of Football Players and Field Line Detection of Football Robots

Daliang Zhou,Gang Chen,Fei Xu

DOI: https://doi.org/10.3389/fnbot.2022.867028

2022-06-29

Abstract:The purpose of the study is to improve the performance of intelligent football training. Based on deep learning (DL), the training of football players and detection of football robots are analyzed. First, the research status of the training of football players and football robots is introduced, and the basic structure of the neuron model and convolutional neural network (CNN) and the mainstream framework of DL are mainly expounded. Second, combined with the spatial stream network, a CNN-based action recognition system is constructed in the context of artificial intelligence (AI). Finally, by the football robot, a field line detection model based on a fully convolutional network (FCN) is proposed, and the effective applicability of the system is evaluated. The results demonstrate that the recognition effect of the dual-stream network is the best, reaching 92.8%. The recognition rate of the timestream network is lower than that of the dual-stream network, and the maximum recognition rate is 88%. The spatial stream network has the lowest recognition rate of 86.5%. The processing power of the four different algorithms on the dataset is stronger than that of the ordinary video set. The recognition rate of the time-segmented dual-stream fusion network is the highest, which is second only to the designed network. The recognition rate of the basic dual-stream network is 88.6%, and the recognition rate of the 3D CNN is the lowest, which is 86.2%. Under the intelligent training system, the recognition accuracy rates of jumping, kicking, grabbing, and starting actions range to 97.6, 94.5, 92.5, and 89.8% respectively, which are slightly lower than other actions. The recognition accuracy rate of passing action is 91.3%, and the maximum upgrade rate of intelligent training is 25.7%. The pixel accuracy of the improved field line detection of the model and the mean intersection over union (MIoU) are both improved by 5%. Intelligent training systems and the field line detection of football robots are more feasible. The research provides a reference for the development of AI in the field of sports training.
Real-Time Scene Understanding Using Deep Neural Networks for RoboCup SPL

Marton Szemenyei,Vladimir Estivill-Castro

DOI: https://doi.org/10.1007/978-3-030-27544-0_8

2019-01-01

Abstract:Convolutional neural networks (CNNs) are the state-of-the-art method for most computer vision tasks. But, the deployment of CNNs on mobile or embedded platforms is challenging because of CNNs’ excessive computational requirements. We present an end-to-end neural network solution to scene understanding for robot soccer. We compose two key neural networks: one to perform semantic segmentation on an image, and another to propagate class labels between consecutive frames. We trained our networks on synthetic datasets and fine-tuned them on a set consisting of real images from a Nao robot. Furthermore, we investigate and evaluate several practical methods for increasing the efficiency and performance of our networks. Finally, we present RoboDNN, a C++ neural network library designed for fast inference on the Nao robots.
Study and Application of Reinforcement Learning in Cooperative Strategy of the Robot Soccer Based on BDI Model

Guo Qi,Wu Bo-ying

DOI: https://doi.org/10.5772/6795

2009-01-01

Abstract:The dynamic cooperation model of multi-Agent is formed by combining reinforcement learning with BDI model. In this model, the concept of the individual optimization loses its meaning, because the repayment of each Agent dose not only depend on itsself but also on the choice of other Agents. All Agents can pursue a common optimum solution and try to realize the united intention as a whole to a maximum limit. The robot moves to its goal, depending on the present positions of the other robots that cooperate with it and the present position of the ball. One of these robots cooperating with it is controlled to move by man with a joystick. In this way, Agent can be ensured to search for each state-action as frequently as possible when it carries on choosing movements, so as to shorten the time of searching for the movement space so that the convergence speed of reinforcement learning can be improved. The validity of the proposed cooperative strategy for the robot soccer has been proved by combining theoretical analysis with simulation robot soccer match (11vs11).

Layered learning of soccer robot based on artificial neural network

An Intelligent Decision-Making Subsystem For Autonomous Soccer Robot

Robotic velocity generation using neural network

Towards a cooperative soccer team in RoboCup

Design of Kinect-Based Human Robot Interaction Systems for A Robocup Middle Size League Soccer Robot

Vision-Action Semantic Associative Learning Based on Spiking Neural Networks for Cognitive Robot.

Deep Q-Network for AI Soccer

Coordination and Machine Learning in Multi-Robot Systems: Applications in Robotic Soccer

Learning Competition In Robot Soccer Game Based On An Adapted Neuro-Fuzzy Inference System

MARLadona -- Towards Cooperative Team Play Using Multi-Agent Reinforcement Learning

Building Integrated Mobile Robots for Soccer Competition

Pose Analysis of Humanoid Robot Imitation Process Based on Improved MLP Network

Prediction-based Hierarchical Reinforcement Learning for Robot Soccer

A neural network based on biological vision learning and its application on robot

ROBO: Robust, Fully Neural Object Detection for Robot Soccer

Two-stage training algorithm for AI robot soccer

Multi-Stage Decision-Making Skill Learning for Soccer Robot

Study and application of reinforcement learning based on DAI in cooperative strategy of robot soccer

Application of Deep Learning Technology in Strength Training of Football Players and Field Line Detection of Football Robots

Real-Time Scene Understanding Using Deep Neural Networks for RoboCup SPL

Study and Application of Reinforcement Learning in Cooperative Strategy of the Robot Soccer Based on BDI Model