eCite Digital Repository

Reinforcement learning from hierarchical critics


Cao, Z and Lin, C-T, Reinforcement learning from hierarchical critics, IEEE Transactions on Neural Networks and Learning Systems ISSN 2162-237X (In Press) [Refereed Article]

Pending copyright assessment - Request a copy


In this study, we investigate the use of global information to speed up the learning process and increase the cumulative rewards of reinforcement learning (RL) in competition tasks. Within the framework of actor-critic RL, we introduce multiple cooperative critics from two levels of a hierarchy and propose an RL from hierarchical critics (RLHC) algorithm. In our approach, each agent receives value information from local and global critics regarding a competition task and accesses multiple cooperative critics in a top-down hierarchy. Thus, each agent not only receives low-level details but also considers coordination from higher levels, thereby obtaining global information to improve the training performance. Then, we test the proposed RLHC algorithm against a benchmark algorithm, i.e., proximal policy optimisation (PPO), under four experimental scenarios consisting of tennis, soccer, banana collection, and crawler competitions within the Unity environment. The results show that RLHC outperforms the benchmark on these four competitive tasks.

Item Details

Item Type:Refereed Article
Keywords:reinforcement learning, hierarchy, critics, competition, agent
Research Division:Information and Computing Sciences
Research Group:Machine learning
Research Field:Reinforcement learning
Objective Division:Information and Communication Services
Objective Group:Information systems, technologies and services
Objective Field:Artificial intelligence
UTAS Author:Cao, Z (Dr Zehong Cao)
ID Code:145508
Year Published:In Press
Deposited By:Information and Communication Technology
Deposited On:2021-07-25
Last Modified:2021-07-26

Repository Staff Only: item control page