eCite Digital Repository

An online human-agent interaction system: a brain-controlled agent playing games in unity


Cao, Z and Yun, J, An online human-agent interaction system: a brain-controlled agent playing games in unity, Proceedings of the 20th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2021), 3-7 May 2021, London, UK (In Press) [Refereed Conference Paper]

Restricted - Request a copy

Copyright Statement

Copyright 2021 International Foundation for Autonomous Agents and Multiagent Systems


Human-agent interactions present people guide an object or agent to act as human intentions. This demonstration work develops an online human-agent interaction system, particularly targeting the brain-computer interface (BCI), which uses real-time brain cortex signals: electroencephalogram (EEG) to control the agent in Unity3D game platform. The developed system also provides the online visualisation of EEG signals, including pre-processed temporal data and power spectral in three frequency bands (theta, alpha, and beta). To build this systematic work, we firstly collect wireless EEG signals via the Bluetooth transmission from a commercially available 14-channel brainware headset (Emotiv). EEG signals are then pre-processed and fed into a trained deep learning model to predict the human intentions, which will be sent to Unity3D platform to control an agentís movements in game playing, such as a karting game scenario. The online testing results show the feasibility of our systematic work that will benefit for human-agent interaction community. The demonstration video can be viewed at the following link:

Item Details

Item Type:Refereed Conference Paper
Keywords:EEG, brain, agent, interaction, human-agent interaction, brain-computer interface, unity game
Research Division:Information and Computing Sciences
Research Group:Artificial intelligence
Research Field:Autonomous agents and multiagent systems
Objective Division:Information and Communication Services
Objective Group:Information systems, technologies and services
Objective Field:Artificial intelligence
UTAS Author:Cao, Z (Dr Zehong Cao)
UTAS Author:Yun, J ( Jie Yun)
ID Code:142991
Year Published:In Press
Deposited By:Information and Communication Technology
Deposited On:2021-02-19
Last Modified:2021-04-07

Repository Staff Only: item control page