eCite Digital Repository

Representation-Induced Algorithmic Bias : an empirical sssessment of behavioural equivalence over 14 reinforcement learning algorithms across 4 Isomorphic Gameform representations

Citation

Stanton, SC and Dermoudy, J and Ollington, R, Representation-Induced Algorithmic Bias : an empirical sssessment of behavioural equivalence over 14 reinforcement learning algorithms across 4 Isomorphic Gameform representations, AI 2021: Advances in Artificial Intelligence 34th Australasian Joint Conference, AI 2021 Sydney, NSW, Australia, February 2-4, 2022 Proceedings, Springer Nature Switzerland AG, Guodong Long , Xinghuo Yu and Sen Wang (ed), Switzerland, pp. 103-116. ISBN 9783030975463 (2022) [Research Book Chapter]

Copyright Statement

Copyright 2022 Springer Nature Switzerland AG

DOI: doi:10.1007/978-3-030-97546-3_9

Abstract

In conceiving of autonomous agents able to employ adaptive cooperative behaviours we identify the need to effectively assess the equivalence of agent behavior under conditions of external change. Reinforcement learning algorithms rely on input from the environment as the sole means of informing and so reifying internal state. This paper investigates the assumption that isomorphic representations of environment will lead to equivalent behaviour. To test this equivalence-of assumption we analyse the variance between behavioural profiles in a set of agents using fourteen foundational reinforcement-learning algorithms across four isomorphic representations of the classical Prisonerís Dilemma gameform. A behavioural profile exists as the aggregated episode-mean distributions of the game outcomes CC, CD, DC, and DD generated from the symmetric selfplay repeated stage game across a two-axis sweep of input parameters: the principal learning rate, α , and the discount factor γ , which provides 100 observations of the frequency of the four game outcomes, per algorithm, per gameform representation. A measure of equivalence is indicated by a low variance displayed between any two behavioural profiles generated by any one single algorithm. Despite the representations being theoretically equivalent analysis reveals significant variance in the behavioural profiles of the tested algorithms at both aggregate and individual outcome scales. Given this result, we infer that the isomorphic representations tested in this study are not necessarily equivalent with respect to the induced reachable space made available to any particular algorithm, which in turn can lead to unexpected agent behaviour. Therefore, we conclude that structure-preserving operations applied to environmental reward signals may introduce a vector for algorithmic bias.

Item Details

Item Type:Research Book Chapter
Keywords:game transformation
Research Division:Information and Computing Sciences
Research Group:Artificial intelligence
Research Field:Autonomous agents and multiagent systems
Objective Division:Information and Communication Services
Objective Group:Information systems, technologies and services
Objective Field:Artificial intelligence
UTAS Author:Stanton, SC (Mr Simon Stanton)
UTAS Author:Dermoudy, J (Dr Julian Dermoudy)
UTAS Author:Ollington, R (Dr Robert Ollington)
ID Code:155325
Year Published:2022
Deposited By:Information and Communication Technology
Deposited On:2023-02-12
Last Modified:2023-03-10
Downloads:0

Repository Staff Only: item control page