eCite Digital Repository

Improving recurrent neural networks with predictive propagation for sequence labelling


Tran, SN and Zhang, Q and Nguyen, A and Vu, X-S and Ngo, S, Improving recurrent neural networks with predictive propagation for sequence labelling, Proceedings of the 25th International Conference on Neural Information Processing (ICONIP 2018), Lecture Notes in Computer Science, volume 11301, 13-16 December 2018, Siem Reap, Cambodia, pp. 452-462. ISBN 978-3-030-04166-3 (2018) [Refereed Conference Paper]

Copyright Statement

Copyright 2018 Springer

DOI: doi:10.1007/978-3-030-04167-0_41


Recurrent neural networks (RNNs) is a useful tool for sequence labelling tasks in natural language processing. Although in practice RNNs suffer a problem of vanishing/exploding gradient, their compactness still offers efficiency and make them less prone to overfitting. In this paper we show that by propagating the prediction of previous labels we can improve the performance of RNNs while keeping the number of parameters in RNNs unchanged and adding only one more step for inference. As a result, the models are still more compact and efficient than other models with complex memory gates. In the experiment, we evaluate the idea on optical character recognition and Chunking which achieve promising results. © 2018, Springer Nature Switzerland AG.

Item Details

Item Type:Refereed Conference Paper
Keywords:natural language processing, recurrent neural networks, sequence labelling
Research Division:Information and Computing Sciences
Research Group:Artificial intelligence
Research Field:Intelligent robotics
Objective Division:Health
Objective Group:Specific population health (excl. Indigenous health)
Objective Field:Health related to ageing
UTAS Author:Tran, SN (Dr Son Tran)
ID Code:140697
Year Published:2018
Deposited By:Information and Communication Technology
Deposited On:2020-09-01
Last Modified:2020-11-09

Repository Staff Only: item control page