File(s) under permanent embargo
Hierarchical topic modeling with Pose-transition feature for action recognition using 3D skeleton data
journal contribution
posted on 2023-05-22, 05:44 authored by Huynh-The, T, Hua, C-H, Tu, NA, Hur, T, Bang, J, Kim, D, Muhammad Bilal AminMuhammad Bilal Amin, Byeong KangByeong Kang, Seung, H, Shin, S-Y, Kim, E-S, Lee, SDespite impressive achievements in image processing and artificial intelligence in the past decade, understanding video-based action remains a challenge. However, the intensive development of 3D computer vision in recent years has brought more potential research opportunities in pose-based action detection and recognition. Thanks to the advantages of depth camera devices like the Microsoft Kinect sensor, we developed an effective approach to in-depth analysis of indoor actions using skeleton information, in which skeleton-based feature extraction and topic model-based learning are two major contributions. Geometric features, i.e. joint distance, joint angle, and joint-plane distance are calculated in the spatio-temporal dimension. These features are merged into two types, called pose and transition features, and then are provided to codebook construction to convert sparse features into visual words by k-means clustering. An efficient hierarchical model is developed to describe the full correlation of feature - poselet - action based on Pachinko Allocation Model. This model has the potential to uncover more hidden poselets, which have been recognized as the valuable information and help to differentiate pose-sharing actions. The experimental results on several well-known datasets, such as MSR Action 3D, MSR Daily Activity 3D, Florence 3D Action, UTKinect-Action 3D, and NTU RGB+D Action Recognition, demonstrate the high recognition accuracy of the proposed method. Our method outperforms state-of-the-art methods in the field in most dataset benchmarks.
Funding
Ministry of Trade, Industry and Energy
History
Publication title
Information SciencesVolume
444Pagination
20-35ISSN
0020-0255Department/School
School of Information and Communication TechnologyPublisher
Elsevier Science IncPlace of publication
360 Park Ave South, New York, USA, Ny, 10010-1710Repository Status
- Restricted