+ Site Statistics
+ Search Articles
+ PDF Full Text Service
How our service works
Request PDF Full Text
+ Follow Us
Follow on Facebook
Follow on Twitter
Follow on LinkedIn
+ Subscribe to Site Feeds
Most Shared
PDF Full Text
+ Translate
+ Recently Requested

Beyond Joints: Learning Representations From Primitive Geometries for Skeleton-Based Action Recognition and Detection



Beyond Joints: Learning Representations From Primitive Geometries for Skeleton-Based Action Recognition and Detection



IEEE Transactions on Image Processing 27(9): 4382-4394



Recently, skeleton-based action recognition becomes popular owing to the development of cost-effective depth sensors and fast pose estimation algorithms. Traditional methods based on pose descriptors often fail on large-scale datasets due to the limited representation of engineered features. Recent recurrent neural networks (RNN) based approaches mostly focus on the temporal evolution of body joints and neglect the geometric relations. In this paper, we aim to leverage the geometric relations among joints for action recognition. We introduce three primitive geometries: joints, edges, and surfaces. Accordingly, a generic end-to-end RNN based network is designed to accommodate the three inputs. For action recognition, a novel viewpoint transformation layer and temporal dropout layers are utilized in the RNN based network to learn robust representations. And for action detection, we first perform frame-wise action classification, then exploit a novel multi-scale sliding window algorithm. Experiments on the large-scale 3D action recognition benchmark datasets show that joints, edges, and surfaces are effective and complementary for different actions. Our approaches dramatically outperform the existing state-of-the-art methods for both tasks of action recognition and action detection.

Please choose payment method:






(PDF emailed within 0-6 h: $19.90)

Accession: 052188808

Download citation: RISBibTeXText

PMID: 29870355

DOI: 10.1109/tip.2018.2837386


Related references

Learning Clip Representations for Skeleton-Based 3D Action Recognition. IEEE Transactions on Image Processing 27(6): 2842-2855, 2018

Skeleton-based action recognition with extreme learning machines. Neurocomputing 149: 387-396, 2015

Representation Learning of Temporal Dynamics for Skeleton-Based Action Recognition. IEEE Transactions on Image Processing 25(7): 3010-3022, 2016

Learning sparse representations for view-independent human action recognition based on fuzzy distances. Neurocomputing 121: 344-353, 2013

Learning sparse representations for human action recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 34(8): 1576-1588, 2012

Learning Spatio-Temporal Representations for Action Recognition: A Genetic Programming Approach. IEEE Transactions on Cybernetics 46(1): 158-170, 2016

Distance-based human action recognition using optimized class representations. Neurocomputing 161: 47-55, 2015

View-Invariant Human Action Recognition Based on a 3D Bio-Constrained Skeleton Model. IEEE Transactions on Image Processing 28(8): 3959-3972, 2019

Skeleton-Based Action Recognition Using Spatio-Temporal LSTM Network with Trust Gates. IEEE Transactions on Pattern Analysis and Machine Intelligence 40(12): 3007-3021, 2018

View Adaptive Neural Networks for High Performance Skeleton-Based Human Action Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 41(8): 1963-1978, 2019

Skeleton-Based Human Action Recognition With Global Context-Aware Attention LSTM Networks. IEEE Transactions on Image Processing 27(4): 1586-1599, 2018

Sample Fusion Network: An End-to-End Data Augmentation Network for Skeleton-Based Human Action Recognition. IEEE Transactions on Image Processing 28(11): 5281-5295, 2019

Learning-based neural model for visual action recognition. Society for Neuroscience Abstracts 25(1-2): 672, 1999

Efficient and Robust Skeleton-Based Quality Assessment and Abnormality Detection in Human Action Performance. IEEE Journal of Biomedical and Health Informatics 2019:, 2019

Human action recognition using extreme learning machine based on visual vocabularies. Neurocomputing 73(10-12): 1906-1917, 2010