Convolutional Long Short-Term Memory Hybrid Networks for Skeletal Based Human Action Recognition
K. Vijaya Prasad1, P.V.V. Kishore2, O. Srinivasa Rao3
1K. Vijaya Prasad is Research Scholar, Department of ECE, JNT University Kakinada, Kakinada, India.
2P.V.V. Kishore is Department of Electronics and Communication Engineering, Koneru Lakshmaiah Education Foundation.
3O. Srinivasa Rao, Professor, Department of CSE, UCEK, JNT University Kakinada, Kakinada, India.
Manuscript received on December 15, 2019. | Revised Manuscript received on December 24, 2019. | Manuscript published on January 10, 2020. | PP: 955-961 | Volume-9 Issue-3, January 2020. | Retrieval Number: C8085019320/2020©BEIESP | DOI: 10.35940/ijitee.C8085.019320
Open Access | Ethics and Policies | Cite | Mendeley
© The Authors. Blue Eyes Intelligence Engineering and Sciences Publication (BEIESP). This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/)
Abstract: The objective is to develop a time series image representation of the skeletal action data and use it for recognition through a convolutional long short-term deep learning framework. Consequently, Kinect captured human skeletal data is transformed into a Joint Change Distance Image (JCDI) descriptor which maps the time changes in the joints. Subsequently, JCDIs are decoded spatially well with a Convolutional (CNN). Temporal decomposition is executed on long short term memory (LSTM) with data changes along x , y and z position vectors of the skeleton. We propose a combination of CNN and LSTM which maps the spatio temporal information to generate a generalized time series features for recognition. Finally, scores are fused from spatially vibrant CNNs and temporally sound LSTMs for action recognition. Publicly available action datasets such as NTU RGBD, MSR Action, UTKinect and G3D were used as test inputs for experimentation. The results showed a better performance due to spatio temporal modeling at both the representation and the recognition stages when compared to other state-of-the-arts.
Keywords: Human Action Recognition, Convolutional Neural Networks, long Short term Memory, Joint Change Distance Image, Microsoft Kinect.
Scope of the Article: Image Processing and Pattern Recognition