DSLR-Net a Depth Based Sign Language Recognition Using two Stream Convents
P.V.V. Kishore1, K.B.N.S.K. Chaitanya2, G.S. S.Shravani3, M. Teja Kiran Kumar4, E. Kiran Kumar5, D. Anil Kumar6
1P.V.V. Kishore, Professor, Department of ECE, Koneru Laksha maiah Education Foundation, Green Fields, Vadde swaram, Guntur, A.P, India.
2K.B. N.S. K. Chaitanya, Department of ECE, Koneru Laksha maiah Education Foundation, Green Fields, Vadde swaram, Guntur, A.P, India.
3G. S.S. Shravani, Department of ECE, Koneru Laksha maiah Education Foundation, Green Fields, Vadde swaram, Guntur, A.P, India.
4M.Teja Kiran Kumar, Department of ECE, Koneru Laksha maiah Education Foundation, Green Fields, Vadde swaram, Guntur, A.P, India.
5E. Kiran Kumar, Department of ECE, Koneru Laksh amaiah Education Foundation, Green Fields, Vadde swaram, Guntur, A.P, India.
6D. Anil Kumar, Department of ECE, Koneru Laksha maiah Education Foundation, Green Fields, Vadde swaram, Guntur, A.P, India.
Manuscript received on 02 June 2019 | Revised Manuscript received on 10 June 2019 | Manuscript published on 30 June 2019 | PP: 765-773 | Volume-8 Issue-8, June 2019 | Retrieval Number: F3425048619/19©BEIESP
Open Access | Ethics and Policies | Cite | Mendeley | Indexing and Abstracting
© The Authors. Blue Eyes Intelligence Engineering and Sciences Publication (BEIESP). This is an open access article under the CC-BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/)
Abstract: Sign Language is the only medium of communication among the hearing impaired and mute people. Sign language includes hand movements along with facial expressions to convey their meaningful thoughts. To decrease the communication barriers with the normal people, we propose a novel framework based on two stream convolutional neural network (CNN). Which is a powerful tool with a combination of depth data obtained from low cost Kinect sensors along with RGB data obtained from Video Camera other than Kinect RGB data to ensure better pixels per inch (PPI) for images and latest deep learning algorithm Two stream convolutional neural network (CNN). RGB and Depth data are given as input to both the streams separately in training and for validating the network only RGB data is required to predict the class labels of the sign. Here the features are shared by both the RGB and Depth streams to recover the missing features by convolution operation in CNN. Which results better classification rates by decreasing the training epochs and can be used for real time interpreters for better performance. To validate our method, we created our own dataset BVCSL3D and the publicly available datasets NTU RGB-D, MSR Daily Activity 3D and UT Kinect. To claim the novelty of our model we tested our data with other state-of-the-art models and recognition rates are investigated.
Keyword: Sign language, Depth, RGB, Convolutional Neural networks.
Scope of the Article: Pattern Recognition and Analysis.