Gesture Detection using Tensor flow lite Efficient Net Model for Communication and E-learning Module for Mute and Deaf
Snehal Patil1, Yash Shah2, Payal Narkhede3, Abhinav Thakare4, Rahul Pitale5
1Snehal Patil*, Pursuing, Department of Computer Engg., PCCOE, Pune (Maharashtra), India.
2Yash Shah, Pursuing, Department of Computer Engg., PCCOE, Pune (Maharashtra), India.
3Payal Narkhede, Pursuing, Department of Computer Engg., PCCOE, Pune (Maharashtra), India.
4Abhinav Thakare, Pursuing, Department of Computer Engg., PCCOE, Pune (Maharashtra), India.
5Rahul Pitale, Assistant Professor, Department of Computer Engg., PCCOE, Pune (Maharashtra), India.
Manuscript received on June10, 2021. | Revised Manuscript received on June 15, 2021. | Manuscript published on June 30, 2021. | PP: 38-42 | Volume-10, Issue-8, June 2021 | Retrieval Number: 100.1/ijitee.H92040610821 | DOI: 10.35940/ijitee.H9204.0610821
Open Access | Ethics and Policies | Cite | Mendeley
© The Authors. Blue Eyes Intelligence Engineering and Sciences Publication (BEIESP). This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/)
Abstract: Human communication plays a vital role; without communicating, day-to-day tasks seem difficult to complete. And the world has an almost 5% population that struggles with hearing or speaking disability, which contributes to 430 million people worldwide, and this will grow up to 900million just in the next 25 to 30 years. With the increasing noise pollution, hearing capacity degrades, leading to various hearing problems. The WHO statistics show that 32million kids are acoustically impaired. With disabilities, there are multiple issues these people face, such as lack of learning facilities, job opportunities, communication platforms, etc. These people need a cooperative environment to express, learn at their pace and level of understanding. This paper focuses on developing an application that bridges the gap between these acoustically disabled people and people unknown to their way of communication. The proposed research is an edge device application provides features like a gesture to text, speech to text, e-learning platform, and Alert mechanism. This paper majorly focuses on developing a friendly all in one platform for mute and deaf community for communication, learning and emergency alerts. The research was conducted with two approaches the traditional CNN and Tensorflow lite EfficientNet model to train the ASL (American Sign Language) dataset for the communication platform, where we obtained accuracy of 98.91% and 98.82% respectively. To overcome the computational barriers of traditional CNN approach, Tensorf low lite Efficient Net model was brought into the picture. The proposed methodology would help build a platform for the deaf and mute community to express themselves better and gain wider exposure to the world.
Keywords: ASL, android application, CNN, image classification, E-learning, alert mechanism, firebase.