Senior Theses & Projects

Permanent URI for this community

Now showing 1 - 1 of 1

Sign Language Commanded Smart Home System
(2018) Abitino, Angelina; Chung, Won; Ganapati, Vidya
Given the increase in massive datasets and powerful data mining techniques in combination with advanced computational power, the use of artificial intelligence in order to improve our everyday lives is currently being implemented in a variety of ways. From innovations as simple as text prediction, to personal voice assistants, and the current research of self-driving cars, we are finding many ways to incorporate AI into all aspects of life. One way in which we are bringing AI into our homes is through the creation of the smart home system. These smart home systems can control the lights and temperature of your home, start playing a specific genre of music, and much more just by commanding them through voice commands. However, a very significant portion of people do not communicate with their voice, and instead use some non-verbal signed language, making use of a smart home system virtually impossible for them. This is where we feel the current state of AI falls short and why we are focusing our research on implementing a smart home system that responds to some non-verbal sign language. With this project we set out to control a single Sarnsung Hue light bulb with several Argentinian Sign Language (LSA) commands. We do this by implementing a computer vision and deep learning system with the combination of a convolutional neural network and a recurrent neural network.