Date of Award

Spring 6-9-2022

Document Type

Thesis (Undergraduate)

Department

Computer Science

First Advisor

Professor Devin Balkcom

Second Advisor

Professor David Kraemer

Abstract

As American Sign Language (ASL), the language used by Deaf/Hard of Hearing (D/HH) Americans has grown in popularity in recent years, an unprecedented number of schools and organizations now offer ASL classes. Many hold misconceptions about ASL, assuming it is easily learned; however due to its rich, complex grammatical construction, it’s not mastered easily beyond a basic level. Therefore, it becomes ever more important to improve upon existing techniques to teach ASL. The Dartmouth Applied Learning Initiative (DALI) at Dartmouth college in coordination with the Robotics and Reality Lab developed an application on the Oculus Quest that helps D/HH individuals improve their ASL literacy. Though the app accurately predicts whether a user is signing letters correctly, it cannot verify signed words effectively due to the complexity involved in tracking motion and rotations. As a result this paper analyzes the effectiveness of using Dynamic Time Warping (DTW), a popular motion similarity comparison technique, to compare user-signed joint trajectories. I compute an 84% accuracy rate as a low bound for my algorithm due to factors involved in this calculation. This is primarily driven by one of the signs being imperfectly signed, and when we exclude that sign from analysis, our accuracy rate jumps to 92%. Therefore, I’ve identified a successful metric for validating the correctness of a signed word.

Comments

Thank you immensely to my family.

Download

Share

COinS