Skip Navigation

Sign Track – American Sign Language recognition system


Brandon Taylor works on Sign Track programming at CMU.

Brandon Taylor has a background in computer science and electrical engineering. He worked on gesture recognition programs at the Massachusetts Institute of Technology and Samsung Electronics before pursuing a Ph.D. at Carnegie Mellon University. Brandon’s wife JoAnna is a deaf education specialist and American Sign Language (ASL) interpreter.

When they first met, JoAnna was working for a school district in West Virginia. Through her job and friends, she witnessed the problems members of the deaf community face due to limited access to interpreters and language resources. JoAnna and Brandon saw an opportunity. Using his background in gesture recognition technology, Brandon could create a tool that would benefit both the deaf and non-signing communities. The idea for an automatic American Sign Language recognition system was born. They named it Sign Track.

Developing the sign recognition software that is the core of Sign Track became Brandon’s dissertation work at the Human Computer Interaction Institute at Carnegie Mellon. He applied for and was awarded a fellowship with the Swartz Center for Entrepreneurship shortly before defending his dissertation.


JoAnna spells her name using American Sign Language as Sign Track translates.

The Swartz Center fellowship allowed Brandon to remain at CMU as a post-doctoral fellow. It also introduced him to resources available through Project Olympus, a startup incubator housed at CMU.  Those resources helped Sign Track to develop a business strategy and focus on turning their research into a commercial project.

Brandon and JoAnna greatly appreciate the assistance and guidance the Sign Track team has received from advisors at the Human Computer Interaction Institute, Daniel Siewiorek and Anind Dey (now at the University of Washington), the UpPrize: BNY Mellon Social Innovation Challenge, and everyone at Project Olympus and CMU’s Swartz Center for Entrepreneurship. Without that support, they don’t think they’d be able to develop this idea.

Sign Track is still an early stage startup. The team is currently looking for more strategic partnerships and working on a beta test that they hope to launch in 2019.

The goal is to have the Sign Track software automatically recognize and translate American Sign Language (ASL), much like a Google Translate does with foreign languages. A tool like this will make communication between deaf individuals and the wider, non-signing world a lot easier, especially when interpreters are not available.


Every day, nearly one half million deaf and hard of hearing individuals who rely on American Sign Language have a hard time communicating with the non-signing public.  Brandon Taylor and Carnegie Mellon University are working to change that, with a new company called Sign Track.

Brandon Taylor – We’re creating software that will allow them to communicate in their natural language and automatically translate it so that you don’t need to know sign language to understand what someone who is signing is saying.

Sign Track uses cameras to read the manual gestures and facial expressions used to articulate American Sign Language.

Brandon Taylor –  We are just really at the very early stages of the startup. We are trying to develop the web site, come up with the pitches, and the business proposal.

Carnegie Melon’s Swartz Center for Entrepreneurship is helping Brandon find answers to the questions start-ups encounter.

Brandon Taylor –  What are the legal processes? What are the accounting processes that you need to deal with? Things that, from a Ph.D. research position, nobody ever talks about.

Brandon credits the Swartz Center and Carnegie Mellon University with giving him the tools he needs to continue his research, and focus on turning Sign Track into a successful company.

Brandon Taylor –  Without the Swartz Center and the innovation fellowships I wouldn’t be here Sign Track wouldn’t be a company.

sign-track logo.png

Find more information about Sign Track at


Turning researchers, hackers, hustlers, designers … into successful entrepreneurs. The Swartz Center for Entrepreneurship at Carnegie Mellon University is a system of programs and activities that offers a unique path of entrepreneurship education, engagement, collaboration and opportunity — for a truly transformative learning experience.

Learn more about The Swartz Center for Entrepreneurship at

Support for Innovation U comes from PNC.

Innovation U is a project from WITF, where we share stories of entrepreneurs and Pennsylvania universities who are working together to make new ideas come to life.  Learn more at

Support for WITF is provided by:

Become a WITF sponsor today »

Support for WITF is provided by:

Become a WITF sponsor today »

Up Next
Innovation U

TalkMeUp - Communication Training