Google's ASL-first sign language AI now available on Pixel phones

DeepMind's sign-to-text model enables Deaf users to communicate digitally via phone without video

DeepMind developed a sign-language-to-text (SL2T) model that powers sign-to-text dictation in Google Gboard and Live Transcribe on Pixel 11 devices. This implementation starts with American Sign Language (ASL) to English translation, trained on over 100,000 hours of sign language data across 50+ languages—25% of which is ASL. The model translates sign inputs as sequences of body pose landmarks (using MediaPipe Holistic) to protect privacy and handles real-world challenges like left-handed signing and one-handed operation. It achieves a zero-shot ASL translation score of 70 BLEURT on the FLEURS-ASL benchmark, meaning it works without prior examples of specific sign patterns. This directly lowers barriers for Deaf and hard of hearing users to access digital communication at lower cost through messaging, web searches, and task execution. Future updates will expand to additional sign languages and devices. The current focus on ASL reflects the model's initial deployment scope, and its performance metric represents benchmarked results—not real-world usage across all sign languages.

Source: Google DeepMind