27 August 2026

Google builds XR prototype that adds hand gestures to AI agents

• Google created AgentHands, a system letting large language models (AI trained on vast text) control hand gestures in virtual reality environments.
• The gesture events sync with spoken words, so when the AI talks, its hands move at the same time.
• Adding gestures improved how well the AI could interact with virtual objects and how clearly it could communicate warnings compared to speech alone.

How it was covered

Deep Learning WeeklyEditorial team

AgentHands is a Google XR prototype where LLMs emit inline gesture events synced to speech. The newsletter emphasises this significantly improves spatial grounding and warning salience compared to speech-only interaction.