Alter3: when GPT-4 turns text into robot motion

What the Short actually shows

Alter3 appears to act out a tiny social mistake: eating popcorn, realising it belongs to the person next to it, then shifting into surprise and embarrassment. The interesting part is not whether the robot is a convincing actor. It is how the researchers get from a sentence to physical movement.

In the study published in Frontiers in Robotics and AI on 27 May 2025, a user starts with a natural-language instruction. GPT-4 first expands that instruction into a detailed description of the motion. A second prompting stage turns those descriptions into Python code that controls Alter3’s body.

That changes the programming interface. Instead of manually specifying every joint for every new gesture, the system gives the language model information about the robot’s body and examples of control code, then asks it to produce a new sequence.

Why the popcorn scene matters

The popcorn example is a sequence rather than a single pose. Alter3 has to move through several states: casually eating, noticing the mistake, then changing its apparent expression and posture.

The paper uses this scenario to explore whether text can be mapped into movements that unfold coherently over time.

There is no real neighbour or bowl of popcorn in the robot’s perception. The story is supplied as language. The system generates commands that make the robot perform a matching physical sequence.

From language to actuators

Alter3 has 43 movable pneumatic axes. The researchers give GPT-4 a description of those axes and the values used to control them. The model used in the experiment was GPT-4-0314.

The pipeline has two main stages. One prompt turns the requested action into an exaggerated, step-by-step motion description. Another turns those steps into Python commands. A code filter then removes or comments out certain problematic outputs before execution.

That distinction matters because the finished motion can look like social understanding. The experiment is better understood as a new programming layer: language becomes an interface for generating robot motion.

What the researchers evaluated

The authors asked 124 participants to rate nine generated movements and included random movements as controls. The generated motions were rated as better matches for the requested actions than the controls.

That is evidence that observers could read the intended gestures more successfully. It is not evidence that Alter3 experiences embarrassment, surprise or any other emotion.

The paper also describes a physical limitation: Alter3 cannot perfectly observe the exact consequences of its own movements. The researchers therefore explored verbal feedback and an external motion memory to refine some behaviours.

Why this matters for Physical AI

The important idea is not simply that “GPT-4 controls a robot.” It is that natural language can reduce the amount of hand-authored work needed to create a new physical behaviour.

If a system can translate an intention into body-specific commands and then improve those commands through feedback, the interface between people, AI models and robots becomes more flexible.

That is a useful direction for Physical AI: language models do not replace mechanics, control systems or safety engineering. They can become one layer that translates human intent into actions a physical system can execute.

The Tomorrow, Tested. Short uses selected excerpts from Video 8, the exact supplementary research file listed below under CC BY 4.0.