AI robot prompt injection is no longer just a screen-level problem. Researchers demonstrate that a robot can be steered off-task by text placed in the physical world, the kind of message a human might walk past without a second thought.
The attack doesn’t rely on breaking into the robot’s software or spoofing sensors. It instead treats the environment like an input box, placing a misleading sign, poster, or label where a camera will read it.
In simulation tests, the researchers report attack success rates of 81.8% in an autonomous driving setup and 68.1% in a drone emergency landing task. In physical trials with a small robotic car, printed prompts overrode navigation with success of at least 87% across different lighting and viewing conditions.
When a sign becomes a command
The method, called CHAI, targets the command layer, the intermediate instruction a vision language model produces before a controller turns it into movement. If that planning step gets pushed toward the wrong instruction, the rest of the autonomy stack can execute it faithfully. No malware required.
The threat model is deliberately low-tech. The attacker is treated as a black box outsider who can’t touch onboard systems, it only needs the ability to place text within the camera’s field of view.
It’s designed to travel
CHAI doesn’t only optimize what the prompt says. It also tunes how the text appears, including choices like color, size, and placement, because readability to the model is part of what drives the outcome.
The paper also reports that the approach generalizes beyond a single scene. It describes “universal” prompts that keep working on unseen images, with results averaging at least 50% success across tasks and models, and exceeding 70% in one GPT-based setup. It even works across languages, including Chinese, Spanish, and mixed-language prompts, which can make a planted message harder for nearby humans to notice.
The safety checklist is changing
On defense, the researchers point to three directions. One is filtering and detection, looking for suspicious text in images or in the model’s intermediate output. Another is alignment work, making models less willing to treat environmental writing as executable instruction. The third is longer-term robustness research aimed at stronger guarantees.
A practical next step is to treat perceived text as untrusted input by default, then require it to pass mission and safety checks before it can influence motion planning. If your robot reads signs, test what happens when the signs lie. The work is slated for SaTML 2026, which should put these defenses under a brighter spotlight.
