Exploring LLMs for Generating Communicational Actions of External Interfaces on Autonomous Vehicles

External human-machine interfaces (eHMIs) help autonomous vehicles communicate with people around them. Conventional designs rely on manually specified actions for predefined situations. This research investigates whether large language models can turn intended messages into expressive eHMI actions.

We developed a prompt framework and a physical pipeline that connects generated actions to an eHMI prototype. A field study with 26 participants compared human-designed actions with actions generated by GPT-o3-mini, assessing both message interpretation and pedestrian reactions.

The study reveals a distinction between understanding a message and responding correctly. LLM-generated actions achieved comparable message interpretation, but led to more incorrect reactions. The findings highlight differences in human and LLM design priorities and motivate further work on communication that supports appropriate pedestrian responses.