An intermediate goal of the roject is to provide a simulation environment for interactions between ARI (the robot used within SPRING) and human agents. The simulation allows to generate plausible human behaviour (for example gaze), the synthesis of high-level sensor perception (for example the perception of emotions), the high-level motion control of ARI (for example to join a group of human agents), and the simulation of conversations between ARI and humans. The aim of the simulation is to allow the training and testing of other software modules in a simulation environment so as to reduce the amount of time intensive real world experiments. The developed software consists of three modules:
1) a 3D Physical Robot Simulation1 used for the realistic simulation of ARI, 2) a Multi-Party Interaction Simulation for abstract simulations of ARI and its physical and verbal interaction with humans, and 3) a Conversational Content Generator that simulates realistic dialogues between ARI and humans. The software is open access to the public, details and links in D6.4 at our results page.
Three software modules have been developed for this simulator which support the project on different levels:
1. The 3D simulation of ARI for Gazebo provides a realistic physical model of the visual sensors and actuators of the robot and their communication interface via ROS. It allows to evaluate visual sensor modules and low-level behaviour controllers.
2. The Multi-Party Interaction Simulation provides an abstract simulation of ARI, humans, and their physical interaction.
It allows to synthesize low-level behaviour (for example, movement of humans) and perception (for example, tracking of humans). Interaction scenarios can be scripted and used to test, for example, if the navigation module of ARI is able to successfully join groups of humans. The simulation is also used to train non-verbal behaviour controllers. For example, a modified version of the simulation was used to simulate the movement of human agents and to train a meta-reinforcement learning algorithm for human-aware navigation.
3. The Conversational Content Generator generates synthetic conversational content simulating utterances that agents produce in a social context. It allows to test and train policies for the multi-party conversational system. All modules are designed to be modular and adaptable to changing requirements. They will be continuously updated and improved upon throughout the project based on the demands for the training and testing of SPRING’s non-verbal and verbal robot behaviour modules.
