PhD offer: Deep Probabilistic Reinforcement Learning for Audio-Visual Human-Robot Interaction (Inria, France)

Reinforcement learning, and in particular deep reinforcement learning (DRL), became very popular in the recent, successfully addressing a wide variety of tasks such as board game playing. It has also been popular in addressing computer vision or pattern recognition tasks for which a differentiable loss function is difficult to find or does not exist [1]. The most popular methodology in DRL is to approximate the so-called action-value function, leading to deep Q networks (DQN) [2] and its derivatives. In these methods, both the action policy and the system’s transition function are implicitly learned within the neural network that approximates the action-value. This is limiting since it is unclear how to incorporate prior knowledge of the policy or of the system. Alternatives to the mainstream methodology exists and they can be based, for instance, in parametrising a deterministic policy, and optimizing the parametrisation by gradient descent [3]. Probabilistic alternatives based on the EM algorithm were first proposed in the late 90’s [4], and revisited in the 2010’s, see for instance [5].

In parallel, progress on deep learning lead to the conception of variational auto-encoders [6], which are non-linear probabilistic generative models. The use of VAE for reinforcement learning is at its very early stage [7], in which all actions, rewards and observations are jointly considered to infer a latent state, which is designed to encode the generation process of both the policy and the value function. In this PhD we would to investigate the use of VAE-based RL for audio-visual human-robot interaction. We aim to combine VAE for RL with probabilistic models developed for other tasks, such as speaker tracking or speech enhancement. VAE is a prominent research line but we are open to other ideas. Our team has expertise in probabilistic models for a variety of tasks [8,9] as well as on reinforcement learning for HRI [10].

Our team is searching for a motivated PhD candidate to investigate new approaches for deep reinforcement learning (DRL) in the field of audio-visual human-robot interaction. Although its high potential, DRL is still in its infancies when it comes to real-world applications such as robotics. Our group investigates DRL methods for the control of robots based on audio-visual inputs [10]. Our current project, in cooperation with several European partners [11], develops new approaches to enable health-care robots to interact and communicate with groups of people by providing information or guiding them. In difference to existing methods we are looking into approaches that combine visual and auditory information which improve for example the identification of an active speaker or their location.