Modeling human need for attention and interruptibility in restaurant scenarios

Abstract

The goal of this work is to train a model to quantify mental states such as neediness and interruptibility from human action patterns in restaurant scenarios. Our long-term vision is to develop robot waiters that can intelligently respond to customers. Our key insight is that human behaviors can both actively and passively communicate customers’ underlying mental states. To interpret behaviors indicating neediness and interruptibility, we automatically label key moments of human service patterns in restaurants based on waiter location and objects, as well as human behavior patterns in terms of pose, facial expression, and facial action units. Our effort to build a model is complicated by a lack of ground truth information, unreliability in waiter actions, and the effect of distractions and non-service social interactions on customer signals, and we propose solutions to each of these issues. We plan to compare the performance of several machine learning methods in predicting moments when waiters attend to customer needs based on this model.