How does the presence of an audience influence chatbot engagement

Summary: The user experience of chatbot engagement is greatly influenced by different audience conditions such as the presence of a queue (causing timing anxieties), the company of strangers (which inspire less performative, or three-way inter-audience dialogue exchanges than acquaintances), and whether, or not the chatbot directly addresses the participant.

This article reports a dual study of the user experience at an art exhibit where visitors interacted ("talked" with) three chatbots representing characters from a book​.  The study included fieldwork observation, together with analysis of over 5000 conversation logs and video recordings to identify any dialogue patterns and possible correlations with audience conditions.  

FINDINGS

-  Awareness of a queue caused participants to rush. 
- Awareness of being observed by known acquaintances encouraged expressions of emotional appreciation towards the exhibit and yet also made it more likely that participants would exchange with other audience members through the artwork, rather than with the chatbots themselves.   This context also made it less likely that participants believed that the chatbots were answering their questions.
-  Awareness of the presence of strangers encouraged participants to ask general questions not related to the plot - and also made it more likely that participants felt part of the conversation and that the chatbots were indeed talking to them directly.
-  Observation by acquaintances, rather than strangers also intensified the likelihood of a public reaction to chatbot failure.  Direct address by chatbots also increased this likelihood - except when observed by acquaintances.
-  These effects were intensified by gender.  Female users observed by strangers in the queue were more likely to feel that the characters talked to them, while male users did not report that.  When observed by strangers around the table, female users (significantly more than male users) were less likely to perceive chatbots talking about other subjects.  Male users were also more likely to 'show off' in front of acquaintances.

The study was undertaken in the context of an artwork called Coffee with the Santiagos by three Brazilian artists. It recreated a dining room of the 19th century populated with physical representations of characters from one of the most well known and acclaimed novels in Brazilian
literature: “Dom Casmurro” by Machado de Assis (the novel
was originally published on 1899).  The researchers deliberately chose a well known title to facilitate social exchange.

DESIGN RECOMMENDATIONS
1: Designers should consider the user’s previous knowledge of content as it tends to affect the social interaction with machines, in particular when users have audiences.
2: Designers should consider that the presence of strangers in a queue waiting to interact with a physical conversational system, may affect how users will experience the system.
3: Designers should consider gender effects when crafting public interactions with conversational systems, including how to handle answers to out of scope questions.
4: Designers should consider tailoring and using direct address in some cases of chatbot utterances according to the presence of an audience.  In general, chatbots should either use the direct address, such as vocatives or pronouns to acknowledge all the participants in the audience or not at all. 


Candello, H., Pinhanez, C., Pichiliani, M., Cavalin, P., Figueiredo, F., Vasconcelos, M. and Do Carmo, H., 2019, May. The effect of audiences on the user experience with conversational interfaces in physical spaces. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (pp. 1-13)