Presence: Teleoperators and Virtual Environments

Smith, Cameron; Crook, Nigel; Dobnik, Simon; Charlton, Daniel; Boye, Johan; Pulman, Stephen; de la Camara, Raul Santos; Turunen, Markku; Benyon, David; Bradley, Jay; Gambäck, Björn; Hansen, Preben; Mival, Oli; Webb, Nick; Cavazza, Marc

doi: 10.1162/PRES_a_00063pmid: N/A

The development of embodied conversational agents (ECA) as companions brings several challenges for both affective and conversational dialogue. These include challenges in generating appropriate affective responses, selecting the overall shape of the dialogue, providing prompt system response times, and handling interruptions. We present an implementation of such a companion showing the development of individual modules that attempt to address these challenges. Further, to resolve resulting conflicts, we present encompassing interaction strategies that attempt to balance the competing requirements along with dialogues from our working prototype to illustrate these interaction strategies in operation. Finally, we provide the results of an evaluation of the companion using an evaluation methodology created for conversational dialogue and including analysis using appropriateness annotation.

Smith, Cameron; Crook, Nigel; Dobnik, Simon; Charlton, Daniel; Boye, Johan; Pulman, Stephen; de la Camara, Raul Santos; Turunen, Markku; Benyon, David; Bradley, Jay; Gambäck, Björn; Hansen, Preben; Mival, Oli; Webb, Nick; Cavazza, Marc

doi: 10.1162/pres_a_00063pmid: N/A

The development of embodied conversational agents (ECA) as companions brings several challenges for both affective and conversational dialogue. These include challenges in generating appropriate affective responses, selecting the overall shape of the dialogue, providing prompt system response times, and handling interruptions. We present an implementation of such a companion showing the development of individual modules that attempt to address these challenges. Further, to resolve resulting conflicts, we present encompassing interaction strategies that attempt to balance the competing requirements along with dialogues from our working prototype to illustrate these interaction strategies in operation. Finally, we provide the results of an evaluation of the companion using an evaluation methodology created for conversational dialogue and including analysis using appropriateness annotation.

ter Maat, Mark; Truong, Khiet P.; Heylen, Dirk

doi: 10.1162/PRES_a_00064pmid: N/A

Different turn-taking strategies of an agent influence the impression that people have of it and the behaviors that they display in response. To study these influences, we carried out several studies. In the first study, subjects listened as bystanders to computer-generated, unintelligible conversations between two speakers. In the second study, subjects talked to an artificial interviewer which was controlled by a human in a Wizard of Oz setting. Questionnaires with semantic differential scales concerning personality, emotion, social skill, and interviewing skills were used in both studies to assess the impressions that the subjects have of the agents that carried out different turn-taking strategies. In addition, in order to assess the effects of these strategies on the subjects' behavior, we measured several aspects in the subjects' speech, such as speaking rate and turn length. We found that different turn-taking strategies indeed influence the user's perception. Starting too early (interrupting the user) is mostly associated with negative and strong personality attributes and is perceived as less agreeable and more assertive. Leaving pauses between turns is perceived as more agreeable, less assertive, and creates the feeling of having more rapport. Finally, we found that turn-taking strategies also influence the subjects' speaking behavior.

ter Maat, Mark; Truong, Khiet P.; Heylen, Dirk

doi: 10.1162/pres_a_00064pmid: N/A

Different turn-taking strategies of an agent influence the impression that people have of it and the behaviors that they display in response. To study these influences, we carried out several studies. In the first study, subjects listened as bystanders to computer-generated, unintelligible conversations between two speakers. In the second study, subjects talked to an artificial interviewer which was controlled by a human in a Wizard of Oz setting. Questionnaires with semantic differential scales concerning personality, emotion, social skill, and interviewing skills were used in both studies to assess the impressions that the subjects have of the agents that carried out different turn-taking strategies. In addition, in order to assess the effects of these strategies on the subjects' behavior, we measured several aspects in the subjects' speech, such as speaking rate and turn length. We found that different turn-taking strategies indeed influence the user's perception. Starting too early (interrupting the user) is mostly associated with negative and strong personality attributes and is perceived as less agreeable and more assertive. Leaving pauses between turns is perceived as more agreeable, less assertive, and creates the feeling of having more rapport. Finally, we found that turn-taking strategies also influence the subjects' speaking behavior.

Demeure, Virginie; Niewiadomski, Radosław; Pelachaud, Catherine

doi: 10.1162/pres_a_00065pmid: N/A

The term “believability” is often used to describe expectations concerning virtual agents. In this paper, we analyze which factors influence the believability of the agent acting as the software assistant. We consider several factors such as embodiment, communicative behavior, and emotional capabilities. We conduct a perceptive study where we analyze the role of plausible and/or appropriate emotional displays in relation to believability. We also investigate how people judge the believability of the agent, and whether it provokes social reactions of humans toward it. Finally, we evaluate the respective impact of embodiment and emotion over believability judgments. The results of our study show that (a) appropriate emotions lead to higher perceived believability, (b) the notion of believability is closely correlated with the two major socio-cognitive variables, namely competence and warmth, and (c) considering an agent as believable can be different from having a human-like attitude toward it. Finally, a primacy of emotion behavior over embodiment while judging believability is also hypothesized from free responses given by the participants of this experiment.

Demeure, Virginie; Niewiadomski, Radosław; Pelachaud, Catherine

doi: 10.1162/PRES_a_00065pmid: N/A

The term “believability” is often used to describe expectations concerning virtual agents. In this paper, we analyze which factors influence the believability of the agent acting as the software assistant. We consider several factors such as embodiment, communicative behavior, and emotional capabilities. We conduct a perceptive study where we analyze the role of plausible and/or appropriate emotional displays in relation to believability. We also investigate how people judge the believability of the agent, and whether it provokes social reactions of humans toward it. Finally, we evaluate the respective impact of embodiment and emotion over believability judgments. The results of our study show that (a) appropriate emotions lead to higher perceived believability, (b) the notion of believability is closely correlated with the two major socio-cognitive variables, namely competence and warmth, and (c) considering an agent as believable can be different from having a human-like attitude toward it. Finally, a primacy of emotion behavior over embodiment while judging believability is also hypothesized from free responses given by the participants of this experiment.

de Melo, Celso M.; Carnevale, Peter; Gratch, Jonathan

doi: 10.1162/PRES_a_00062pmid: N/A

Acknowledging the social functions of emotion in people, there has been growing interest in the interpersonal effect of emotion on cooperation in social dilemmas. This paper explores whether and how facial displays of emotion in embodied agents impact cooperation with human users. The paper describes an experiment where participants play the iterated prisoner's dilemma against two different agents that play the same strategy (tit-for-tat), but communicate different goal orientations (cooperative vs. individualistic) through their patterns of facial displays. The results show that participants are sensitive to differences in the displays of emotion and cooperate significantly more with the cooperative agent. The results also reveal that cooperation rates are only significantly different when people play first with the individualistic agent. This is in line with the well-known black-hat/white-hat effect from the negotiation literature. However, this study emphasizes that people can discern a cooperator (white-hat) from a noncooperator (black-hat) based only on emotion displays. We propose that people are able to identify the cooperator by inferring, from the emotion displays, the agent's goals. We refer to this as reverse appraisal, as it reverses the usual process in which appraising relevant events with respect to one's goals leads to specific emotion displays. We discuss implications for designing human–computer interfaces and understanding human–human interaction.

de Melo, Celso M.; Carnevale, Peter; Gratch, Jonathan

doi: 10.1162/pres_a_00062pmid: N/A

Acknowledging the social functions of emotion in people, there has been growing interest in the interpersonal effect of emotion on cooperation in social dilemmas. This paper explores whether and how facial displays of emotion in embodied agents impact cooperation with human users. The paper describes an experiment where participants play the iterated prisoner's dilemma against two different agents that play the same strategy (tit-for-tat), but communicate different goal orientations (cooperative vs. individualistic) through their patterns of facial displays. The results show that participants are sensitive to differences in the displays of emotion and cooperate significantly more with the cooperative agent. The results also reveal that cooperation rates are only significantly different when people play first with the individualistic agent. This is in line with the well-known black-hat/white-hat effect from the negotiation literature. However, this study emphasizes that people can discern a cooperator (white-hat) from a noncooperator (black-hat) based only on emotion displays. We propose that people are able to identify the cooperator by inferring, from the emotion displays, the agent's goals. We refer to this as reverse appraisal, as it reverses the usual process in which appraising relevant events with respect to one's goals leads to specific emotion displays. We discuss implications for designing human–computer interfaces and understanding human–human interaction.