Strategies using Facial Expressions and Gaze Behaviors for Animated Agents

- Method of Selecting a Volitional Facial Expression in Game Theory -

Masahide Yuasa

  • Soft Game Theory
  • Soft Game Theory deals with the decisions of two players who exhibit feelings toward each other in a situation similar to Prisoners' Dilemma [7]. Traditional models based on Game Theory do not consider these feelings among players. However, practically, players may have certain feelings toward their opponents; for example, player A likes player B or player B does not like player C. Additionally, the relationships between players may influence their decisions. For example, if a player likes an opponent, he/she may select a better option for the opponent. Soft Game Theory proposes that the player may estimate the opponent's choice by using the feelings exhibited by the opponent.

    The models of Soft Game Theory assume that players have just one opportunity to exchange messages with each other. The player must estimate the opponent's choice and select his/her own option solely on the basis of the exchanged messages. For example, in Prisoners' Dilemma (Table 1), prisoner A can send the following message to prisoner B: ``I will select Cooperate, so please select Cooperate." After sending the message to their opponents, the players must select that option. Traditional models based on Game Theory do not consider the exchange of messages between players.

    Table 1: A Game of Prisoners' Dilemma. The first number in each cell represents Prisoner A's benefit and the second number represents that of Prisoner B
      Prisoner B
      Cooperate Defect
    Prisoner A Cooperate 3,3 1,4
      Defect 4,1 2,2

    Nevertheless, after sending the message, the player may be tempted to deceive the opponent. For example, the player could select ``Defect" despite informing the opponent that he/she would select ``Cooperate." Soft Game Theory assumes that this temptation is resolved by the players' feeling toward the opponent. Although there is a temptation to deceive the opponent, if the player has a positive feeling toward the opponent, the player could trust him/her.

  • The Effect of Changing a Choice in a Dilemma
  • Although the Soft Game Theory deals well with the feelings of players, it does not describe the generation of these feelings in the players. Instead, it assumes that players already have feelings. It is not natural for the player to have unmotivated feelings.

    Therefore, in this paper, I assume that the player's feelings toward the opponent are caused by the opponent's nonverbal behavior (for example, facial expressions and gaze behaviors) with the messages that are exchanged in the Soft Game Theory. Additionally, I assume that the player's trust is affected by the nonverbal behavior. For example, Prisoner A says the following to Prisoner B along with a HAPPY expression: ``I will select Cooperate, so please select Cooperate." The HAPPY expression makes Prisoner B trust the message sent by Prison A and select ``Cooperate," although the former may notice that the latter is tempted to deceive him/her by selecting ``Defect." On the other hand, if Prisoner A sends the same message to Prisoner B but avoids making eye contact, the latter will not trust the message and he/she will select ``Defect."

    Fig. 1 : CG animations: BOW, LOOK AWAY, HAPPY, ANGRY, SAD, COOL (This is only a sample. I used other animations in this experiment)

    I conducted an experiment to observe the relationship between selection and CG animation. I developed a new tool to send facial expressions and gaze behaviors through a computer network by using a tool for animated agent (TAA) [5]. In this tool, players must select ``Cooperate" or ``Defect" after they receive the message with CG animations as nonverbal information. There are six patterns of CG animations (Figure 1).

    Eight students from the department of computer science in the Tokyo Denki University were the subjects of this study. As their opponent, I used a program to automatically and randomly select ``Cooperate" or ``Defect" and send messages to them. The subjects did not know who their opponent was. After receiving these messages with CG animations, they were required to select ``Cooperate" or ``Defect." Table 3 shows the results of this experiment. This table indicates the probabilities of the messages - sent with animation - from the opponent to the subject stating ``I will select Cooperate, so please select Cooperate." I received 204 messages from the eight subjects during this experiment. The cases involving the use of the SAD and COOL facial expressions are not shown in this table due to insufficient usage. Before performing this experiment, I observed that four subjects selected ``Cooperate" and the remaining four selected ``Defect" without CG animation through a computer network.

    Table 2: Probabilities of Selecting Cooperate and Defect
    Cooperate 36.1% 29.0% 30.0% 30.0%
    Defect 63.9% 71.0% 70.0% 70.0%

    This table shows that the subjects selected ``Cooperate" a greater number of times in response to the use of the BOW expression (36.1%) than in the case of other CG animations. I infer that the subjects trusted the message to a greater extent because of the use of the BOW, and thus, selected ``Cooperate." On the other hand, the use of the HAPPY expression may not have resulted in the same effect, and some subjects agreed to this possibility. Additionally, it appeared that the subjects were unable to understand the opponent's intention when the ANGRY and LOOK AWAY expressions were used. I will attempt to perform the experiment with other subjects and obtain the same results by using another game in Game Theory.

  • References
  • 1
    Picard RW (1995) Affective Computing. The MIT Press

    Ortony A, Clore GL, Collins A (1988) The Cognitive Structure of Emotions. Cambridge University Press

    Velasquez J (1997) Modeling Emotions and Other Motivations in Synthetic Agents. Proceedings of AAAI 97:10-15

    Ekman P, Friesen WV (1975) Unmasking the Face. Prentice Hall Trade

    Yuasa M, Yasumura Y, Nitta K (2003) A Tool for Animated Agents in Network-Based Negotiation. Proceedings of RO-MAN 2003 Conference:259-264

    Hasegawa O, Sakaue K (1997) CG Tool for Constructing Anthropomorphic Interface Agents. Proceedings of IJCAI-97 WS (W5), ANIMATED INTERFACE AGENTS:23-26

    Howard N (1990) `Soft' Game Theory. Information and Decision Technologies 16(3)