FrenchIntelligent and Interactive Characters
Project Description
CATGames is creating new intuitive tools for characters that interact in real-time with game players using knowledge of how faces and bodies move, express and emote. The tools allow for the creation of emotional interactive character performances that are more dramatically subtle and engaging, while making for a more creative, faster, and cost effective authoring pipeline.
This work is human centric with knowledge of how people emote, so can be use in entertainment as well as in serious games for use in many fields including our work in human emotion and expression and face to face autism communication.
Simon Fraser University
Faculty:
Researchers:
- Ari Arya (Carleton University)
- K.J. Lee
- Veronica Zammitto
- Hasti Seifi
- Benny Lee
- Joshua Tanenbaum
Details of the Work:
Intelligent and Interactive Characters: Facial Systems
CATGames is creating new intuitive tools for characters that interact in real-time with game players using knowledge of how faces and bodies move, express and emote . The tools allow for the creation of emotional interactive character performances that are more dramatically subtle and engaging , while making for a more creative, faster, and cost effective authoring pipeline.
This work is human centric with knowledge of how people emote , so can be use in entertainment as well as for serious games for use in many fields including our work in human emotion and expression and face to face autism communication .
Personality Modeling in Believable Game Characters
Personality is the combination of qualities and traits that makes us unique. Good game character personality models allow for more natural and believable character scenarios hence increasing player engagement.
Our Facial Animation System is a multi-dimensional hierarchical approach to modeling personality into a customizable and parameterized facial character system. It is composed by four spaces: Knowledge, Personality, Mood, and Geometry. There is no hierarchy among them in order to keep them autonomous.
For the Personality module, Wiggins and Five Factor Model have been considered, capturing visual cues such as eyebrow movement, tilting head, and gaze and linking them to personality factors.
Intelligent Artistic Rendering and Creativity Tools
This research parametrically models the cognitive mechanisms human use, when they are acting creative and creating artistic works of people. This research uses creative evolutionary systems (AI: genetic programming) to explore computer creativity for various applications including graphic depiction. It uses new cognitive science research on how humans are creative and models this into computer algorithms to make better creativity design tools.
Interface Theory : Narrative & Embodiment
There has been a recent surge of innovative interface devices available for home gaming systems. Games like Guitar Hero and consoles such as Nintendo's Wii bring new opportunities for the design of game interfaces.
The research explores how the gestural, embodied interactions supported by these devices can be understood as ludic, kinesthetic and narrative experiences. Through observation and critical analysis of gameplay, the Interface Theory Group demonstrates how these three dimensions allow researchers to understand the evolving role of interface in game experience. This critical understanding will inform the creation of new tools for designing game interactions.
Publications
Perceptually Valid Facial Expressions for Character-based Applications
Arya A, DiPaola S, Parush A. Perceptually Valid Facial Expressions for Character-based Applications. International Journal of Computer Games Technology.
Abstract - This paper addresses the problem of creating facial expression of mixed emotions in a perceptually valid way. The research has been done in the context of a “game-like” health and education applications aimed at studying social competency and facial expression awareness in autistic children as well as native language learning, but the results can be applied to many other applications such as games with need for dynamic facial expressions or tools for automating the creation of facial animations. Most existing methods for creating facial expressions of mixed emotions use operations like averaging to create the combined effect of two universal emotions. Such methods may be mathematically justifiable but are not necessarily valid from a perceptual point of view. The research reported here starts by user experiments aiming at understanding how people combine facial actions to express mixed emotions, and how the viewers perceive a set of facial actions in terms of underlying emotions. Using the results of these experiments and a three-dimensional emotion model, we associate facial actions to dimensions and regions in the emotion space, and create a facial expression based on the location of the mixed emotion in the three-dimensional space. We call these regionalized facial actions “facial expression units.
“Incorporating Characteristics of Human Creativity into an Evolutionary Art Algorithm”
DiPaola S, Gabora L. “Incorporating Characteristics of Human Creativity into an Evolutionary Art Algorithm”. Genetic Programming and Evolvable Machines Journal.
Authoring the Intimate Self: Identity, Expression and Role-playing within a Pioneering Virtual Community"
DiPaola S, Turner. Authoring the Intimate Self: Identity, Expression and Role-playing within a Pioneering Virtual Community". Loading… - Journal of the Canadian Games Studies Association, in press.
Computer Facial Animation
DiPaola S. Computer Facial Animation, by Parke F, Waters K, 2nd Edition, AK Peters, 2008. Book Excerpt - pgs. 133-136,225-251,368-369. 2008. Authors: Parke,Waters.
Blending Science Knowledge and AI Gaming Techniques for Experiential Learning
DiPaola S., Akai C. Blending Science Knowledge and AI Gaming Techniques for Experiential Learning. Journal of the Canadian Games Studies Association.
A Framework for Socially Communicative Faces in Games and Interactive Applications
DiPaola S., Arya A. A Framework for Socially Communicative Faces in Games and Interactive Applications. Futureplay ‘07 Proceedings.
Abstract - In this paper, we describe a modular multi-dimensional parameter space for real-time face game-based animation. Faces are our most expressive communication tools. Therefore a synthetic facial creation and animation system should have its own tailored authoring environment rather than using general purpose tools from image, 2D and 3D animation. This environment would take advantage of a knowledge space of faces types, expressions, and behavior, encoding known facial knowledge and meaning into a comprehensive, intuitive facial language and set of user tools. Since faces and face expression work on so many cognitive levels, we propose a multi-dimension parameter space called FaceSpace as the basic face model, and a comprehensive authoring environment based on this model. We describe the underlying mechanisms of our environment, and also demonstrate its early game applications and content process.
The Trace and the Gaze: Textural Agency in Rembrandt's late Portraite from a Vision Science Perspective
DiPaola, Steve. The Trace and the Gaze: Textural Agency in Rembrandt's late Portraite from a Vision Science Perspective. Electronic Visualization and the Arts. EVA conference proceedings, 2008, pages 68-79.
SFU CATGames
DiPaola, Wakkary, Bizzocchi, Zammitto, Tanenbaum, Boileau, Lee. SFU CATGames. http://wiki.iat.sfu.ca/
Presentation and documentation of projects lead by SFU.
“You must be an experienced Thief”: Intelligent Personalization in The Elder Scrolls: Oblivion
Joshua Tanenbaum & Jim Bizzocchi. “You must be an experienced Thief”: Intelligent Personalization in The Elder Scrolls: Oblivion. Canadian Game Studies Association. Paper presented at the 2 annual CGSA conference: http://contagion.edu.yorku.ca/cgsa/CGSAconferencesched.pdf.
Looking at the Interactive Narrative Experience through the Eyes of the Participants
Milam, D., Seif El-Nasr, M., and Wakkary, R. Looking at the Interactive Narrative Experience through the Eyes of the Participants. International Conference on Interactive Storytelling.
Rock Band: A Case Study in the Design of Embodied Interface Experience.
o Tanenbaum, J., & Bizzocchi, J. (2009) Rock Band: A Case Study in the Design of Embodied Interface Experience. ACM SIGGRAPH 2009, New Orleans, August 4-8.
Abstract - There has been a recent surge of novel interface devices available for home gaming systems. With the rise in popularity of games like Guitar Hero and consoles such as Nintendo’s Wii comes new opportunities for game design at the interface level. In this paper we propose three interrelated dimensions for the analysis of embodied and gestural game interface hardware devices. We demonstrate how gestural and embodied interactions can be understood as ludic, kinesthetic and narrative experiences. We ground this discussion in a close analysis of the interface affordances of the game Rock Band and demonstrate how these three dimensions allow us to understand more clearly the place of the interface in the design and the experience of games.
Exploring Non-Verbal Behaviors Using Acting Methods
Seif El-Nasr, M. and Wei, H. Exploring Non-Verbal Behaviors Using Acting Methods. International Conference on Interactive Storytelling.
Believable characters
Seif El-Nasr, M., Bishko, L., Zammitto, V., Nixon, M., Vasilakos, T., and Wei, H. Believable characters. Book Chapter in Handbook of Digital Media in Entertainment and Arts. (Forthcoming 2009).
Player's Perception of Interactive Narrative.
Seif El-Nasr, M., Milam, D., and Wakkary, R. Player's Perception of Interactive Narrative. Book Chapter in Handbook of Digital Media in Entertainment and Arts. (Forthcoming 2009).
Is your Character Neurotic Enough? Personality in Believable Characters
Veronica Zammitto. Is your Character Neurotic Enough? Personality in Believable Characters. Canadian Game Studies Association. Paper presented at the 2 annual CGSA conference: http://contagion.edu.yorku.ca/cgsa/CGSAconferencesched.pdf.
Visualization Techniques in Video Games
Veronica Zammitto. Visualization Techniques in Video Games. EVA London 2008 Conference Proceedings. EVA conference proceedings, 2008, pages 267-276.
A Methodology for Incorporating Personality Modeling in Believable Characters
Zammitto V., DiPaola S., and Arya A. A Methodology for Incorporating Personality Modeling in Believable Characters. Proceedings of International Conference on Games Research and Development Cybergames 2008, pages 24-31.
Abstract - Strong human personality models exist but are not easy implemented within current gaming systems. The main stumbling block to utilizing classical personality models to drive real time game characters is a hierarchy, standards-based methodology that ties low level 3D movements to semantic gestures, through to emotion and time based personality sequences. Personality is the combination of qualities and traits that makes us unique. Good game character personality models allow for more natural and believable non player character scenarios hence increasing player engagement. In this paper we overview our multidimensional hierarchical approach to modeling known human personality systems into a customizable and parameterized facial character system. We describe well established psychological personality models and detail our XML based facial animation language for scripting such systems.
Attachments
System for Automated Interactive Lighting (SAIL)
Zupko, J. Seif El-Nasr, M. System for Automated Interactive Lighting (SAIL). Proceedings of the International Conference on the Foundations of Digital Games ‘09.