Virtual Babyz: Believable Agents with Narrative Intelligence

PF. Magic / Mindscape

88 Rowland Way

Novato, CA 94945

+1 415 895 2714

andrews@pfmagic.com, http://www.pfmagic.com

apstern@ix.netcom.com, http://www.netcom.com/~apstern

This paper describes the interactive narrative experiences in Babyz, an interactive entertainment product for the PC currently in development at PF Magic / Mindscape in San Francisco, to be released in October 1999. Babyz are believable agents designed and implemented in the tradition of Dogz and Catz, Your Virtual Petz. As virtual human characters, Babyz are more intelligent, expressive and communicative than their Petz predecessors, allowing for both broader and deeper narrative possibilities. Babyz are designed with behaviors to support entertaining short-term narrative experiences, as well as long-term emotional relationships and narratives.

The most popular contemporary mediums for telling stories are books, movies, television and theater. Yet as the number of people that own personal computers continues to grow, the potential for the computer to become a new medium for stories also grows. And because the computer is an interactive medium, artists must begin thinking about how to design and implement interactive stories, in which the user's participation develops and shapes the narrative structure itself.

While there is already a thriving industry producing interactive entertainment, namely the videogame and computer game industry, there has been little success in creating powerful interactive narrative experiences in these games. Virtually all of today’s computer games focus on some sort of action-oriented, strategy-oriented or puzzle-oriented interactivity as the core the experience. Some incorporate a story-line to accompany the game, but invariably the story is linear and unchangeable, only serving to justify to the user (i.e., the player) the need to solve yet another puzzle or continue fighting another opponent. Players have little or no control over the course of the narrative, and AI plays little or no role in developing the narrative (Stern 1999; Mateas 1999; Stern 1998). Games often have characters in them, such as in adventure or role-playing games, but with few exceptions they are not "believable" (Bates, 1992), behaving one-dimensionally and predictably, with little potential for more than the most shallow interactivity. Perhaps most fundamentally, the point of today's computer game experience is to play a game, not to be engaged in a story-like experience in the first place.

Recognizing a dearth of meaningful interactive experiences with virtual characters, we began the Virtual Petz products with the design goal to create the richest interactive "illusion of life" we could on a personal computer, within the framework of a non-goal-oriented play environment. Users "adopt" their virtual Dogz and Catz as puppies and kittens, and play with, raise and nurture them in the same manner that one would real pets, with petting, toys, food, going places, behavior training, and so on. The Petz characters are directly interactive, with rich personalities, emotions, and the ability to express themselves in a performance-like way through action and behavior. To implement these socially intelligent agents we developed a behavior-based architecture with a model of personality and emotion, all tightly integrated with an expressive realtime-3D-rendered animation system and seamless user interface (Frank, Stern and Resner, 1997).

In early versions of the Petz products, users interacted with one virtual character at a time. Then in later versions as we put multiple characters on-screen together and allowed their behaviors to play off one another (with the user as an ever-present interactive participant), we found to our surprise that the interplay between the variety of rich personalities gave rise to many dramatic situations. Small "stories" seemed to emerge as these complex synthetic characters acted out their innate personalities. Without explicitly building narrative into the system, recognizable short-term narratives were occurring (Stern, Frank and Resner, 1998). To be sure, the user's subjective experience was enhanced by the "Eliza effect" -- the tendency for people to treat programs that respond to them as if they had more intelligence than they really do (Weizenbaum, 1966). But it was clear to us that creating a broad base of richly interactive behaviors for virtual characters laid fertile ground for interactive narrative -- much to the testament of our users, who posted hundreds of messages on our website bulletin board describing their experiences and relationships with their individual Petz. (Please refer to the Appendix of this paper for a few examples of real customer letters we’ve received.)

The success of the Virtual Petz, with over 2 million copies sold worldwide as of 1998, as well as the success other virtual character products such as Tamagotchi (Bandai, 1996), Furby (Tiger Electronics, 1998) and Creatures (Grand et al, 1997), is an indication that people are interested in more than the traditional computer game genres. Our next step was to create even more intelligent virtual characters, and to explicitly endow them with some narrative intelligence, in order to increase the potential for more explicit interactive narratives.

Encouraged by the emergence of story-like experiences in Virtual Petz, we decided to make interactive narrative a design goal for our next product. To achieve this, we set out to design new characters and environments using the following criteria:

- we continue to use our tried-and-true direct interaction interface, where the user controls a hand-shaped cursor to directly touch and pick up characters and objects, and that the characters have this same direct interaction with objects and each other;

- that these new believable agents be more intelligent (e.g., better able to manipulate and use objects), more expressive (e.g., facial expressions and simple language) thereby making them more capable to perform in narratives, and more communicative (e.g., able to understand simple spoken words via voice recognition);

- that the virtual environments these characters live in have many opportunities for dramatic situations to occur, stocked with objects and props designed for playful mischief and humor;

- that the characters be familiar and recognizable enough that we can leverage off of the user's own knowledge and expectations, for dramatic effect;

- and that we choose characters that we can successfully implement at the current state of animation and artificial intelligence technology, so as to stay believable.

Human cartoon baby characters fit all of these criteria quite nicely. Even within our simplified cartoon-like visual style, Babyz can display a wide range of emotional facial expressions such as happy, giggling, laughing, frowning, crying, throwing a tantrum, angry, curious, tired, and so on. They can crawl around their virtual house and pick up objects, throw them, use them, eat them, carry them to different places. Through voice recognition Babyz can understand simple words spoken by the user in the form of praise, discipline, and the names of objects. Babyz will "learn" to speak back these words in the form of "baby talk", allowing them to truly say what they want and feel.

The Babyz live in a virtual house, with all the traditional baby accoutrements such as cribs, highchairs, and changing tables. However a few of the toys allow for mischief and fun, such as mushy food that can be thrown and splatted, rubber balls that tend to bounce around the room and knock things over, cookie jars placed on challengingly high countertops, goofy clothing and outfits, and so on.

The Babyz personalities are based on well-established cartoon archetypes, such as the clever naughty kid, the spoiled brat, and the sweet little angel. By making the Babyz sound and act in these recognizable ways, it becomes much easier to design understandable and entertaining narratives. (Interestingly, in the history of traditional animated cartoons, there have been few baby characters, perhaps because cartoons are often a bit violent, for which babies are not as well suited as cats and mice or coyotes and roadrunners.)

There are a variety of short-term narratives that can occur in Babyz, each of which may last anywhere from twenty seconds to several minutes. These narratives are sequences of actions and behaviors that follow some sort of recognizable continuity. Two examples are described in this section: soiling a diaper (leading to a diaper change), and playing a mischievous prank on a fellow Baby.

The Babyz behavior architecture is designed to allow only one short-term narrative to be occurring at one time. Yet over its duration it is possible that other short unrelated actions and behaviors can occur, either due to interruptions from unpredictable user interaction, or due to other ever-present influences like a character's metabolism (e.g., the character may get hungry and eat a quick snack). However if the narrative is distracted for more than ten or twenty seconds, it may abort and allow another narrative to begin.

Short-term narratives are implemented as high level behavior goals, each goal having multiple possible plans that can be executed in a non-linear order. Narrative goals are often spawned as reactions to user interaction, to other events in the environment, or to the character's own internal metabolism. Note that many goals in the system are non-narrative, such as being tickled or being picked up and carried; in a reactive situation, narrative and non-narrative goals compete for execution. At any decision point, each goal’s filter function is queried to compute how important it is for that goal to execute under the current circumstances. Filter functions are custom code in which the programmer can specify when a goal should execute. In a reactive context, a filter function for a narrative goal is customized to respond strongly in situations in which objects its behavior requires are now available. Part of the craft of authoring behaviors is balancing the output of these filter functions; it is easy to accidentally code a behavior to happen far too often or too seldom for believability.

Narrative goals can also be spawned deliberately as a need to regularly express the character's particular personality. This is achieved by regularly querying all goals’ filter functions in a non-reactive context, each using the character’s personality attributes as a basis for evaluating its goal’s importance; only the filter functions for narrative goals are coded to respond in this context. Additionally, a "drama manager" is keeping track of how often certain dramatically interesting narrative goals have occurred over time, and will deliberately spawn such a goal if the user hasn't experienced one in a while.

All instantiated goals are sorted in order of priority, with only one goal executing at any one time (the highest priority goal); all others are suspended, waiting to start or resume executing. At any time the current active goal can be interrupted and suspended if a higher priority goal is spawned. In fact goals can spawn other goals, allowing for multiple goals to queue up for eventual execution. Goals may delete themselves if their context conditions become invalid.

As a first example of a short-term narrative, the metabolism model is keeping track of how much time has passed since a Baby last ate, and may spawn a "soil diaper" goal. This goal always begins with the same plan, "poop in diaper", in which the Baby sits still wherever it may happen to be on-screen at that time, making sounds and facial expressions to the effect of dirtying its diaper. After this plan finishes the goal suspends itself to allow other goals to execute, such as "play with toy", "crawl and explore", "build blocks" and so on. (Which goal gets chosen is influenced by, for example, what toys the user may be interacting with at the time.) However each of these user-interactive goals will customize its behavior in context-sensitive ways, by choosing appropriate alternate locomotion animations, facial expressions, vocal sounds and diaper sounds to be seen and heard during the behavior. Eventually the "soil diaper" goal will resurface as the active goal and begin choosing from more crisis-oriented plans such as "act cranky", "cry", "itch bottom", or "babble" with baby-talk words like "poo poo" or "doo doo", each expressing more severe facial expressions and sounds during the behavior. The goal will continue choosing from this set of plans indefinitely until the user puts the Baby on the changing table, causing the goal to begin choosing from plans such as "happy diaper change", "resist diaper change", "act cranky" or "crawl and explore". Once the diaper is actually changed, the goal finishes and deletes itself, thereby ending the narrative.

Note that at any time during this narrative the user could interrupt and cause new higher priority goals to be spawned, such as "being tickled", "being picked up and carried", "react to toy shaken in my face", and so on. However these non-narrative goals are designed to end as quickly as possible if any narrative goals are suspended and waiting. Many user interactions will not disrupt an ongoing narrative at all but instead influence its execution, such as saying "shhh" if the Baby is crying, or the influencing the choice of which toy to play with, as described earlier.

Another short-term narrative example is the goal "play a prank", perhaps spawned by a naughty Baby reacting to the need to regularly express its personality, in conjunction with the presence of another Baby nearby. Before choosing a plan, the goal first scans the environment for objects tagged as good prank toys, such as a glob of wet food or a bouncy ball. (The user can influence this choice by putting certain toys within reach.) The goal first chooses the "pickup toy" plan with the chosen toy. Depending on what toy it picked up, the Baby may choose plans such as "throw toy at other Baby" or "shoot toy at other Baby". Depending on the outcome of the plan, as well as the user's reaction (such as verbal praise or discipline), the goal may finish with a plan such as "point and laugh", "act angry", "act shameful". Note that if this "play a prank" goal is interrupted by the user, for example by tickling the Baby, this goal is designed to delete itself, since the nature of its narrative requires tight continuity, and has no suspension-of-disbelief consequences for aborting. In the case of the earlier "soil diaper" goal, it will never delete itself until the diaper actually gets changed.

Other examples of short-term narratives include "playing musical instruments" to improvise a song, and "reading of a picture book" (a sort of narrative-within-narrative).

The Babyz characters (as well as their Petz predecessors) are designed to be regularly played with over the course of many weeks or months. Over time the Babyz will change and develop, eventually able to walk, talk, and understand a few spoken words. They may change how they feel about and behave towards the user and each other. Over the long-term the hope is that users will suspend their disbelief that these are artificial characters, and bond with their virtual Babyz, forming rewarding emotional relationships with them.

Babyz have an persistent fuzzy "association matrix" memory, where they keep track of how positively or negatively they feel towards the user and the other Babyz and objects they encounter. This memory is constantly being updated as they interact with their environment. When Babyz first meet, their initial attitude towards one another is established as a function of how compatible their particular personalities are, how well their first meeting goes (which can be influenced by the user), as well as some randomness. If two Babyz feel negatively towards each other, this can begin a long-term sibling rivalry, where they take turns playing pranks on each other over time. (The previous section on short-term narratives describes an example of a prank.) At first this rivalry begins with simple mild pranks such as stealing a toy, or startling the other Baby by saying "boo". But over time the pranks get more and more elaborate, such as throwing objects at each other, knocking over building blocks, messy food fights, and so on. The rivalry can continue indefinitely this way, or subside from user intervention such as keeping these Babyz apart to prevent them from fighting, or praising them in the moments when they are together and not fighting.

A long-term rivalry narrative is not implemented as a behavior goal, as short-term narratives are. The history of a rivalry is kept track of with a simple persistent episodic memory, which is queried by the drama manager to decide what short-term prank narrative goals to spawn next, and when. It is the overall continuity of related short-term narrative goals executed over time that constitute the long-term "narrative".

Another set of long-term narratives in Babyz are the development of skills. (All behaviors are pre-authored, with the user’s interaction unlocking them over time, to create the illusion that the Babyz are learning.) Over the course of several weeks, through the help of user encouragement, Babyz can advance from crawling on their stomachs to crawling on all fours, to taking their first steps, to walking. (To advance beyond walking, users have to wait for next year's product, Babyz Two Year Old.) When first adopted Babyz only make simple cooing and gurgling sounds, but if stimulated by the user's voice, they will begin trying to pronounce single syllables, and eventually become able to say simple words in a baby-talk fashion. Over time Babyz can graduate from exclusively suckling on a milk bottle for nourishment to eating food from a spoon, and feeding themselves. At first Babyz will only timidly tap at a toy piano or drum, but with the user’s encouragement they will begin playing simple songs, eventually in synchrony with the user and each other.

Like the rivalry long-term narrative, it is the overall continuity of related short-term narrative goals executed over time that constitute the long-term development narratives. Each of these development stages are kept track of by a set of simple persistent variables, from which the behavior and animation systems constantly refer to, always modifying how all other goals and plans are performed.

This paper has described the overall design philosophy behind Babyz as well as giving examples of short-term and long-term narrative experiences it offers. As you may conclude from reading this paper, when implementing Babyz we found it expedient to borrow techniques from many different disciplines, including artificial intelligence, artificial life, videogames, improvisational acting, and traditional cartoon animation. Our goal was to create a complete, polished, mass-appeal interactive entertainment product, as well as to make progress in the areas of lifelike computer characters and interactive narrative.

In her recent book, "Hamlet on the Holodeck: The Future Of Narrative in Cyberspace", Janet Murray suggests that interactive virtual characters "may mark the beginning of a new narrative format", taking on the task of redefining what it means to be human in the face of artificial intelligence (Murray, 1997). By endowing our new set of virtual characters with some explicit narrative intelligence, as well as some simple natural language capability, we hope that we are one baby step closer to achieving that goal.

Babyz was made possible by a passionate team of designers, programmers, animators, artists, producers and testers at PF. Magic / Mindscape that include Adam Frank, Rob Fulop, Ben Resner, John Rines, Mike Filippoff, Andy Webster, Jan Sleeper, Dave Feldman, Neeraj Murarka, Bruce Sherrod, Bret Berry, Laurie Sirois, Darren Atherton and many more.

I had a dog that was a chawawa and his name was Ramboo. Well he got old and was very sick and suffering so my parents put him to sleep. Ever since then I have begged my parents for a new dog. I have wanted one soo bad. So I heard about this dogz on the computer. I bought it and LOVE it!!! I have adopted 9 dogs. Sounds a bit to much to you ehhh? Well I have alot of free time on my hands. So far everyday I take each dog out one by one by them selves and play with them, feed them, and brush them, and spray them with the flee stuff. I love them all. They are all so differnant with differant personalitys. After I take them out indaviually then I take 2 out at a time and let them play with me with each other. Two of the dogs my great Dane and chawawa dont like to play with any of the other dogs but each other. This is a incrediable program. I had my parents thinking I was crazy the other night. I was sitting here playing with my scottie Ren and mutt stimpy and they where playing so well together I dont know why but I said good dog out loud to my computer. I think my parents wondered a little bit and then asked me what the heck I was doing. But thankz PF.Magic. Even though I cant have a real dog it is really nice to have some on my screen to play with. The only problem now is no one can get me away from this computer, and I think my on-line friendz are getting a little mad cause im not chatting just playing fetch and have a great time with my new dogz. Thanks again PF. magic. I love this program and will recomend it to everyone I know!!!!!!!

I am a teacher and use the catz program on my classroom PC to teach children both computer skills and caring for an animal. One of the more disturbed children in my class repeatedly squirted the catz and she ran away. Now the other children are angry at this child. I promised to try and get the catz back. It has been a wonderful lesson for the children. (And no live animal was involved.) But if there is any way to get poor Lucky to come homze to our clazz, we would very much appreciate knowing how to do it. Thanks for your help, Ms. Shinnick's 4th grade

Dear PF. Magic, I am an incredible fan of your latest release,Petz 3,I have both programs and in Janurary 1999,my cherised Dogz Tupaw was born. He is the most wonderful dogz and I thank you from the bottom of my heart, because in Janurary through to the end of April I had Anorexia and i was very sick. I ate and recoverd because i cared so much about Tupaw and i wanted to see him grow up. I would have starved without you bringing Petz 3 out. Please Reply to this,it would mean alot to me. Oh,and please visit my webpage,the url is http://www.homestead.com/wtk/pets.html. Thankyou for releasing petz 3,Give your boss my best wishes, Sincerily, Your Number One Fan, Faynine

I just reciently aquired all your Petz programs and I think they are great! I really love the way the animals react. I raised show dogs and have had numerous pets of all kinds in my life and making something like this is great. I am a school bus driver and have introduced unfortunate kids to your program. Children who not only can they not afford a computer but they can't afford to keep a pet either. This has taught them a tremendous amount of responsibilty. I am trying to get the school to incorporate your programs so as to give all children a chance to see what it is like to take care of a pet. It might help to put a little more compassion in the world. Please keep me updated on your newest releases. Thanks for being such a great company. Nancy M.

Dear PF.Magic, Hello! My name is Caitlin, and I'm 10 years old. I have Dogz 1 and Catz 1, as well as Oddballz, and I enjoy them all very much. Just this morning was I playing with my Jester breed Catz, Lilly. But I know how much better Petz II is. For a while, I thought I had a solution to my Petz II problem. I thought that if only I could get Soft Windows 95 for $200, that would work. Well, I took $100 out of my bank account (by the way, that's about half my bank account) and made the rest. I cat-sit, I sold my bike, and I got some money from my parents. Anyway, I really, really love animals (I'm a member of the ASPCA, Dog Lovers of America, and Cat Lovers of America) but I can't have one! That's why I love Petz so much! It's like having a dog or cat (or alien for that matter) only not. It's wonderful! I have a Scrappy named Scrappy (Dogz), Chip named Chip (Dogz), Bootz named Boots (Dogz), Cocker Spaniel named Oreo (Dogz), Jester named Lilly (Catz), and Jester named Callie (Catz). And then every single Oddballz breed made. =) I don't mean to bore you as I'm sure this letter is getting very boring. I would love SO MUCH to have Petz II. I really would. (At this point in the letter I'm really crying) I adopted 5 Catz II catz at my friend's house, but I go over to her house so little I'm sure they'll run away. I'd hate for them to run away. Is there anything I can do? I love my petz, and I'm sure they'd love Petz II. Thank you for reading this. Please reply soon. ~*~ Caitlin and her many petz ~*~

My husband went downtown (to Manchester) and found Catz for sale, and having heard so much about it he bought it on the spot. He put it on his very small laptop and came back from one of his business trips saying, "How many Dutchmen can watch Catz at once on a little laptop on a Dutch train?" The answer was TEN. I asked if any of them said, "Awww," the way we all did, but he said they all walked off saying it was silly. I bet they ran out to buy it anyway, though! Yours, Mrs. H. Meyer

Dear Sirs, Just wanted to thank-you for the pleasure my petz have brought me. I am paralyzed from the neck down yet your program has allowed me too again enjoy the pleasure of raising my own dogz. I have adopted 5 so far. I love them equally as if they were real. Thanks again

When I first got PetzII I went to the adoption center and RIGHT away I took out a Persian and a Calico. I liked both of the breedz so I just adopted. I loved both of them. Then kept bringing them out together because they were my only two catz. Then one day I put one away to give the other a treat and oh my god the other one got sad! Now my petz do that alot in PetzIII but they had never done that before. From then on I watched them closely. They licked each other and gave each other these looks that made me want to barf! Well I saw all these things about PetzIII. I wanted it SOO badly. I finally got a strait A's report card and my parents paid $10 or my purchase and I paid $20. My best friend (who had PetzII and put them on my computer) bought DogzIII and I bought CatzIII and we swawped. As soon as I got PetzIII I brought out the two catz.(Callie and Marcie) I made Marcie the girl and Callie the boy. I then changed Callie to Call(Cal). Within the first few seconds I had them out together Marcie the persian became (what else) Pregnant! YEAH! Ok so they are on their 6TH baby. (Wow.) I would NEVER give them away!! -Katherine

Bates, J. 1992. The Nature of Characters in Interactive Worlds and The Oz Project. Tech Report CMU-CS-92-200, School of Computer Science, Carnegie Mellon University.

Frank, A.; Stern, A.; and Resner, B. 1997. Socially Intelligent Virtual Petz. In Proceedings of the 1997 AAAI Fall Symposium, Socially Intelligent Agents, FS-97-02, pp. 43-45. Menlo Park: AAAI Press.

Frank, A.; and Stern, A. 1998. Multiple Character Interaction Between Believable Characters. In Proceedings of the 1998 Computer Game Developers Conference, pp. 215-224. Miller Freeman, San Francisco.

Grand, S.; Cliff, D.; and A. Malhotra. 1997. Creatures: Artificial Life Autonomous Software Agents for Home Entertainment. In Proceedings of the First Intl. Conference on Autonomous Agents, Minneapolis, ACM Press, pp. 22-9.

Mateas, M. 1999. Not Your Grandmother's Game: AI-Based Art and Entertainment. In Proceedings of the 1999 AAAI Spring Symposium, Artificial Intelligence and Computer Games, SS-99-02, pp. 64-68. Menlo Park: AAAI Press.

Murray, J. 1997. Hamlet on the Holodeck: The Future of Narrative in Cyberspace. The Free Press, New York.

PF Magic / Mindscape. 1995 - 1999. Virtual Petz website. URL: http://www.petz.com. Virtual Babyz website. URL: http://www.babyz.net

Stern, A.; Frank, A.; and Resner, B. 1998. Virtual Petz: A Hybrid Approach to Creating Autonomous, Lifelike Dogz and Catz. In Proceedings of the Second Intl. Conference on Autonomous Agents, pp. 334-5. Menlo Park: AAAI Press.

Stern, A. 1998. Interactive Fiction: The Story is Just Beginning. IEEE Intelligent Systems, Volume 13, No. 5, pp. 16-18. Los Alamitos: IEEE Computer Society.

Stern, A. 1999. AI Beyond Computer Games. In Proceedings of the 1999 AAAI Spring Symposium, Artificial Intelligence and Computer Games, SS-99-02, pp. 77-80. Menlo Park: AAAI Press.