Narrascope 2019 - David Kuelz - Designing Games that Listen
I'm going to talk about a game that I did a lot of the writing and narrative design for recently called Starship Commander, a virtual reality game that is driven entirely by human speech, which means that there is no controller. Instead you talk out loud to each character via the microphone. We run that audio through our system and then the character you spoke to responds, so it's a game that you play by having an elongated conversation with the script.
This speech recognition technology we're working with was disruptive. It gave players a lot of agency. They could say anything they wanted to at any point they wanted to. There's no point in which the player can't talk. That both opened up a lot of cool doorways of things that only we could do. But it also opened up a couple of Pandora's boxes of real design and production problems that we knew we had to approach carefully.
So almost every decision I made about the narrative design was trying to work with the technology and figuring out how we could highlight the parts we wanted to and either avoid or at least contain potential problems that were coming up.
THE DIALOGUE STORY IS THE GAMEPLAY
We built Starship Commander using the Unreal engine because it had two features that we really wanted. First was the blueprint system. If you guys aren't familiar with blue, It is kind of like having twine embedded into the engine, and I say kind of because it behaves very similarly. Similarly, it's about creating a flow of logic. Once you're used to nodes and branches and flow, you can learn blueprint very easily. The trick is that while twine artists draft really for story, logic, and story flow blueprint is really for game logic and gameplay flow. It's about what buttons the player is pressing and how that changes all the math and the numbers we're keeping track of and what that means for our animations and sound effects and enemy AI and all of that. So you build it in a similar way, but it's for a different type of logic. And usually those two logics are kept separate from each other.
In most games the combat or whatever it is you're doing, usually doesn't affect the story. So we'll pause the gameplay and nothing's going to attack you, and you can decide what dialogue options you want to make.
But in our case the only way that we had for the player to interact with the game was to speak to a character or their ship… So we needed one spot where we could build both of those in the same place and see the entire flow all at once. And blueprint was the answer to that for us.
The other thing that Unreal did that we really wanted is it allowed us to build our own tools. We actually initially attempted to build our own voice and speech recognition system. before eventually deciding, let's just use something someone else has already built, that's cleaner and quicker. But in that case, Unreal allowed us to build in those outside sources to the code to the engine and a way that was effortless.
As soon as the Oculus microphone picked up what people were saying, we would send it out to Microsoft and Facebook's wit.ai program, we use those two services because they had the least amount of internet latency. And then it would send back text.
NATURAL DIALOGUE AND MANAGEABLE INTENTS
We really wanted the dialogue to feel natural, which meant there were Hundreds of thousands of potential things that anybody could say at any point in time, so we needed to funnel those down to a limited number of things we could actually script.
So our programmers built this tool kit and this sort of flow of logic that we called the intent system. It works this way. any individual idea or concept of something that the player could say, was labeled an intent. So up there, you're looking at a question, Where are we going? As an intent? It could also be, who are you? It could be a comment, like, I think that's a bad idea. But any individual thing that somebody said and that we wanted to have a unique response to was an intent. Inside each intent. There was a list of utterances and utterances were essentially all of the different ways somebody could phrase that intent. So if we were going to use Who are you? as an example, some of the utterances inside that might be Tell me about yourself or What's your story? Or Tell me your story? Those are all different ways of phrasing that same idea. So ultimately, utterances became all of the different ways a player could activate this one intent.
And the last thing we needed to do was take those and phrase them in the context of a specific conversation. Because what we're saying can mean very different things depending on the context, like whether or not a provocative comment is a joke, or an insult. So, somebody would speak into the microphone. We would send it out to Facebook and Microsoft get back text, the computer would then figure out what model am I in, which is something I would determine for it in blueprints, then it would check all of the intense inside that model, and all of the utterances inside all of those intents and that would run an accuracy test and the utterance that was the most similar to the text that Microsoft sent us back determines which intent would activate. And then we had all of that system hooked up to a custom node here in blueprint that depending on what the player said, would take it off on different branches of logic and that's how our game would react to the players speech.
FULL MOTION VIDEO
We created our content using full motion video, partially because it was really quick and cheap and easy to do fast
BUILDING STORY TO FIT THE TECH
Once the tech was in place I was hired to build a plot to match. There were a lot of really cool, exciting things that we could do with our system and we wanted to make sure we had a story that capitalized on it. And there were also a lot of pitfalls that we thought we could use the story to kind of help mitigate. So I ended up coming up with a plot that was not necessarily about how excited I was as a writer to tell the story, but that I thought would fit the technology. And I was aiming to do it by thinking about these four things, which I will go through one at a time.
IF YOU”RE USING BRANCHING DIALOGUE KEEP YOUR EXPERIENCE SHORT
If you have a story that branches heavily a section that takes the average player roughly 15 to 25 seconds to get through demands a lot of writing and recording and programming. So we knew we weren't going to be able to keep this up for a long experience. Instead we're aiming for a short 90 minute to two hour experience.
YOUR BUSINESS MODEL CAN BE A CREATIVE PROMPT.
We wanted players to be comfortable dropping a little bit of money on out game, even though it was really short. SO we made it re-playable because we already had all of this alternate dialogue anyway, just by virtue of the technology. So, we did things like enabling players to find branches just by saying something specific not because they were presented with a choice. This was simultaneously a design and business decision.
TRACK AND PLAY TEST REAL CONVERSATIONS AND USE CONTEXTUAL CONSTRAINTS TO HELP MANAGE THOSE CONVERSATIONS:
During every play testing session we recorded all conversations and we would be uncovering a lot of utterances that I hadn’t thought of – and that worked, but it was still a time intensive process. So I also needed to find ways to contain the number of intents that we would be plausibly expected to have. So I created specific contexts, around scenes. Essentially, I decided to write a thriller, where every single scene has this imminent point of conflict and danger with a ticking clock element because in that situation, it does not occur to players to ask, “What's your mom's favourite colour?” This also gave us an easy out anytime someone made a comment or had a question that we hadn't thought of, so a very logical character response would be 'What does that matter right now? Can you please focus on the bomb?'
BTW we would play test in New York and because of the way people in New York naturally spoke, a scene would flow well. And then we would take it out to the west coast, and people would have different ways of putting the same idea together and it would get stumped. So actually, a big part of what we discovered with the play-testing process was to take it around to a lot of different local areas.
The biggest problem that we had was ambient noise in the room, disrupting what people said.
There was also delay in the dialogue, we couldn't avoid that but we disguised it a little. Because you are in your ship, and the entire show we disguised it as static. Anytime we needed to buy a second, the screen would fizzle and then they heard you and talked to you.
USE SURPRISE BRANCHES.
When your player can say anything at any time, at some point, some player is going to say something that disrupts the plot. Now on one hand that was a problem, but on the other hand, it can be an opportunity. Because if a player is clever enough to be able to disrupt what's supposed to happen with genuinely good story logic, we wanted to be able to reward them for that by having a concrete change in the story result. But that also put us in a weird situation, where all of a sudden, we were at the risk of players creating endless new branches, so I wanted a plot structure where we could add branches as we needed without having to build a whole lot of new content.
We also needed to account for the knowledge gap between the player and the avatar character. In our situation, the player was writing their own lines, without knowing any of their avatar’s backstory. And we needed to make that plausible or help the player figure out how to be this person.
So I created a MacGuffin. This is the anomaly…
At the beginning of the story humanity controls the MacGuffin, but they don't know what it is, or does. And in the opening scene, players role play a scientist trying to figure out what it is. They fly a one person ship into the anomaly in the scene, merging with it, potentially explaining any strange memory loss and also potentially explaining why the player know things about the future because the characters shouldn't. but maybe they've experienced alternate timelines.
In the middle of this experiment, it is disrupted, everybody is attacked by a race of war like aliens and the anomaly disappears, it blinks out of existence. It goes somewhere else within the Milky Way galaxy. They rest of the story is about you the one person with this mysterious connection to the anomaly and your military escort Sergeant Sara Pearson, exploring the Milky Way trying to figure out what the anomaly is, where it's gone, and what the human race want to do with that when they find it.
This structure gave us a lot of freedom to improvise later on. ... A mystery is made up of clues. And those clues don't necessarily make sense individually, but when you put them together, they coagulate into the bigger picture of what is happening and why it matters. As long as the clues are structured in a way that's clever, you don't necessarily need to have all of them either. When you start, you start out in the prologue scene every single time you have an experiment with the anomaly. And then depending on the choices you make in that scene, we send you to one of four different tier one scenes. In any individual playthrough you will only ever play any one of these missions and these missions are all isolated from each other. The plot of each mission you go on is dependent on where you are in the Milky Way galaxy, what is happening in this part of the galaxy, and they don't directly relate to one in each other.
So once you get to the rescue mission, that mission will proceed as usual, regardless of what missions you've happened to play before it. But the reward for each one of those missions is a clue in the greater story of what's happening with the anomaly, and why.
There were different combinations of five clues that I had to make sense and I always had to get the player to the finale of where the anomaly was and why it's there. But once I knew there's only ever going to be one tier one clue. There will only ever be one tier two clue I could create a sense of progression. A good mystery isn't, Oh I have one clue. And it doesn't make sense, only when I have five can it be solved. Instead you want to slowly build and curate a picture of what's happening and then start to correct it. Now that I have two clues. I get the sense that these are connected in this way, but I don't know how yet. And now that I have a tier three clue Oh, that's different than what I thought that it's starting to get clear. And depending on the progression the player took through the game, they would get a slightly different version, a different perspective of what was happening and why you couldn't understand everything about the story until you played it through multiple times.
Let's walk through an example. Let's say the player has played one playthrough of the game, and in the first playthrough, they participate in the alien ship mission. And they get that clue about the anomaly. When they restart the game, technically, that fact doesn't exist in the game, but the player still knows it. The story is the same every time the player just gets different pieces of it. So potentially, the clue that the player learned in the alien ship, they might be able to use to change the way the traversal Memorial pans out. They might want to create a surprise branch there. And we could actually then reward them for that by making a branch that goes from the traversal Memorial sit down to Caltech runes, that would otherwise not be connected. So we can rearrange the flow.
Also at any point, if we're running out of time, or there's a tech problem that we don't know how to solve, we can just say, you know what, let's get rid of that scene. …and the reality of the story structure is still intact. You could make loads of cuts and as long as you don't make too many cuts from any one branch, the actual story is still playable. So we could sort of decide as we went and adjust ourselves from there.
Certain types of stories are more meandering than others and you do not want that. And when I think of sci fi and fantasy, I usually think of the setting as a huge star, but ultimately, we found that meandering was not a fun activity and very difficult to build. So in terms of certain genres of story, I would say, plot driven stories, like thrillers specifically, or maybe a very strong character driven story like a romance might work really well for this approach.
BUT PLAY TESTS SURPRISED US
As it turns out the players have a much better experience when you actually do have a lot of control, and you really can guide them through something specific.
One thing we had learned from actually our very initial brief tech demo scene, was that we really needed to have a way of allowing the conversation to continue regardless of what the player said if they happen to get stuck, because in a system that is this open ended, we really didn't have a plausible way of suggesting to the player what they should say to another human being right now without breaking the fourth wall in a way we didn't want to do.
But when it is that open ended, and when you are waiting for the player to say something specific, like I'm ready or Let's go, ultimately, it's not going to occur to a lot of people to do that.
There's a lot of people that are going to get stuck not know what to say and loop around and endless spiral. So we tried to have a way of progressing automatically. At some point if enough time had passed we would have the character, “You know what, the anomalies out there, we got to keep going and we got to move”. Down here, that's a different face of the conversation with different contexts and different reactions to what you might say. We did that in a very linear way.
But it is very, very difficult to write compelling dialogue when you only know what one half of the conversation is going to be.
I didn't know what my character were reacting to. I didn't really know what the player just said, I knew the idea. I knew the intent. But I didn't know the words. I didn't know the tone. I didn't know the utterance. And so all of the dialogue that I was putting out was very generic. It was very vanilla. I wasn't in love with these people. It wasn't good writing.
Eventually I realised that this was that this was a gameplay problem, and not a writing problem. So it wasn't that the writing was bad. It's that the flow of the activity was not fun. I took the time to do some extra drafts and try to find voice. But what actually I should have been doing what would have been more productive would have been treating my initial draft in twine, not like a draft of a script, but like a paper prototype for the gameplay to try and figure out if the actual activity of this specific conversation and the way it moves is fun.
When we did start play testing the scenes, we discovered that players are actually very bad at coming up with things to say. Most of the things that they said were incredibly short, very simple. A couple of words of one word answers.. 90% of players just did whatever they were told, without getting into it. Only about 10% of players actually explored.
Unlimited freedom shuts down our ability to actually make a decision.
So in open ended sections in the script, we would say, do you have any questions? Do you want to know about me? Do you want to know about your ship? Do you want to know about this planet? The aliens? What do you want to know about ask? People go? I don't know. I don't have any questions. And we keep moving.
But the concept of asking anything turned out to be an inherently unfun activity. The more intents we have, the more utterances we have to have for each one. The more directions we go, the more general we have to be. But as it turns out, that the freedom wasn't what players wanted, they wanted to be listened to, very, very specifically.
Instead, the consistently successful part of our play test was a very brief moment early in the briefing scene, where Sergeant Pearson is filling out some of your information and she would ask How old are you? And we had written out a response of every age from like five to 95. Nobody came in with an answer. We weren't prepared. But what worked about that moment is because the constraints of the scene were very tight, so we could build out 90 different personalised answers to this one question.
And in that moment, players felt listened to. And more importantly, because we knew what was going to happen we had a funny little joke for every age. If you came in and said, I'm 43. We have a funny little joke about 43 year olds for you. Now that did a couple of things. This call and response dialog did a couple of things for us. But most importantly, it gave us a feedback loop.
The idea is that it's the physical concrete process by which people play a game, which usually means that they push a button. That action has an effect inside the world of the game, but the loop isn't complete until the player is informed about the effects of their action. Through feedback, the player presses a gob or presses a button and attacks the Goblin. Once we have an animation where the Goblin staggers back and a little sound effect where it yells and a blood texture shows up on the floor that's when the player really understands what they did and are equipped to now take another action.
This is where controlling the context of each scene became very important. Your character could legitimately take the reins to focus in on this one thing that's very important that we have to do. But then because we're not doing anything else, we can create all of these different options and listen to the specific things that each player says, each utterance can become its own intent, and then we can respond to exactly what they said and the exact words that they used, in a way that's immediately engaging and rewarding. Once I did this, I immediately knew how to write again. I immediately had the ability to look at exactly what a player had said and bounce off of it in a way that uses their own words, and I could create dialogue that was engaging. And the writing became a reward for the players creativity. When the player said something specific, a little bit unique to them, we could respond to it in a way that was actually entertaining, we could respond in a way that had value in and of itself. And when we rewarded players for being creative, all of a sudden player started being creative. They started opening up they started talking a little bit more as if they were talking to an actual person.
So overall lessons:
- Pick technology that you can control pick technology, you can build your own tools,
- Pick a plot where you can tightly control the context of a scene.
Kuelz, David. 2019. Designing Games that Listen. U.S.: Narrascope 2019/YouTube.
Narrascope 2019 - Julius Kuschke - How Dialogue Systems Make or Break Player Engagement
.Julius Kuschke, Chief Product officer at RTC creates tools for game writers. RTC makes Artists Draft, a software to create branching stories. In this talk he reviews highlights from the ways that previous games have ensured what Julius sees as the four secrets of a good dialogue:
Previously, in a hub dialogue system, you could choose all the options offered in turn,. And that's very unnatural. It's also very much like an interrogation, because only the player is driving the whole dialogue. This turns NPCs into information wending machines, where the player feels obligated to punch every button, just to get each available tidbit of data.
So, this is why I think waterfall structures are so popular today because they feel so much more natural. Players understand that each option not chosen is gone forever, they don't have a second chance. The conversation will move on.
Just as an example for that, let's take a look at Assassin's Creed Odyssey.
“Tell me mysterious. Did you learn anything worthwhile in your dealings with the world?”
“I'm done talking, I should tell you where you stand.”
“They said you'd be different. And blood is blood, I suppose.”
So, even though it's clearly a waterfall structure, without reoccurring hops, the dialogue often feels slow and artificial. Whenever there's a choice, everything stands still and the game waits for your input.
The most simple solution for this problem is to have time decisions and Telltale Games in particular made that system very popular.
Timers are great to create tension, and they reduce that weird awkward silence in between choices. They can be very difficult for non-native speakers or just slow readers, however. So most games offer a short preview of texts that hopefully can be understood instantly – but that can create more problems again because unclear options making the player character behave in unwanted ways or options leading to the exact same line of dialogue are frustrating, so frustrating even that one of the most downloaded mods for Fallout 4 replaced these short preview texts with a complete line of dialogue.
So, just to keep in mind, if you shorten your choice texts to be able to do something like time decisions, clarity is always more important than reading speed.
If we don't think that the characters are believable, chances are very low that we are interested in what they're saying.
Oftentimes when we talk about games, it's more about the illusion of choice. It's not about real choice all the time. …It doesn't have to be real agency either as long as it is believable.
I don't think that we need a completely advanced AI. We just need to give NPC the illusion of agency by creating more believable characters.
One way to do that is to give them their own agenda. Basically, to give them a life of their own and great example for that are the NPC conversations in the camp in Red Dead Redemption 2.
“Come on, Jack.”
“I'm hungry Mama.”
“We're all hungry son. Just try reading later.”
“Gotta get some to eat Arthur.”
So this scene would also happen without the players standing so close. But the last sentence is only triggered if player's are positioned close by and that really impacts.
The feeling that the NPCs are aware of my presence, they know that I'm there. So I as a player make a difference. But what is more important is that I'm not in the center of everything. The NPCs they have their own worries, they have their own thoughts and feelings, and they just talk to each other.
In real life, many things happen at once. You can't be everywhere or experience everything that's just not possible. In The Last Express a game released back in 1997 by Jordan Magna, the creator of Prince of Persia things happen whether you're there or not. The NPCs have their own schedule, they sleep, they get up, they go to the dining carriage, and they have conversations there. And if you don't go there, they still do that. It still happens, the story progresses anyway. And this really elevates NPCs from being marionettes only reacting to the player to believable acting personalities.
A simpler approach to make NPCs feel more alive is to just let them actively seek interaction with a player. And I would call that mixed initiative, with just a simple timer in the background paired with a few variable triggers, it can seem as if the characters have a will of their own because it was not predictable when they will do that.
So if you always ignore them, if you always decide against what they are proposing, they can also just leave you. They can tell you at some point, Okay, I'm done with you. If you don't listen to me, I’ll go my own way.
But to be honest, there was one thing that could lead to really awkward situations and that was because the characters didn't have a really good knowledge about the game world, or what has happened so far. …but a game that does that so much better is again Firewatch because they have a system where the characters know quite a lot about the game world. They know everything that has happened so far. And that really influences how they behave and what dialogue choices they have. In Firewatch dialogue lines are selected by a system that tries to find the line with the most matching requirements, so they don't have a traditional dialogue tree, they have a completely different system. And how that works is, I think best explained with an example. On day two of the game, the NPC Delilah would start a conversation with a player. But it makes a huge difference how much the player already told her about his wife Julia. If Delilah knows nothing about Julia, you get a pretty generic statement about relationships. But if the player had spoken about Julia and also told them that Delilah wasn’t feeling well they heard a more fitting response. “What does she have?” And the magic about the system is that players probably won't even notice its complexity. It's very invisible in a way. But still, it really makes me feel that all the choices I make matter, and they have consequences. And the NPC respond to me in a very natural way.
Without conflict, there is no drama.
Agency is the level of control the player has over the game world. It's the feeling that my choices matter. Usually NPCs have no agency. They respond in pre-programmed ways, and this makes them feel like marionettes, which destroys the illusion of interacting with another character.
Kuschke, Julius. 2019. Narrascope 2019 - Julius Kuschke - How Dialogue Systems Make or Break Player Engagement. U.S.: Narrascope 2019.
Narrascope 2019 - Ian Thomas - Making Horror: Hacking the player's brain!
Industry veteran Ian Thomas talks about evoking emotions in the player’s head and how to closely connect player and character together, with example from video games, immersive theatre, and LARP.
- Suggestion and atmosphere is more powerful than detail
- You can use them to charge the player's emotional state
- t's important to leave space for the player's own imagination to engage with the material
- Use everything at hand to show, or invite an attitude, rather than tell people how to feel, or think
OUR BRAIN MAKES THINGS UP
It imposes faces where there aren't any. The brain is constantly scanning the environment and trying to impose patterns on it in case it sees a bear or a saber-toothed Tiger or another person who's going to stamp you. So dark shapes in the corner of a room that we extrapolate and turn into werewolves. It happens before your rational brain kicks in and knows what to do. We're good at imposing stories on patterns and we're also really, really good at lying to ourselves.
So let's take a quick example from Alien Isolation. You’re in a room in an alien base in the game, and there's a scuttling in the air ducts up there, and then it goes over there, and then an alien drops down. Basically it’s a sound effect there and then an animation, but we've imposed a story on those fragments of evidence already. We didn't see it crawling around the tunnels, but we could imagine it's crawling along the tunnels.
|In the middle of a ghost story LARP we had somebody asleep in bed and suddenly they wake up and smell burning and this fire coming up from under the door and smoke everywhere, and the door bursts open and this floating creatures zooms into the room turns, looks at her grins and then disappears again. That’s what the player reported anyhow…as it happened the floating puppet couldn't even turn its head and it absolutely couldn't grin. But because of the player’s mental state at that point, that's what they believe they saw.
A really good lesson to draw from that is to do everything in as much darkened smoke as you possibly can.
In things like Jaws the mechanical shark broke, so they could hardly ever show it on screen, which made for a much better horror movie, than if they'd had a rubber floppy shark in every scene.
Isn’t that why Aliens is an action movie? You see glimpses of the alien through bits of smoke, the body is basically a lycra suit with some panels on It, but it doesn't matter because you see it out of the corner of your eye, and the brain kicks in. And so by suggesting and just sketching out those things, the players brain does the work for you.
e.g. So you come into a room with a smashed mirror, a hammer on the floor and some blood over there and some teeth lying in the blood. And already, we're constructing stories of what happened here and how that might have come….and there's a lot of skill in environmental storytelling, not giving you all the facts because you're jumping to conclusions all the time.
e.g. If they saw the 3d model for a creature up close, and in good lighting they would think it's utterly ridiculous and it wouldn't be scary at all, it would just be a comedy thing. And we sort of reinforced that with the whole don't look at it message. Because the player isn't playing YOUR game. They're playing a mental model of the game they've constructed in their heads.
If you can see all the detail it isn't scary, make the brain do the work.
There's a book by a guy called Scott McCloud, who draws comics about making comics. He has a theory that the magic of comics lies in what happens between two panels in a comic book. It's entirely up to the reader to generate the moment between those two panels. And so that's hugely interactive, right?
For example, Agatha Christie has a line “she was the sort of woman who wore limp dresses”, which is just a great, it conjures different images in everybody's heads. Yeah. Or somebody like Raymond Chandler with his, she was a dame, the sort of game to make a bishop kick a hole in the stained glass window. He doesn't need to say anything else.
Making the player collaborate with you in building these spaces is important because it binds them into the world and immerses them in the world. It speaks to the difference between sympathy and empathy, the difference between controlling something at a distance, poking them, puppeteering them and being that person so a lot of the things we try and do is to collapse the player character down so that if something happens to the character, it's affects them directly rather than you feeling sorry for somebody else.
SHOW, DON’T TELL
In a game we made Soma, the lead character Simon is pretty vanilla very much a blank slate because we didn't want to impose too much of a personality on him because we don't want him to be too much variance with whoever happens to be playing. Another example would be this from our job the picture the second from our game God rescue Mary, which was a Christmas ghost story.
You can make a space feel magical. We did a little bit about it in this where you step into a beautifully decorated space, and there's a fire crackling in the heart, and there's the smell and all that kind of thing. So for me, an awful lot of it has to do with with mood and setting in an environment. There are also things you can do with language with characters, you can make people pay attention to certain characters, you can do things with eye contact, you can do things with gestures, if people have come across NLP, neuro linguistic programming, little sort of mirroring actions and things you can do, but it depends on the setting. It's easier in LARP, with good actors because you can use them to build bonds with character that you might not expect and then you can rip them apart. I think it's hard doing computer games but it is possible by use of the dialogue between you and that character for example, and if you've got animated scenes.
The reason fear and horror is easier is simply because of that fight or flight kind of hardwired thing that hindbrain thing kicks in.
For years we've been running laps where we form a supporting cast, we would hand out quite detailed descriptions of who you were and where you were from your greengrocer, you hate slugs, whatever it happens to be. And then when we started doing one off marks rather than anything continuous, we provide briefs to the players because they need to arrive at some context, they need to arrive with relationships, because if they don't have those relationships, you have no drama happening. They have to have secrets, they have to have all of this stuff. And so we started writing those kinds of briefs throughout our players as well. Until I went to one event run by somebody else, which is like the fifth in the series or something and I was given some briefing documents for the character I was paying to play. And it was just unplayable. It made me uncomfortable. That was the big problem is that I thought that I might perform this thing wrong and I was paying to beat it. So what did next... We had a quick back and forth with each player, which was john the player, a gardener who's nursing a dark secret or something basic like that. And they'd say, Yes, that sounds fine. And then, not long before the event, we'd send each play an individualized pack of paper. And those papers were in game pieces of paper, they're in character props. They might be a national identity card, which has things like your name and your place of birth and your age. There might be a magazine article about you a photograph from your lover, a bunch of correspondence, correspondence from other people. And we tried to build up a picture of what it's the space that your character took up in the world. What we tried not to do is to say how your character felt about any anything here and how they felt about any of this. Which meant that, for example, in one pack, there was clearly a story thatyour character had given up a baby when she was only a teenager, understood the birth certificates and as the adoption papers and all these kind of things, but we never said why she did it. Because we're leaving gaps, we're leaving holes for the player to bring their own interpretation and make the character their own and make it feel like it's them and to try and mesh the two things together.
Which meant that they bound in much more tightly to the characters and the emotion levels were much, much higher and things happen to them. There's an interesting side effect of all of this. For a start, it's still a lot of paper, it's 43 pages, but we hope it's more of a joy opening this parcel and going through it rather than anything else.
Secondly, if you forget anything, you take it all with you and you can look it up, you can go through your old letters and check things. And the third interesting thing is it's also a blackmail material, which means you've got deep personal secrets. If somebody uncovers it, they can use it for leverage. This is partly why I think first person helps horror quite a lot. Because you can't see behind the character, you're inside the eyes of the characters. Something's happening behind your ears and sound is massively important. But it's it's not just a case of being the player character. It's also the player experience.
For example again, when we were first developing Soma, the lead character spoke quite a lot to call attention to things we wanted to call attention to, for example, you would go into a room and he would go, Oh, look, there's a body in the corner. And you get this weird disconnect between the npc and the player. And so what we did was simply remove those lines and highlighted those things in other ways, and it worked a lot better.
SHOW WITH CONTROLLER FEEDBACK
It applies to things like control systems too. One of the games, you have a character who is trying to crawl through an electrified fence or a barbed wire fence, and you press one button, and you have to duck down and you press another button, then you have to bring up your neck like this. And then, and by the time you've got all the controls, Masters, your hands are kind of mashed in this weird shape around the controller. And there's so much tension built up here. And that's exactly what the character is feeling. They're completely tense. And in that situation, you've got this wonderful physical feedback loop, which is a really hard thing to do in games.
Even things like Rumble, because if you do it right, people won't even realize that they're feeling feedback through their fingers. It's just a kind of direct thing happening to them. And that’s the best thing.
We don't want the players to be thinking about our game as a puzzle. We want the players to be thinking about it as a horrific experience with a real creature in it. And again, that's partly about obscuring things, hiding things.
I’ve been to games where we've had weird ghostly effects, and a referee standing in one corner with a speaker. And that's just no good for emotion. Whereas if the mysterious noises are coming from nowhere, then that's fantastic. Similarly, if you're doing some ridiculous vision or trick, something like a floating creature or something like that, you don't want people to see the wires.
SHAPE THE PLAYER’S EMOTIONAL STATE
The best trailers get you into the right mental state to see that movie. The classic example of this is Hitchcock. When psycho first came out he issued a little booklet, which is how to survive your viewing of psycho. And he told everybody that there were two paramedics on call in the cinema in case anybody had a heart attack. Of course, it was all fiction it was just to get people in that particular mental state. You see the same in Disney as the huge build up to ghost train rides that kind of thing.
We call that crossing the threshold. It’s like going through the door, like going to an escape room. So the room has flickering lights and slightly cracked patina windows….So it can go into your box art. It can go into your advertising campaign, it can go into whatever you're doing at your stand at your convention. It can go into your UI and a great example of that is this game, Dead Space… So Isaac's oxygen meter and his health a part of his suit. They're part of the world. They're not floating above and out of the world. They're not numbers popping up on the screen. But this bit is cool, which is when he's using menus it's a hologram protected from his suit. And the cool thing about that is when you've got the menu open, it's real time and you can still be attacked from behind. So you can't be safe just by hitting escape.
In horror, it's not only about those moments of jump scares, the moments where something leaps out and you kind of leap out of your skin. What we tried to do is consider what happens when the player puts the controller down and is lying in their bed. What thoughts can we put in their head at that point, because those are far more effective long term,
So …a 1950s themed event. Players arrive at this lodge at the end of the road. They're normal 21st century people at this point. And they are greeted by some of the crew of the game. And they get changed into their 1950s outfits. And this just in the background, a doll kind of musical tone playing in the background, some sort of low level Horror Music without really paying attention to it. And as it gets dark, I take each of them individually into a room just to make sure they're happy with the characters happy with the rules, happy with how the game is going to work. And then I get each one to draw a card from a deck of Tarot. And they take the card and they look at the cards and show me the card and I write it down, had no meaning whatsoever, nothing to do with the game whatsoever. It's just starting to make them wonder about their fate…. And the second thing is I get them to stand up against the wall, close their eyes, and I take their photograph with a bright flash. Again, absolutely no reason for that in any way at all. But I had people coming to me afterwards going, I thought I was going to find my photograph in a morgue or something like that. And it's just to try and sort of push them gently from one side to another to get them to the point where you throw them into the game. And the point we throw them into the game is when they’ve been driven up by car from the local station but there's flooding so they have to get out of the car and they have to carry their backpacks themselves up into up into the dark woods. It’s night-time and as they walk up the through the woods in a small groups kind of five or six people, there's mist rising around them from the trees because of course, the special effects department are out there, creating some fog. And they meet somebody on the road who's a servant from the house carrying a lantern. And they lead them through the darkness to an entirely darkened house because there's been a power cut. And so they're led into this dark house. And they leave their bags in the hole and they join the growing group of people in the front room, and then it's a Christmas ghost story. So the front rooms actually done up pretty nicely. We've got sort of Christmas essense in there, we've got mulled wine for them to drink and they're sort of gathering there. But the rest of the house around them is dark and just as they're sort of settling into things a servant will come and tap them on the shoulder and say, Excuse me, madam, your rooms ready, please come with me. And we take that individual player out. And we take them with one small candle and we lead them by the most secure route they can possibly find through this ancient sprawling manor house up to a room somewhere we'd leave them with that candle in that room in the dark on their own. Of course, we set dressed all those rooms. And most of those rooms are connected in some way with a ghost to their theme. They've got things like the, the, the happy couple, or you open your wardrobe and there's a whole pile of photos, which are a shrine to a 1930s, movie star, all these kind of things. So that's, you know, the House did an awful lot of work for us anyway, just because of its character. But the second thing is they find their luggage has been unpacked and put away. And this is interesting for a bunch of reasons. Firstly, it reminds the player of the genre that they're in and how it works. And then they need to start following those rules, because that's the sort of thing that happens in that house. At that time period, it puts them into that mindset.
It's also a violation for the character because if you remember, they probably have a bunch of blackmail material in those bags, which we gave them earlier, and who knows what the servants saw. So it puts them on edge.
So it's a really interesting way of getting them just on the edge of jumping at shadows just on the edge of feeling uncomfortable out of their depth before we start throwing ghosts at them, and we didn't actually throw ghosts at them until I think the next day there may have been a few spooky noises. But one player left that first night because he couldn't cope with the the levels of fear.
STAGE WAR STORY MOMENTS
For lots and games these days, it comes back to this thing that I call war stories, which is what do you want people to be sitting around talking about in a bar after your game is over what do you want them to be telling other people about for years and years about this thing that happened and happened to them? Things like that moment where Hulk punched Thor, you know, we have these moments where people are going, did you see that bit where is great, but it's so much better when it's personalized. And it's a personal experience, partly because they love telling it, but partly because those tales grow.
You don't need everybody to see them actually. In fact, sometimes it's better if only a few people see them because the stories are so much better than the thing that actually happened when you get right down to it.
So we do a lot of our design around these things.
If we're working with three musketeers source material we ask, what makes you feel like a musketeer? What would make you feel like you could become a musketeer? Is it the moment when you are dueling the Cardinals gods and there's 50 of them and you can drop a chandelier on somebody's head? And if we can find ways to allow the players to have those moments, whether they're deeply scripted set pieces, or whether they are built into the rule system for them, so that they might happen, or whether we just leave the ingredients for that thing lying about
for it to happen, then that's brilliant, we've won if they come out, having experienced any of those things was make them feel at that.
It's an opportunity for moments and that's how we build our LARP games.
And it's also how we design some of the computer games that I'm involved with. At that point we can look at act breaks and rise and fall. But they're all built around these pillar moments. If the connecting material is great, that's fantastic. It really raises the bar for the rest of the game. But if all you have is the connecting material, you don't have those pillar moments, nobody will remember your story and the thing that you may and in a more traditional story format could be things like a major reversal or a twist, or it could be a hugely emotional moment between two characters. It doesn't have to have explosions. It just has to be a twist.
And to do this, my best advice is use everything that you have you sound smell, use graphics, use your cover art, use your PR emails that go out to people, badges, whatever you can, music.
Thomas, Ian. 2019. Narrascope 2019 - Ian Thomas - Making Horror: Hacking the player's brain! edited by Narrascope 2019. U.S.: YouTube.
The State & Future of AR Games: Rose-Colored Glasses
In this 2019 GDC talk, Niantic CEO John Hanke takes a deep look at AR today and helps you imagine possible AR games and experiences that can deliver persistent shared experiences in the real world.
His big tip? AR GLASSES
I am a big fan of a quote by Alan Kay, which is the best way to predict the future is to invent it, or to build it.
I know some people know us as the company that made Pokémon Go with a company that spun out of Google, which is cool. That's what we are. But we're our own thing too.
When we started we wanted to take 3d technology, new digital satellite imaging technology, and broadband and make a map of the world, unlike any map that have been made before, a map of unprecedent level of detail, built on imagery for the entire planet (google earth), so we started with that. We actually were acquired by Google along the way and with help from a bunch of other talented people inside of Google we were able ultimately to realize that vision to build a map and put the whole world on your desktop initially, and then later, we brought it to the palm of your hand.
So after we built the map, people started doing all kinds of crazy things with maps on thousands of mashup sites. St we started thinking, well, what can you do with this geographic substrate of the world?
So we started Niantic wanting to explore what happens with maps and location wearable technology … By the time we spun out of Google, Pokémon Go was in development.
During development we identified 3 key design themes
The key thing that we settled on was the idea that in every neighborhood, there's probably a story that's interesting. There's a mystery that can be built there, everywhere in the world.
Being computer people and spending a lot of time at our desks we realized that everybody needs a little nudge sometimes to go outside and get their 10,000 steps in or get their daily workout in. Good. If you get tired of sitting inside, we've created a nice space for you out there.
And we heard back from some of the early users of our games from Ingress that people who didn't consider themselves athletes really appreciated that gamified kind of nudge to exercise.
This really came from feedback, not from our own insight, but from what people told us when they were using our products. They love the opportunity to meet new people and have something to do together with friends, or family, so we adopted that.
We measured the total number of places that have been observed, as you know, special unique places in neighborhoods where people live, that people have photographed and named and described and added into this global game board which now numbers in the millions, so it's growing rapidly worldwide.
So this idea of stitching together a global game board, a place that we can play out all the interesting nooks and crannies in the world is going pretty well.
In terms of social we measure it in terms of the formal connections that people form in the game. We added the friend feature and Pokémon Go last year so you can identify your friends you can exchange gifts with those friends, you get bonuses for playing together and raids. So now there are over 190 million connections.
And we also measured in terms of the events that we hold. Has anybody been to a go fest event or a Pokémon Go community day? These events look a little bit like music festival if you've been to like Outside Lands or Lollapalooza or something like that, maybe combined with like a healthy dose of Comic Con, with a little bit of five K, if you can, like, combine all those ideas. Last year, we had 3 million people in total come to these events. We've had events with over 100,000 people. So this idea that games can be part of that festival/outdoor world is very much true.
By having these big festival style events, families learn about the history of the town in this really fun interactive way. We’re overlaying gameplay onto an existing kind of civic festival, dramatically increasing their attendance and drawing in an audience that wouldn't otherwise attend.
So we want to more events like this. And that means opening up our platform, so that many people can build hopefully really fun experiences on top of it.
BROAD INCLUSION/APPEAL IS POSSIBLE
Whenever I first started investigating the idea of us doing a game, people talked about casual games, and mid core games and casual games had a certain look and feel they attracted a certain audience, but mid core games were for real gamers, and they had a completely separate dynamic to them. We didn't want to put ourselves into a single category. We wanted to build something that was broadly appealing, but you know, we had ambitions around retention. We wanted it to be financially successful, so this was a risky shot for us.
And what I'm reporting back to you is that it is possible. I'm going to call them accessible games, that appeal to males, females, people from all walks of life, multiple demographics. Based on third party research polls we see more than 40% of the players are women. We see underrepresented minorities in excess of 30%, so gamers are not a subset of the world, but look like everybody else.
A. Short AR sessions
We know that people really don't like to hold up their phones for a long period of time. We think the right session length for holding up your phone is not more than two to three minutes, for example. So if you think about AR you want to think about it as a type of play that happens within a broader game, where a lot of the gameplay is not happening in AR mode.
B. Social sensitivity
You also need to be aware that there’s social stigma so when you hold up your phone and wave it around, it looks like you might be taking a picture of everybody who's standing around you. This is a big drawback to AR in certain situations and it's something that really needs to be designed around.
By that I mean two things:
So short sessions and really thoughtful design are important, we think the right approach here is to design things for AR that can't be done outside of AR. So. ensure that it’s obvious why you're there: you're have an experience that you can really only realize through AR.
The other thing that I want to sort of flag here is you hear a lot of hype in the industry, about augmented reality but there are inherent limitations to AR and phones. I, for one, don't believe that AR is going to take over the phone and everybody's just going to walk around in AR mode all the time. But I do believe there's huge potential, and I believe that potential is going to come in future devices like augmented glasses.
So what's the killer feature for a real world AR game?
You know, some people ask us a lot and I don't think it's the AR, I think AR is nice embellishment on top. It's the icing on the cake.
The things that we think are really the killer features are our core principles. That's why we adopted them: exploration and exercise and real-world social interactions.
MAPS FOR MACHINES
We made Google Earth and Google Maps for all of you, for people. It's designed to be a fun, friendly, accessible UI for human beings, so that we can never be lost again. We can navigate wherever we want in the world. The AR maps that we're building, are built for machines, so it's a very different kind of map in some ways. It's in the service of helping us be better human beings and those computers, our cell phones today, those computers, maybe glasses in the future. And by the way, they may be robots beyond that, because robots that want to move around in the world need that very same kind of precise localisation. So what the map does by accumulating data is to allow the computer to know xy latitude and longitude exactly where it is in the world down to an order of centimetres.
What does detailed AR look like? So you start with lots of images, many different ways those images might come into a system. But from that, you're going to derive a very, very detailed three-dimensional understanding of space. (shares recent research and development effots)
At this scale, the world is nuts. Stuff changes all the time. It's incredibly dynamic. So this is probably a map that never gets finished. Some things actually change second by second. So the idea of mapping them in advance is a non sequitur. People walking down the street. cars driving down the street, you can't create a map of that. So that's something that you actually have to understand in real time, if you want to augment the world in a way that's realistic.
So in our view, we think this kind of thing can really only be built in a cooperative way by people that are using a system so a collaborative ongoing effort to constantly map and remap the world. An evolution from photos to points to putting holograms into the world.
AR GLASSES = FUTURE PLATFORM SHIFT
But ultimately, the question that's interesting is, is it something that is meaningful with a capital M? Like is it's something that's going to affect our lives in a significant way and many people's lives in a significant way?
What is happening on handsets today is a warm up, that's the pregame. What's gonna happen with glasses in the future is the real deal, but I think it's one of those platform shifts. You see it once every 10 or 20 years. We saw PCs, we saw the cloud, we saw mobile, I think AR is one of those shifts, so it's something that I think is worth our time to understand. It's worth time and money to invest early in.
Hanke, John. 2019. THE STATE & FUTURE OF AR GAMES: ROSE-COLORED GLASSES. In GDC Vault. U.S.: YouTube.
The USW Audience of the Future research team is compiling a summary collection of recent research in the field of immersive, and enhanced reality media