My name is Matthew. I'm a UX writer at Google Daydream and this presentation is about how to tell stories as a developer, as a designer, as someone with a pretty cool tool using the techniques at your disposal.
 
My interview with google was literally the day that Google Assistant was announced. I was hired to make video games without video and at the time, we had no idea what that meant.

These days, I am working in almost the exact opposite media, immersive computing. It's virtual reality, augmented reality, anything you can see that isn't really there. Ideally, the written word will intrude as little as possible.
 
But our goal is the same: To take the user through an action experience as naturally as we can, and to have our users spend the minimum time thinking about the medium that the experience takes place in, and the most time being in that experience and participating in it and interacting with it.
 
When you play a game, you experience the story by telling it to yourself.

Any game gives you tools, a weapon a spell book, the ability to make monsters vanish by jumping on their heads. And by picking up these things, or casting them or jumping on their heads, you're telling your own story within the boundaries imposed on you by the master storyteller, the game designer.
 
AR gives you a whole collection of game mechanics, constructing, crafting, discovery and more. AR core is Google's engine for running AR, which appears on most of the latest generations of Android phones. Apple has a similar engine called AR kit. And there are a few others too. They have different features and annoyances. But basically, what they do is they look at what your camera is seeing at any given moment (plus the moments just before the moments after), and they extrapolate all this information to put together a picture of the world around you in 3D. Augmented reality is a way to see virtual content in the real world. The way I explain it to my mom is VR is real if you are in an imaginary world, ar is imaginary objects in the real world. AR core is a platform for building AR apps on Android phones. It keeps track of three variables to construct these AR worlds. There's motion tracking, environmental understanding, and latest dimension.
 
Although these design principles relate to AR Core, they still apply to all sorts of tools.
 
GET TO KNOW THE TECHNOLOGY
​
I am always looking for ways to like dig into the technology because if I can make myself understand it, then I can really do anything with it. I start to figure out how the developers are seeing this world. And therefore instead of just being like, make this happen, or can you make this happen, I'm like, hey, can you use these tools to make this happen this way?

MAKE ON-BOARDING PART OF THE NARRATIVE

Once you've established your place, you need to convince the user to detect the real world surroundings, in order for the phone to calibrate its whereabouts. That creates an intrinsic delay that's built in to mobile AR, so we need to give the user a mission. We need to give the user an excuse to discover the world and physically move the phone around. So we give them a fetch quest, which are normally done badly. Because fetch quests so often fill in the dead space between fighting, it's like you're saying to the player, the thing you usually do is run around and fight people. Now let's get rid of the fighting people part and just like run around for no good reason we've effectively freemium qualified them.
 
In the user journey that running around and the technical functionality of scanning for surfaces are exactly the same. So the idea of discovery is at the core of the experience, so we want to embrace that moment and make the narrative wedded to that, not just have it be a do this so that you can play the game, but have that be the beginning of the game.
 
So we want to make it awesome.

These AR techniques that I'm talking about, like plane finding and setting up boundaries. They don't necessarily feel like storytelling techniques, but they are.
 
ADD DEVELOPMENT, NOT REPETITION

I'm going to tell you all about one of my favourite game designers, Aristotle. Aristotle created a storytelling structure that a lot of us still use today.  First, there's an inciting incident, an explosion of birth and death, like getting lifted up by a tornado and landing on a witch. Something happens that the main character is not necessarily in control of. Then the character reacts to it. They strap on the witch's slippers and follow the yellow brick road. That's the Act One climax. That's the hero's call to action. And then act two which is most of the experience. This is the middle hour and a half of a two hour movie.  In involves a series of events where the character faces challenges, each one gets harder than the last, and each one reveals a new part of the character, a part that the audience has never seen before. That's also why if battles are too repetitive, a game gets grindy. Since it's no longer revealing something new and unseen. You're not getting character development or character building or character amping up new swords.
 
Ideally each episode or interaction gets more challenging until the end of that when a character faces complete despair. It's the lowest moment, the dark night of the soul, you must fall completely before you get up again.
 
Blake Snyder, who wrote save the cat, which is this book that all these Hollywood writers use, has this thesis.  I don't know if I agree with it, but I've seen it used in such great ways.  The thesis is that, um, you always show an interaction with the main character at the very beginning of a movie or a game or whatever you're doing, where the character saves a cat, because even if they're an evil person, they've saved the cat so you identify with them. I don't know. Let's talk about that later.
 
But then we get to the moment where we need a complete reversal from the darkest night into the final blades of glory. So that's the story of every story. It's also the story of each moment of a video. The first moments of Pac Man are filled with Tension the frantic thoughts of escape of limited motion of being eaten I can only move left or right.
 
DESIGN FOR AWE AND RESPONSE
You want to design a world that's both amazing to look at, and one that reacts to your presence. Make your world real. On the screen, you're combining virtual and real world objects. So let them interact, the more they play together and roll off each other, either metaphorically or literally, the more your world will feel like an actual living inhabited place. And let players mess with it. The virtual and real world, the virtual world and the actual world are only two dimensions of the experience. In order to make it feel real, we need to add the third dimension, the user. Let people touch, manipulate and change as many virtual objects as you can add, as many as makes sense to be changed and manipulated.
 
Here's the biggest obstacle … Users can be in one of four positions, 1) either seated with your hands fixed, 2) seated with your hands moving around, 3) standing still with your hands fixed or 4) walking around and moving around in a real world space.
 
PHYSICALITY IS EMOTION
By designing the mechanics for your experience, you can change their physical position as well as their mental experience. In other words, you can blow their minds by blowing their bodies out of their seats.
 
One of the greatest advantages we have in AR is the size of our space, it's theoretically infinite. The problem is most users don't remember that they're stuck here, right? So you want to give them something to cheat, so they actually have to move the phone around. And that little nudging icon will help them do that.
 
Now I'm going to talk about what players can actually do in AR game mechanics.
 
Like every story, you can break it down every moment into a first, second and third act, a call to action, a hero's quest and a combination. You want to take it advantage of the real world environment, put things just out of reach to users and offer them rewards for moving around exploring.
 
Hidden bonus levels are at time honored tradition, finding them in your living room gives you an extra measure of delight.
 
When you place objects in your AR scene, users will want to play with them and the more non necessary stuff they can pick up and play with the more than one a hunt for the objects that they do need to use.

But you want to give breadcrumbs too.  You can break reality selectively. 
 
SETTING IS A STORYTELLING TOOL
But as a world builder, you're empowered to decide when you want it to be realistic, and when you want to withhold that realism. If you need to draw the user’s attention to an object or an area or an evil robot, the entire world is at your command. You have lighting, shading, texture and physics at your disposal. You can highlight things you can play things down and move them into the shadows. 
 
MOTION IS EMOTION
We can use a single effect like jump scares to achieve a bunch of different emotional goals. The best jump scares, all the best moments of connection happen when you forget there's a screen separating you from the movie from the game.
 
Having the action right in front of you makes that separation even easier to forget. It's in my room. It's on my bed. It's right in front of me and it's happening with me. In one morning, you just don't want to make the user move backward without looking behind them. Because that can have real world disastrous effects. At some point in the history of games, sneaking past enemies was just a way not to get killed. At some point the game designer started recording your visibility percentage lines of sight and sneaking became an actual measurable mechanic. Just think of how recording everything in the real world can let us change enemy AI.
 
USE SURPRISING INPUTS
If your games input is the camera, then let anything you see be the input trigger, we can record degrees of light. And light is especially useful because it's so easy to manipulate when you are indoors anyway. And because it's so unexpected to the user, you can see that when the light is switched off here, cars turned on their lights and buildings laid out, like at night.
 
But it's also something that the user just didn't expect, but we have it completely at our disposal. Here’s a game that’s really cool. It's really simple. The first player chooses a spot to bury the treasure. You tap anywhere and the game buries it. Then you hand your phone to the next player and they dig around until they find it.
 
When you create or play in the world, you always have the potential to interact with other users. Cloud anchors make this uniquely possible by matching virtual content with real world locations, then serving the same content to different users. All you need is another person and a path to the shared world. This game is played on two phones and both phones are sensing in the same environment. There are techniques to do that I will show you how to find them.
 
 
CREATE MANY DIFFERENT PATHS TO ENGAGEMENT
In some way, AR is a great way to create an all access role for differently abled users to see things in their own scale. However, it comes with a whole set of new challenges. If you tell users to reach up and grab something or take two steps forward, what happens when your user can't reach the device or take steps?

Here, we've added in an alternate way for users to reach faraway objects. There's a reticle that stretches and extends based on the angle of your phone. This is a very Googley concept. This is something that we keep talking about having many paths to success. People who like keyboard shortcuts, vs people who like mouse interactions, they're people who want to do things the slow way. We create different paths to success. 

Roth, Matthew. 2020. VISUAL STORYTELLING IN IMMERSIVE REALITY edited by Narrascope 2020. U.S.: YouTube.


​

 
In this presentation Aki Jarvinen, Senior Experience Researcher, Ph.D, Digital Catapult, author of the report The Immersive Audience Journey summarises the report’s main findings and explains why he encourages creators to think about the immersive experiences beyond the story, "Framing immersive experiences under the banner of 'storytelling' in the age of ‘peak TV’ and omnipresent story content is not necessarily a working strategy. Emphasising the experiences seems to differentiate immersive. Immersive producers need to think how to blend production activities into marketing, and adopt holistic thinking around audience engagement. Community building and pre event launch activities can significantly increase awareness and facilitating post event activities contribute to loyalty".
 
Download the full report here
 
The original need for this research emerged from the question of audience. There was a recognition that we lacked insight regarding what is immersive audience, or what are immersive audiences.
 
What is immersive?
 
Whereas producers tend to associate the term with particular technologies, audiences tend to associate it with stepping into a responsive and alternative reality that has been artificially created for them.
 
Immersive technologies are typically good at creating these kind of experiences. So immersive is many things, and can be created using a variety of technologies. 
 
Our analysis highlights a common, high level customer journey map approach which has traditionally been employed in marketing, service and design thinking contexts, but we want to encourage more thinking about this throughout the production process. 

If immersive producers are able to leverage this more holistic approach, we think that it contributes to a more structured planning of the project which puts it in a better place to succeed. There's so many things grabbing our attention, so alternative distribution approaches are important. One definition from service design literature says that a human centered tool, like the journey map not only includes steps where a customer is interacting with the company, but also reveals all the key steps of the experience. So journey maps help us define gaps in customer experiences, and explore potential solutions.
 
The Unfolding Immersive Audience Journey in Phases:

Phase 0: Segmenting Immersive Audiences
 
What we are finding is that the immersive audience is a collection of subsets of audiences, ranging from festival goers who happen across an immersive production exhibited in that festival, whether it's online or location based. Immersive audiences include gamers who are into VR. They include casual AR mobile AR audiences, but also more niche cohorts that might be more profitable and engaged like secret cinema enthusiasts. immersive theater enthusiasts, even VR documentary enthusiasts. The key thing to understand is that the context defines the audience for any given production. So for instance, if it's a location based experience where is it delivered, exhibited, but also on which digital platforms, as the device might dictate and shape your audience, but also social dynamics. So is it something that enables friends and families to go together?
 
Already we are seeing some clever solutions for post Covid-19 engagement that facilitate for instance, a family or a household come in at the same time to a location based solution.
 
Phase 1: Reaching Audience Awareness

This is the phase where a potential audience member becomes aware that an immersive production exists, which requires promotion, marketing, and community building. And the key takeaway, I think, from this phase is that immersive productions are challenging to market by traditional means.
 
In the age of peak TV, where there is an overwhelming amount of streaming television storytelling format content, framing your experience, as the peak in immersive storytelling is not necessarily the most strategically sound choice.
 
Instead you need to emphasize the unique component of your immersive experience. And that tends to be immersion, whether it's the sense of presence that you feel through, let's say, VR, or the physical set, with projections and props. However, the paradox here is that creating awareness around this isn't very easy to do with traditional marketing means. It’s  hard to communicate what is special about an immersive experience via print, or video marketing.  You need to be creative.

My one takeaway would be that you should show audiences engaging with the experience in a unique way in the promotion video.  Don’t just show the visuals of the experience because that also factors to the next phase which is consideration.
 
Phase 2: Audience considerations
 
After becoming aware, your potential audience starts to weigh up whether to attend, or participate, engage, and pay for your experience.  Managing their expectations and almost literally telling them the degree of expected interaction is very important. There is also a duty of care, to show that you will be on-boarding them through this and taking care of them during it as well. This may require that you lower the threshold of attendance explicitly in certain cohorts, and this relates to technology acceptance, so how comfortable people are putting on a headset or engaging with a technology that they're not familiar with in their everyday sort of media, and technology repertoire.  Multiple things go into those considerations, including the genre and type of content and experience.  For example, to engage a broader audience in the thrill seeking aspect of that experience, you might engage your comic lead in your communication and that just might work for you that they can connect with that certain niche.
 
Putting your production out there takes time, money and resourcing.  You need to have people on hand to help participants go through that experience. This is one thing that I feel requires further development on the digital side of things, because people are just not comfortable trying to engage with new equipment like headsets on their own. So how could you facilitate that process? How do you make it as easy as possible to get on board. Also, can you draw attention away from the technology towards the experience itself?  That's another element that you want, might want to do.
 
Phase 3: Evaluating Audience Experience
 
The decision for an audience member to attend or not to attend might contribute directly to your bottom line and your opportunity to continue the work. Therefore, you need to be aware of the different dynamics that go into these decisions. For instance, the size of a market that you are targeting is one important factor. At Digital Catapult we're trying to get more actual data about Immersive Audiences size, demographics and limits to help guide startups.
 
Important factors include price and location. A location can be a platform too. So if an AR mobile experience is only available to Android or iOS, then that shapes your audience. A certain kind of headset shapes your audience and price factors into that too, as well as more qualitative characteristics such as technology acceptance in a certain demographic for instance, or the fear of missing out regarding this experience, which might be something that you want to deliberately build. But there are other considerations, like the expected level of interaction.  The whole point of an immersive theatre piece might be to take it out from the theatre to the streets and make it a bit more chaotic if you will. But then some of the audience members complained that they didn't always see all the actors and therefore didn't always know where to direct their attention. So if your target participants are more used to traditional theatre where you have a good view to the stage and all the techniques that have to do with lighting and drawing audience's attention to this on the stage, immersive disruption can violate those expectations. If it does that audiences will leave unsatisfied and won’t recommend the experience to others.

We have a tool available to help you test your interfaces.
 
If the intention is create an experience which gives participants the option to make choices, ideally they will then come out of the experience feeling that it was unique and personalised. But people bring their own personal context, habits and expectation to every experience that will shape how fully they interact with the piece.  So if there's a lot of self-interaction, there might be people who just are not familiar with those conventions of interaction, and they might come out of the experience by feeling that they didn't get as much as somebody else. That might feel like a design failure.
 
User testing is a skill set and a set of know how that small studios don’t necessarily possess, so it tends to get de prioritised in production. In the interviews that we conducted, nobody discounted the need and potential usefulness of user research and testing, but because they weren't well versed in the methodologies, production itself tends to take over, with ad hoc user tests conducted very close to the end. And that also means that their methodology might not be entirely sound, which then might lead to skewed results. For example, the testing cohort might not be representative of the target audience. So it's just another example where time and resource needs to be allocated early on to this and there needs to be somebody leading on it as well.
 
In larger organisations like museums, there might already be a culture of user research. So, they have experts in house who gather user insights from past projects, to inform a new project and then advocate those insights along the journey.
 
And they have a very good sort of set structure of where at which phases is production tested, and how that feeds into the process. So we hope to assist people find these relatively accessible methodologies and tools.
 
But it is very challenging to be able to assess if there's any longer term impact of an experience, particularly in terms of behaviour change over time.
 
Phase 4: Satisfaction and Loyalty

This is another moment of truth, satisfaction and loyalty. It’s very important especially if you hope people will re-engage or recommend the experience to others. So, if you want to create so called organic growth word of mouth around your production, then you should think about how to facilitate that.
 
This is where community management comes into the picture. To create a community around your product, where people share stories about using the product and experiencing it. And they've tried to leverage this by engaging the community and listening to them, and so on so forth. But again, somebody has to do, perhaps even a small team, and allocating that responsibility can be challenging, but down the line, it does pay off in most cases.
 
Another thing to consider is merchandising. This approach has been tried and tested in the amusement park industry. Merchandise can reinforce a participatory experience.  By buying a T shirt, an object, a piece of memorabilia that they can take home, audience members might also contribute to that word of mouth.
 
And the other benefit from thinking about and planning for merchandising is that it creates another complimentary revenue stream for your ticket or download revenues. Again, this is something that needs to be planned in order to be effectively executed.
 
TEMPLATES

This audience journey is an approach that can be applied in practice, and can inform your thinking about your production in a holistic manner. So therefore, we have included two templates to the report, and also they will be available to download separately as printouts.
 
Template 1:
The audience journey template which illustrates the various touch points to consider from the point of view of the responsibilities and stakeholders involved in different phases of the journey.
 
Template 2:
In a 2nd template the user journey is also mapped from an emotional perspective, to help you think in a more deliberate way about how to take your audience from being indifferent, or casually interested, let's say, to being excited and waiting to see your production and then also possibly eager to re-engage or hear about what you're going to do next.

Download the full report here

Jarvinen, Aki. 2020. The Immersive Audience Journey: Insights and perspectives on immersive art, culture, & entertainment. edited by Digital Catapult. U.K.: Digital Catapult/YouTube.

Melissa Kagan is a game studies academic and incoming Assistant Professor of Communication at Curry College. She's published in Game studies, Convergence and Game environments, and she serves as an associate editor of the Journal of Gaming and Virtual Worlds. Her book project Wandering Games is forthcoming. 

---------
 
When I started analysing wandering games I started thinking about, particularly walking sims and gender, and how walking sims get gendered female. Some work was starting to get done on walking sims as queer spaces and I started to get really fascinated at how walking sims are usually defined by their lack. The definition that I cobbled together is that they’re exploratory non0-violent games without points, goals or tasks in which the undying first person player character wanders around a narrative leverage space.

The term walking simulator originated as this derogatory smear intended to denigrate games that were less violent, less task oriented, or less difficult to complete, which immediately implies this sense of lack. They're not violent, they don't have guns, they don't have tasks, they might not have wind loss conditions, there's something missing, which in part comes from their history of being modded. 
 
But, the more I thought about this, the more I wanted to rethink Walking Sims and reclaim them as wandering games.  Wandering games don’t lack something, rather they draw from a vast intellectual history.  So over the last decade, "walking simulator" has become a catch all term for games that are interested in alternative modes of expression, and alternative considerations of embodiment, environment, orientation and community. The genre now serves as a catalyst for debates about anti game ascetics, changing gamer demographics and the radical potential of poetic spatial storytelling in video games. In process, the term walking simulator also accidentally tapped into something brilliant: The vast heritage and intellectual history of the concept of walking and wandering in performance, philosophy, pilgrimage, protest and literature.
 
Originally my book was going to be about walking simulators, but I shifted it towards wandering games in order to bring this intellectual heritage explicitly into the discussion of walking. I'm interested in showing how this genre has inflected and continues to inflect some of the most interesting hybrid games of the last couple years.
 
These games are connected to the concept of wandering as a theme, a formal mode and aesthetic metaphor, a player action. And through troubling the concept of wandering, They tap into what I would consider some of the most crucial conversations going on in game studies today, wandering in games exposes the multiplicity as possibilities of a simple human act of moving through space, and complicates what such movement might mean within different game worlds.
 
Labor and capitalism: 
How does wandering games attempt to reinstate a radical boundary between work and play? How can we understand the reaction against wandering games within gamer culture as something like a resistance to any game that criticizes the unthinking replication of capitalists success paragraphs, if most games are premised on this same paradigm?  Are wandering games binge designed to provoke unproductive contemplative, anti-capitalist play? How does the construction of empty landscape in walking Sims replicated colonizers understanding of space and place? How does the heroic monomyth of the wanderer and the ubiquity of exploration with narrative gaming, invite colonialist game design? How does the negative space violence of an empty landscape connect to broader conversations of violence in gaming?
 
Gender and sexuality:
How and why are wandering games often gendered female? And how does this gendering map onto long standing discussions of female agency and presence in the public sphere? How do wandering modes queer traditional video gameplay, and finally death and violence? How does the central tension of simulating a traditionally an undying Player Character traversing a dead haunted world help us to understand ludic conventions, metaphors and obsessions surrounding death, and play?
 
A lot of the wonderful work that's already been done in this space has been done on walking simulators.
 
Right? Okay? So, very ambitiously, I'm now about to take you through one slide per chapter.
 
Introduction
An incomplete background about wandering as a concept for games before digital game, and then how those discussions and discourses get kind of imported wholesale oftentimes implicitly into contemporary video games.
 
I include a survey of the digital game genres that developed conventions surrounding ludic exploration, and the embodied performative game genres that enable players designers and landscapes to co create narratively rich spaces through which to wander.
 
Then I zoom out a little bit more and look across the 20th century In general, the array of artistic and activist practices related to wandering, which are relevant to the development of walking, including 20th century performance art. So interactive insights, theatre, situationist international Capra's happenings. I also look a little bit at digressive landmarks of literature, particularly modernist ones that demand respect for meandering, purposeless anti-capitalist, leisure. And then third, an activist understanding of the connections between wandering and marching when does wandering in a group turn into marching occupation resistance protests I draw from Rebecca Solnit Sarah Jean Servin ack, Carla Wallace and Lauren Elkin in particular.
 
Late capitalism

This is a pun, I'm interested in thinking about the corpse as media ties and archival object. Wandering games are archival adventures. You're working through big fields of different kinds of text and you're piecing all that text together like a researcher in an archive.

So from there, I go towards some thoughts about death and walking since in general, wandering games have a reputation as these peaceful ludic environments. Oftentimes, that's a major complaint about them. But I argue that these games fully participate in the tradition of death and violence that characterizes the history of video games. The difference is time. In so many games, you walk into a peaceful environment and you kill everything in it.  (Overdone – where you discover and try to solve the mystery of a ship full of dead bodies) assumes the opposite, you walk into a peaceful environment in which everything has mysteriously already died. IN other words, you’re not divorced from death, you are instead exploring the aftermath of death rather than causing it directly.
 
The archival poetics of the game hinge on the mythical existence of that impossible moment when the living person transforms into an archival and archival object.
And finally, I conclude by discussing what that process of objectification means, in the harsh capitalist world in which every death no matter how bloody must eventually be reduced to a number in a ledger.  In this chapter I draw a lot from Margaret Schwartz’s Dead Matter. Jennifer Murkowski’s Dying in Full Detail. And Amanda Phillips work on macro politics, which is an article rather than a book.
 
I also discuss Each Shade, an early 2019 beautiful game where you wander around being a traveling artist in a beautiful place, painting it. You take commissions for your paintings as you explore the whole island get to know everybody. It's so lovely. It's perfect fantasy, if unrealistic. The maker shade rebels in this gap, offering the player a fantasy tailor-made to soothe the particular fears of precarity in 2019 2020, and doing so with such charm that the player hardly realizes that it's happening.
 
So this chapter looks at how this game is delivering a fantasy of late capitalist precarity. I start by jumping back to the 19th century, and tracing the aesthetic socio-cultural mythmaking surrounding the wandering artist, a character who became increasingly important during the 19th century. I show how the players response to East Shade landscapes relates to the aesthetics that developed during European romanticism, that particular art movement. And then in addition to the romantic ideal of the wanderer, I discuss how the player is also put into the role of that character’s much more realistic cousin, the mid 19th century artisanal journeymen who migrated in search of work.
 
The experience of playing East Shade is this constant back and forth between the dream of being a wandering artist and the reality of finding commissions and managing all of these different economies, but you have to keep track have largely in a traditional RPG sort of way. But when juxtaposed with the promise of the fantasy of just being a wandering artist and painting, whatever you like, it's really interesting how much of your time is spent, you know, building up your relationship with pivotal NPCs and making sure you have enough inspiration to paint the paintings that are on commission even though that's not what you really wanted to paint and so on and so forth.
 
So I analyse the key economic considerations that drive the player (and designer, and game studio). And I read their actions within the context of millennial anxieties. Here I'm drawing a lot from Ergin Bulut’s A Precarious Game: The Illusion Of Dream Jobs In The Video Game Industry, a super fascinating work on favour, parity, and the neoliberal instrumentalization of love and passion, the dangerous promise that of the do what you love ethos, and how it's perfectly designed to produce and reproduce a creative precariat.
 
Gender

I then turn to consider feminist considerations, first in the context of moon visuals as a queer feminist take on wandering. In the game Under Constraints, a player character is a powerful witch, who has been banished from the entire earth and relegated to a tiny circle of movement, which she walks and re-walks daily. Each day she conducts an identical ritual and decides whether or not to protect the Earth today against this apocalyptic comet, which we see about to crash into the earth. Every day, the player must remember to play, but it's constrained to play for only five minutes at a time, you have to play every single day for 28 days, but only for five minutes each day.
 
So when such constraints are placed around movement, and then mirrored by the game, they force the player to have constraints around time.  The player is denied the chance to exert agency over space, or to wander freely in this game. What is left? To discuss this I construct wandering as a metaphorical concept, not a physical one, the practice of which emancipates those to whom society gives no spatial outlet. Internal wandering enables a rebellion against an external order dedicated to the immobilization of black and female bodies.
 
Connotations of freedom are defined in relation to the unfreedom of others, freeing certain people in other words, only through the subjugation of others into non wandering immobility. Explicating the ritual of the moon through this lens shows how feminist game studies, queer game studies and disability studies can help us to conceptualize repetitive steps in circles as a powerful and alternative mode of wandering – and create places also where multiple temporalities intersect.
 
Decentering

After this I think through three different ways of traveling through space as the explorer in 80 days, connecting each one to the heroic monomyths of the wonder and ubiquity of exploration within narratives. Obviously, exploration is not something new. What I'm interested in, in this chapter is thinking through different types of exploration and how all are not equal when it comes to age, or colonialist outlook.
 
So first I show how narratives of progress and colonialism are woven inextricably into walking simulators. In this context, the player travels through the colonizer with a predatory gaze seeking exploitable land. The perception of landscape got imported from numerous first person shooters and a lot of Walking Sims are still proceeding through landscape in a similar predatory, exploitative way that often goes under the radar.
Imagine, you’re entering a space where you're not the one doing the killing, the exploiting the colonizing, taking over. But your perspective on the space is still one of taking and looting.
 
So, right. The second way of traveling through this game doesn’t do this. The protagonist offers an experience of peaceably traveling through a complex world, a world that is much more complicated than the protagonist. I love the idea of deconstructing 80Days (discussed in another panel) or trying to make all the wrong choices. I find that so hard to do as a player, so I appreciate you doing that work, that worked for me.
 
Many Giants has spoken and written quite a bit about the intention to invite post-colonial play by intentionally disrupting 19th century in our activities, dissolving the players expectations of class colonialism, heterosexuality, and creating expectation that the player is not the center of the world. They're traveling through a complex world rather than centered within it. I analyze the subversive choice of a queer, lower class cleric character, while keeping sight of the fact that he's still a white European man. And I try to speculate on critical play of colonialist and post colonialist works.
 
Finally, in the third section, I turned to a hyper colonialist board game, based on the same narrative, the first Ravensburger board game published in 1884.

This is a board game that's based on the genre goose games, basically, it's the Spyro game. Nothing to do with geese, very sorry to report. But it has a spiraling board. And I argue that it is hyper colonialist not just in its text and content, but it transforms spatial travel into time travel. The player is traveling through time rather than space.  The board represents one day rather than one place through which one can travel. And in this way, colonial progress is made logically inevitable. Believe it or not, in this era, dice and the whims of the dice were regarded as unacceptably subversive or could be with enough bad luck and enough bad dice rolls. Maybe the player would not land on the space that they wanted to invade or takeover whatever it was in the diegetic parlance of the game. So by taking that option away and saying no, no, each square is just a day rather than a place, conquest is inevitable because time will pass and the player will continue on the board.
 
So these three different ways of wandering through game worlds as exploitative colonizers or prospective colonizers, as curious travelers, and as inevitable conquerors illustrates how different kinds of lunar exploration effects can reflect different relationships towards space.
 
After this I analyse Heavens Vault and show how language can create spaces for confronting death and colonialism through play. Basically, I'm trying to think of language as a space to explore as well.  In Heaven’s Vault so many different kinds of language are available and even required from the player. So, it happens to be an archaeologist Aliya who roams around the nebula following the traces of a missing person. The more she learns of sneaky Brenda's disappearance, the deeper she gets into the mythology of the nebula, a universe of looping time, of use robots, fallen and forgotten empires, and a language called ancient which is inscribed on short phrases on many of the artifacts that Aliya encounters during her travels with each human visited, and each inscription decoded. The player gains a little more knowledge about the language and is able to understand the past and it's eraser a little more completely.  The knowledge of encroaching death in the Forgotten past, and maybe the near future because of the time loop mythology or religion of this game gives weight and meaning to the players understanding of the game’s language.  The threat of all-encompassing imperial power, which is made manifest in the way language works, which I'll get into a little bit more in a second.  So by conceptualizing language as a space co created by player and game, I draw from theories of digressive literature to posit the player as an anti colonialist wanderer in a world made of a dozen different types of language and through the way that they navigate that linguistic space and mix them together through their digressive passage through it, they're doing anti colonialist work
 
The main mechanic of the game is translating the glyphs of ancient into English. So one is attuned to thinking about language from the beginning and the player is making charts and inferences about what the grammatical systems are. And then in the process of playing the game, there's all of this cultural code switching that Aliya does. They're switching between the speech on the colonized worlds for the player care where Leo was born, the player character, and then there's the speech of Empire, which she uses professionally, in the university where she works. In some cases, her specific cultural knowledge is what cracks the code and makes semantically legible, something that would have been incomprehensible otherwise. So she's going back and forth between different kinds of literacy and legibility, all of which are necessary to her solving these translation puzzles. How is it different for a machine versus a human to read a piece of text? And this then becomes a question of identity as well. Since one of the main characters in the game is a robot with sentience, personality, emotions, quite an intense personality actually. So the ways in which a machine can read something are different than the ways that Aaliyah can read and can be read. In navigating these identities, and these various kinds of text, the characters make manifest the many kinds of linguistic fluency, legibility and communication that the game offers, in which the player too has to learn how to navigate and read.
 
Digression

I also want to make a note about digression. The notion that if one keeps talking, keeps writing, death can't take you. And so the result is digressive literature because you've just continuously written and talked until, well, you can't stop, you can't stop.
 
One of the texts I'm drawing from here is Astrid Ensslin’s Literary Gaming, particularly the work on Primo and situationists.
 
I also discuss Death Stranding, a game where the player wanders around a post human, post apocalyptic, post death world. The player is made hyper aware of the particularities and limitations of body as they re-attune themselves to their new physicality throughout the game. In Death Stranding you need to manipulate objects on to the main player character Sam and be aware of how the weight of them changes his ability to move through the world. So he roams the devastated landscape of what was once the US delivering packages to a bunkered down populace and connecting each city to a network that is basically the internet. The world is also infested with ghostly creatures. We’re either in some sort of post death moment or death doesn't really exist, Sam cannot die. Sam goes into any sort of afterlife mode and then comes back and because of this ability his entire body is capitalized on shall we say, his blood and his urine and all of his exertion is sort of monetized in the fight against these ghost like creatures. So Sam is between life and death. The world is between life and death. So many of the characters are crossing back and forth across that boundary and the experience of playing the game is gives us an almost offensive sense of connecting with each bunker and going between one task and another task. And to me that is a smokescreen to try to make the game commercially legible. I think it's really about the horror of his perpetual motion in-between. He's always in the process of wandering from one place to the other.
 
Because of all of this monetization of every instant of every characters or fans in particular life, and physicality. I'm drawing most explicitly here from Alenda Chang's book Playing Nature: The Ecology of Video Games, especially the last chapter on the ecologies of games.
 
Wandering Games is due to be published towards the end of 2021.
 
Further recommended reading:

- Consalvo, M. and Paul, C.A., 2019. Real Games: What's Legitimate and What's Not in Contemporary Videogames. MIT Press.
- Reed, J.E., 2020. Feminism in play: edited by Kishonna L. Gray, Gerald Voorhees and Emma Vossen, Cham, Palgrave Macmillan, 2018, 271 pp.,(ebook), ISBN 978-3-319-90539-6. Feminist Media Studies, 20(3), pp.458-459.
- Shaw, A., 2015. Gaming at the edge: Sexuality and gender at the margins of gamer culture. U of Minnesota Press.

Kagan, Melissa. 2020. Narrascope 2020: Wandering Games. edited by Narrascope 2020. U.S.: YouTube.

These heuristics were then mapped thematically, and those groupings were evaluated in the 1st instance by 3 design experts.  Following further adjustments in response to this 1st round of feedback the heuristic themes were re-evaluated, this time by 5 experts, to identify any doubling up, or lack of relevance.  The heuristics (14 at this point) were then tested in practice during the design of two AR applications.  The experiential insight gained through application revealed a few more inconsistencies and overlap, which inspired the researchers to streamline them further into the 9 heuristics presented above, which also underwent a 3rd round of evaluation by 5 expert reviewers to assess inter-item consistency and inter-rater reliability before being finalised.  

Endsley, T.C., Sprehn, K.A., Brill, R.M., Ryan, K.J., Vincent, E.C. and Martin, J.M., 2017, September. Augmented reality design heuristics: Designing for dynamic interactions. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 61, No. 1, pp. 2100-2104). Sage CA: Los Angeles, CA: SAGE Publications.

I almost think of it as this thing that happens when your cognitive capacity has been overwhelmed. You're trying to comprehend something that has a magnitude that you almost don't have enough neurological connections to deal with. And … your brain sort of overflows ...
 
In this summary Michael Frederickson, lead technical director at Pixar Animation is interviewed by Douglas Rushkoff from the Team Human podcast, to talk about this deeply human experience of awe.
​
It’s a conversation that spans the “awful” to the “awesome” and all those ambiguous spaces in-between. Rushkoff and Frederickson dig into questions of technology and storytelling, the narrative arc, and the evolutionary, even empathetic value of having our minds blown.

MAIN POINTS:
1) Awe - inspired by vast and novel scenes - has a narrative function.  The experience of awe encourages people to really take notice of the world around them
2) Awe, also creates a narrative rhythm.  It opens people up to new insights and experience.  Awe is different from spectacle, which can attract, but also overwhelm people.


How did you come upon the notion of awe and what sparked your fascination with it?
 
Awe ended up being the confluence of just about everything I found myself interested in over the years and it almost became like this inevitable thing to look into, especially after some of the work I did on Inside Out a couple of years ago. For me, it was precisely how difficult it is to articulate what the experience of awe feels like, what it is and why it exists that got me interested in it. I think my entry point initially was looking at why a director or any kind of storyteller might be trying to draw out a particular emotion in us. I felt like every time I would ask someone about awe, rather than saying how it made their body feel they'd talk about the experience that caused them to feel awe and I set out to discover what research has been done on this recently.  But also, if you try to apply that to storytelling when and why does it make sense to try to make people feel awe?
 
Right, so if we think back there are these moments in movies where you come upon an awesome sight, like the world of the Avatar people or the roof-tops of a great city like London when Peter Pan flies out at us, or when we first see the giant panorama of a new city in The Game Of Thrones …so there are these moments of awe that storytellers tend to put in to a story?
 
Yeah. And I was curious why we were doing that? Talking to different artists over the years and myself as an artist, I've found people who have different levels of comfort with how much they want to talk about the conscious act of producing art, like some people seem to be made really uncomfortable by talking about having almost mechanical tricks to producing something creative.
 
Because it feels more contrived if you've manufactured it?
 
Yeah, a lot of artists seem to be concerned with creative spontaneity and regard it as this thing that's hard to articulate, and it's almost a little bit ephemeral and hard to grasp. But it seems to me that some directors and storytellers are instinctively using awe at specific times in a story arc, and I’m curious why. The first step in that process for me was understanding what awe is as an emotion and that led to some research that defines it as something you experience when you encounter something that's both vast and novel. That value doesn't have to just be spatial vastness, like with the Grand Canyon or something like that. It could also be the vastness of somebody's talent, the vastness of time – something so vast it exceeds perceptual limit. Like I remember feeling it when I saw the Sagrada Familia in Barcelona, and it was both huge. And also, I was just overwhelmed by the amount of time that went into constructing that human effort. And then also novel, I mean, if you're seeing the Grand Canyon every day, you're probably not going to feel, awe every time.
 
One really helpful thing to do is ask what physiological change you go through when you're experiencing an emotion.  There's not a universally, cross cultural agreed upon expression that one makes when they're experiencing awe. But it tends to be head forward, mouth agape, eyes open, which points to the idea that you're trying to take in as much as possible, you're trying to taste and smell and see as much as you possibly can. Because the thing you're taking in is so enormous that you want to maximize your sensory input.  People describe feeling goose-bumps and there's actually some studies that say that awe can ultimately reduce inflammation, so If you're super inflamed, you know, go check out the Grand Canyon.
 
But the thing I got really interested in was recent work in evolutionary psychology which asks questions like, what might be the purpose of you, why has evolution selected us to continue having this very peculiar feeling and to what advantage is it to look up in the sky, for example?
 
I was also thinking about the root of the word awesome and it's, it's interesting because to be accurate, if something is full of awe, then we could say it is awful. Which is interesting, because it almost speaks to the history of the emotion. Researching emotions there's this term I've come across, which is the valence of the emotion, which refers to whether or not it's subjectively considered positive or negative. So, most people would describe happiness as having a positive valence and sadness at least initially, as a negative feeling. And most Westerners describe awe as being a positive emotion, which is super interesting because in the same way that comedy and horror exploit our experience of surprise, there’s a razor thin line between those two things. I think it really just comes down to the stakes, when awe is positive, you're experiencing something vast and novel, but something that is not mortally threatening to you.  Yet a long time ago, when the world didn’t always make sense awe was a more negative feeling
 
Despite the importance of having emotions that we can’t control, the study of their contribution to our humanity has traditionally been limited to the realm of poetry and philosophy in the arts.  But some of the earlier research pointed to the purpose of awe was to help us organize into groups. So maybe you'd feel awed by the power of a really strong leader, and that would make you feel humbled and part of something bigger, and then maybe that might help you organize and join a community that would then have a greater chance for survival.
 
When you use an example like that all I can think of is Hitler at the Nuremberg rallies, you know, shooting aircraft lights in the sky.
 
I think with any emotional response like that, you can either use it for good or evil. Yeah, I've often thought our proclivity to recognize lots of different things as faces is one of the reasons we can deal with animation and the abstraction of animation. It's because you can buy into that being a character, when it is not at all close to the real thing.  This reflects our need to accommodate something within our model of the world. This is one of the reasons that magic is entertaining, because you're kind of exploiting the fact that when something levitates you so desperately and instinctively need to square that up with your understanding that things don't float, and you're like, almost instinctively driven to explain how is that happening? And that's one of the things that sort of makes magic entertaining.
 
On a certain level, we're talking about a kind of emotional cognitive overload that throws me into a different state. Now, if it's a dinosaur popping out through the woods I'm going to go into fight or flight and overflow into terror. Whereas if my wife or my child encourages me to see a sunset in a way I've never seen it before, I start connecting things and I can ease into a state of awe.
 
So then there's the question of why. Take a very simple assumption that people might have about the world like what might be on the table at a romantic dinner: A candle, wine, two pairs of hands, a rose.
Most people assume that candles would be on the table. In one study, the researchers exposed three different groups to emotional stimuli. So they showed the awe group that Powers of Ten video from the 70s, where they sort of zoom out as far as possible into the cosmos and then down into the human skin. Another group saw somebody unexpectedly winning a gold medal. They were supposed to feel happiness or joy. And then another group, a neutral group just watched someone assemble a cinderblock wall.
They then told all the groups the same five minute story about a romantic dinner, where they very specifically did not describe candles on the table and gave them a distracting task after that, and then asked some follow up questions, one of which was, hey, were there candles on the table?  And after the study was done, they noticed that the awe group consistently did better than any other group at remembering that there were not candles on the table.  So one of the potential conclusions that came out of the study was that maybe by making you temporarily feel humbled, in relationship to something vast and huge, maybe awe has this temporary effect of stunning you and reminding you that you don't know everything, so you should look at information, new information for what it is and not apply a kind of bumper sticker reaction to it.  So, when things are going fine in your life, you might hear about a romantic dinner and go, Oh, I know all about romantic dinners, there's going to be candles on the table. But if maybe you've just been made to feel small and humbled you decide to really listen closely to the story, and to be alert to what’s actually there rather than apply pre-existing assumptions.
 
That's really interesting, because my senses are assembling movies all the time and a lot of the times we assemble stuff that isn't even there, we just assume that it is. So awe almost wipes the slate clean, it's almost like a rebirth.
 
Because you've been cognitively overwhelmed, you've effectively blown your mind, so maybe now your mind is kind of clear for a few minutes and can take in new information afresh.
 
I found this some this quote in an interview with Kubrick about The Shining, where he seemed almost a little bit stressed hoping that people didn't just go see his films and take away what they already believed based on their pre-existing models of the world. And he said, You know, I wonder how often people are fundamentally changed by a piece of art? He said, if you're 15 or 16 years old, you might be ready to do this. And I loved it because he was talking about his use of the subconscious in The Shining, he does all sorts of tricks like messing with the spatial layout of the hotel and all these things to kind of, say, have a new emotional experience. And so as soon as I heard of this study of awe I thought, maybe this is part of the answer. Maybe one answer for why we might want to go through the emotion of awe is to prime us to internalize the lesson of the story.  In other words, maybe all these TV shows and movies that inform resolution on us, even the ones that are supposedly strange like a Westworld or Memento or Inception still suggest that there's a right way to understand what happened at the end. Whereas with a Kubrick movie it's like, wait a minute, what happened here?
 
Maybe real art makes you question the world you're in and keeps you in an alert, open state?
 
And when I hear that I hear you setting up a dichotomy between reason on one side and passion on the other. I really believe that the best stuff is somewhere in the middle. It has a tasteful contrast and balance of both. I think Kubrick, for example, walked a great line of being able to have enough structure that his stories weren't just, you know, experiential nonsense that not many people could relate to.  Trying to keep those things in harmony is a really fun challenge.  I mean, he did screw with us in that movie. I mean, there's the scene where the old guy is sitting in his room thinking about a boy and sometimes you look at him and there's nothing behind him on the wall. And then you cut back to him and there's this velvet painting of a naked woman. Is the picture really there? No, he's doing something to our brain, intentionally. That subtle realization that something's not right adds to a feeling of eeriness. So you're playing with things that are just below that line of conscious recognition.
 
I think there are certain I don't want to say morals, but messages that are almost incompatible with the traditional kind of hero's journey. You know, if you just give people the standard version of the Hero's Journey I don't necessarily think you can deal well with what most of life is like, which is ambiguity and having to wrestle with a lot of unclear, subjective things. So it's made me wonder, are plot structures or games where you go around and explore and you're not following a linear narrative, potentially better suited to messages that aren't really cut and dried all the time?  Indie films where it's very unclear if the main character lives or dies at the end, or what the conclusion is don't tend to do as well commercially, because people like some resolution. Well, in a film, I don't have any interaction with the narrative, I can't affect it, so it feels good to have resolution. But I wonder if more interactive storytelling might be better suited to ambiguity, because at least you have a role in it, and you have time to speculate and think about how things are unclear because it resembles life a little bit more than a clear cut story.
 
I'm intuitively suspicious of virtual reality type entertainment, partly because I understand that it's really two or three corporations that own 99.9% of the VR tech, whether it's via Google Facebook, or Sony I can't imagine that they’re using it to try to open up new avenues of human imagination and intelligence.  They're more oriented towards human manipulation. But, you know, many people in VR worlds are yearning for this verisimilitude, this granular copying of the real. I'm wondering how open ended they'll be willing to make those experiences?
 
Well, this is why I tend ignore their efforts and focus on those people exploring how to make an experience that helps someone empathize more or that puts participants in a situation that they otherwise couldn’t experience, in order to think about it because now they have agency and a different relationship to it.  For now, people need to make lots and lots of VR sketches to say is this mechanic enjoyable? Is this humanizing? Is it dehumanizing? How does it interact with our emotions? Does it get a point across to people.  If it feels good, it'll survive and hopefully, you know, that doesn't get co-opted again to dehumanize us in some way. But even if that does happen, I'm confident that the people who are doing it with good intentions are gonna keep doing that.
 
There's definitely a difference between awe and spectacle.  You can create a spectacle that throws someone into something like a state of awe, but it's really just a powerlessness against you know, the state or the money or Las Vegas or whatever it is you're not doing in Las Vegas to humanize people. What are some of the litmus tests you use to decide if a piece of work is humanizing or dehumanizing?
 
Wow, that is a big question. For me, it's heavily dependent on the context, but some of the things I look at are the amount of vulnerability that the storyteller is showing. I mean, I love pieces of art that come from personal experience. And when somebody is trying to honestly work out something that is difficult to share with a mass audience, and they're trying to say, I want to, you know, through this piece of art through this narrative or this interactive experience, give you an opportunity to experience what I went through, and what my conclusion was so that that might help you ifyou go through something similar. To me that is extremely humanizing, because it involves empathizing and effectively an artist saying, you know, I want to connect with this audience and help them by explaining what I went through. I also ask whether this thing encourages me to interact with other human beings more? Or does it encourage me to interact with them less? Like I said, I think some of the best art walks a careful line between appealing to the conscious and the subconscious and balancing passion and reason. I like pieces of art that don't depict horrific exploitative technological dystopia and don't make us all want to give it all up and become completely agrarian, you know, they, I think we're in need of more and more art that depicts a world where those things are in imbalance. It's kind of stymied me sometimes and thinking about, you know, the art I want to produce. Because if that's the final conclusion that you want to get across, well, it's hard to do that with a big action sequence. But I do believe that the solution isn't to sit around and philosophize about it. It's really to prototype short things and see what works and what doesn't, right. prototype and iterate, prototype and iterate
 
And it doesn’t always have to be the indies.  Pixar is an environment to be creating things in you know, I mean, I would refer you to, to the director of InsideOut, Pete Docter makes some of the most humanizing stories between Monsters Inc, and Up and InsideOut.  He’s a very vulnerable, sensitive, amazing storyteller, and he wants to tell stories to help people feel the same thing.
 
So, yes, there are rare pockets where you can do both well, you know, make something commercial that’s also meaningful to people.
The difficulty of any storytelling is you can't just sit there at the end and go, Hey, in case you didn't notice, sadness is important. A lot of people refer to the end of the second act when they lose Bing Bong, Riley's imaginary friend, as being a devastating part of the movie to watch because they're really emotional about that loss, but I think that sadness almost physiologically primes people to receive the message of the third act. You’re not going to see somebody say, hey, you know, sadness is really important. Still, after you feel sad about something, you often have this peak afterwards, like you feel relieved physiologically after you cry, right? So, so you're now in the state watching the third act feeling a little better, because you've just gone through being sad, and you didn't get to choose to feel that way. I don't think that anyone sat down consciously and said we were going to do this, but it's my pet theory that one of the reasons the message of that film works is because you go through that feeling….and that's kind of what I'm getting at. You mentioned the dichotomy or, you know, the difference between just raw spectacle and awe, I think, you know, big action movies and stuff like that, you're always going to try to have some spectacle in the third act because that’s almost a tradition.  Or stuff like, you know, the first reveal of the dinosaur in the first act of Jurassic Park.  That might be more inspiration than spectacle, because it’s signalling that you're about to be in that environment where these characters are going to go through a major journey and they're going to change, so you relate to them a little bit more, which helps you invest in their story. That scene would be a lot less successful if they didn't show you the dinosaur or they just showed them coming back to the hotel saying, holy shit, did you see that?
 
The fact that the visuals make the audience actually experientially go through that as well is good filmmaking.  You're not communicating it just with the dialogue or lecturing, you have to give people that feeling. And by virtue of doing that, they understand because they go through it emotionally.
 
I think it’s enough that we have this capacity. When we go through emotions, or we fall in love, or eat. We like these things. And that's enough, just being human is enough. And that's why I'm not excited about emulating our consciousness with AI. I'm much more interested in how to use technology and narrative to make us feel emotions, because to me, and at least in 2017, that's the thing that feels the most human because we experience emotions and we don't have a whole lot of control over them. So, learning how to deal with them, using them to tell stories and help us make sense of human experience. That's the thing I care the most about.

Lance Weiler, founding member and director of the Columbia University Digital Storytelling Lab shares findings from the labs re-imagining of the work of Arthur Conan Doyle titled Sherlock Holmes & the Internet of Things, a prototype that had 1,200 collaborators from 60+ countries working across 70 events to create a massive connected crime scene.
 -------------------------------------------------------------------------------------------------------------
 
Main Takeaways:
Identifies 4 Immersive Participation Design Theme Principles
1) Trace: Provide a way for people to leave their mark/personalise the experience
2) Grant Agency: Shifting between individual and group tasks helps to promote, as well as reinforce a sense of individual agency.
3) Thematic Frame: Everything is so much easier when you're working with a known starting point e.g. a popular source IP
4) Serendipity Management: Let things evolve/Aim to show, not tell and leave gaps for the audience's own interpretations.

In addition he recommends dividing participants into groups of 5 - 6 people
​
The Digital Storytelling lab kicked off a prototype about a year ago called Sherlock Holmes and the Internet of Things and it's kind of like peanut butter and chocolate right? This idea of mixing Holmes and the IoT (the juxtaposition is consciously unexpected).
 
Because what was fascinating about Arthur Conan Doyle's work is that he was writing about emergent technologies that nobody knew about outside of certain small circles: non contaminated crime scenes, ballistics and different types of blood tests.  Law enforcement was reading that and saying, wow, these are really good ideas, we should adopt these. 
 
And so in a lot of ways, Arthur Conan Doyle was starting to influence fact. ... and I love this quote by Arthur C. Clarke.
 
“Any sufficiently advanced technology is indistinguishable from magic.”
 
And I think the Internet of Things is very much like that for most people. ..all these common objects within our lives connected to networks seems like magic. 

We also wanted to experiment with this idea of copyleft, so we made the whole experience open to whoever wants to be a part of it, all over the world. It's released under creative commons, share alike license, so people can commercialize whatever they create within it as long as they give back the code base and share their learnings.
 
Taking this original source material from Arthur Conan Doyle, and using it as this fertile playground to start to experiment with what we could do was a large, massive global storytelling experiment.

THE GLOBAL STORYTELLING EXPERIENCE
 
The experience starts when people go to the source material.  We ask them to find an object from one of Arthur Conan Doyle stories and adapt it using physical computing sensors to create this enchanted object that's intended to be something that they can use as a way to tell a story.
 
We didn't plan to use IOT at the start. Instead, we started for about six months through a series of meetups, and did everything in physical form, paper testing.
 
SHORT
Our analog prototype consisted of four key areas. It would take about 90 minutes to run it.

TEAMS OF 5 or 6
5 - 6 is optimum.
When` we had less than five, certain personalities would dominate and people would have a miserable time. And then when it was too large, it started becoming almost like a consensus vortex where nothing ever really happened.
 
TASKS
1. Create a crime scene.
We hand each team a roll of masking tape, and say okay, now go out and create a crime scene with your fellow teammates.

2. Create clues
When they came back we would lay down Brown. paper and tell them to empty their pockets in their bags and define three things that they thought were really interesting to tell a story with.
 
And then they would just trace those things. So if they had, you know, sunglasses or their phone, or whatever it was, they would trace the outline of it. And they’d do that individually.
 
3. Mix it up
Then we ask them to step back, pick three things that aren't yours, rip them out, cut them out, and now you have them. Go back to any of the bodies except the one that your team just created and place these objects with purpose in another crime scene, and be creative with them.

4. Set the scene
a) 1st individually

That famous phrase by Ernest Hemingway: "For sale. baby shoes never worn" a very evocative statement, that's this idea of generative, flash fiction.
 
We gave each team a packet of post it notes and some pens. You can go around, you can pick up that phone on the ground to hear the last voicemail message from a crying ex girlfriend. If you see glasses, you can leave teeth marks on broken lenses.  But you do it as an individual. The only rule is you can't make any changes to your own crime scene, your own body, your team's body.

b) Next collaboratively
 
And then they come back and we'd say, Okay, now you're a team again. Go back to the dead body crime scene that you originally created, but you're going to realize that there are all kinds of clues there now.

5. "Solve" the crime
Now we'd challenge them to be a collaborative Sherlock Holmes, by making up a story about what happened here. They could pick and chose what they wanted to respond to. But then they crafted these narratives about the scene that were incredibly dynamic and they did it very quickly.  
 

SCALING THIS EXPERIENCE
So we workshopped what to do through a series of meetups and then with a global community, which probably came close to 2000 collaborators from 60 different countries. Last year, there were 70, self organized events all over the world. There's been a ton already this year.

 
FOUR DESIGN PRINCIPLES emerged from this prototype. And remember we tested this thing all over the world.
 
1. TRACE
People really responded well, whenever they could see some element of themselves within the story, you know. And what's interesting is when we first started this, the very first prototypes, we thought we'll write the stories will populate the crime scenes.  We'll lay out the bodies and then people can come in and be Sherlock Holmes and Watson and they'll love it.

But what was actually more engaging was when they were creating everything themselves, and we were just letting it happen. So that one step of the trace was really important where they could see a contribution of themselves within the story.

2. GRANT AGENCY
Breaking the event up from a group task to an individual task and back and forth led to these really these potent moments that allowed people to feel like they actually had an impact over what was going on.

They didn't feel like they were just following somebody else who might be a dominant personality within a team. And that was that was interesting.
 
3. USE AN EVOCATIVE/ACCESSIBLE THEMATIC FRAME

Most people probably know Sherlock Holmes, even if they don't they know what a detective mystery is. And if they see a body on the ground, they probably know there was a crime. So there's a common language there that that really helped quite a bit. It made the experience accessible and quickly gain context, mutual understanding, engagement and back story.
 
4. SERENDIPITY MANAGEMENT
 
A classic screenwriting principle is that you're supposed to show, not simply tell, but interactive designers often get very concerned that people won't know how to interact with something in the proper way, so end up telling too much. 

But we left a lot of gaps in this experience and because we left those gaps, the imaginations of the participants filled them, leading to these really wonderful kind of moments where they would collide into each other in unexpected ways. And these collaborative, creative sparks would pop up out of it, which was really exciting.
 
When we were trying to work out how to manage the creative process of thousands of people. We ended up running an open MOOC usually known as a massive open online course, but we called it a massive online offline collaboration, open to whoever wanted to participate.
 
PARTIPATORY URBANISM
So it was interesting how something so simple was so effective in terms of people being able to organize themselves. 

We had this road rotary phone that had a Raspberry Pi with a text to speech engine.   So you could write to it from a distance and it would ring, it was also beacon aware so whenever  the phone got near something, it would recognize that it was there and ring, right. So then all of a sudden, somebody would start interacting with a character, you know, a killer and a cat mouse game or whatever.

We made that code open source because we want these IoT objects to be something that stimulates creativity within a group of people.
 
Ubiquitous computing is like calm technology, it's all about being in the peripheral, it's not necessarily about being buried in your device. And in fact, when we did have a device and we used it, it broke the flow, because everybody became very interested in the screen. So what we're trying to do with this is say, can we use objects that are enchanted in some way or smart objects that people can use as a way to tell a story. So it becomes about the human interaction and not necessarily solely about a screen interaction.
 
And the other thing that's interesting about the project is there's no win scenario. We remove that, you know, there's not one right way that a crime is solved or crime is created. So it becomes very much a collaborative form of story and play.
 
At TheDigital Storytelling Lab we’re now exploring participatory urbanism and the collaborative design of neighbourhoods through storytelling. For example we partnered the City of Los Angeles and did a storytelling innovation lab with them. The project was called My Sky Is Falling and it was shaped with foster youth in collaboration with the students.

70% of whom tend to end up addicted to substances, pregnant or homeless. It was heartbreaking. But together with the students they made an immersive science fiction story. We also teamed up with MIT and they gave us a bracelet that participants could wear to track their emotions as they went through the experience. And instead of relying on pre survey and post survey, and audio video transcription we came up with 26 different feedback loops within the experience itself. There's a really cool white paper on it if you're interested you can find it at myskyisfalling.com
 
When it pointed to and, just like Sherlock was this idea that stories can spark a really interesting innovation because that prototype was made for a couple hundred dollars. And then the UN invited us to run it there - and now it's being adapted in three different states as a framework to train, potential foster care parents and social workers to understand the emotional journey of a foster youth because when you go through the experience. .So this idea of inspiring greater understanding is really interesting and making use of technology to do so is something that we are very interested in doing
 
STORIES ARE NOW EVENTS
This is an amazing time to be a storyteller.  A story maybe isn't just by one person anymore, it’s a collaborative community event.  And increasingly now we’re seeing the rise of creative technologists who are using stories to create a common understanding.
 
EVENTS CAN CHANGE THE FUTURE
​I did a project with David Cronenberg called body mind change. And it was kind of about this idea of a personal recommendation engine that you just put into the back of your neck through synaptic entanglement. That's a word we made up it didn't exist. The engine would know everything you wanted needed and desire before you did. And so that story was all about the quantified self and about artificial intelligence and about this idea of emotional intelligence. But what is interesting about this the moment is that we have the opportunity to change the way that these stories are told.
 
For example, if an object is connected to a network it can send you media. You could build subscription models off of that. That object could change over time, it could change state, it could recognize other things around it and it could change the way it interacts. It could do simple things like change colours, it could change the lights when you walked into a room, whatever you want.

This idea of allowing those formerly known as the audience to be collaborators in the creative process allows them to create something or co create something that they care about, that can evolve over time. And new business models can come from it.
But this apparoch also breaks from permission based culture, so you don't have to spend years trying to convince somebody that you want to make something, you can just make it.

Weiler, Lance. 2018. Story Driven Innovation. U.S.: fitc events.

SUMMARY
 
TECHNOLOGY LESSONS:

• 1. Don’t try to build your own speech recognition system from scratch (very difficult)...so plan to use an internet connection and hitch onto another.
• 2. Other than this, the ability to build your own tools can help you iterate faster.  When you are constantly sending it out, recording it and plugging it in, every ounce of efficiency you can get out of your tools is multiplied 1000 fold.
• 3. Whatever content creation method you use, it should be efficient and fast because you're going to be constantly doing it. 

 
PLOT LESSONS:

• You need to make the activity of having a conversation fun, and a lot of that is going to come down to the flow.
• Players do need guidance.  People will shut down if they feel like they have too many different options.
• The ability to listen specifically is important. It's better to create tons and tons of alternate responses to one line of dialogue, as opposed to creating lots and lots of different generic subjects that the player probably won't explore anyway.

 
 
I'm going to talk about a game that I did a lot of the writing and narrative design for recently called Starship Commander, a virtual reality game that is driven entirely by human speech, which means that there is no controller. Instead you talk out loud to each character via the microphone.  We run that audio through our system and then the character you spoke to responds, so it's a game that you play by having an elongated conversation with the script.
 
DEFINING TECHNOLOGY
This speech recognition technology we're working with was disruptive.  It gave players a lot of agency.  They could say anything they wanted to at any point they wanted to.  There's no point in which the player can't talk. That both opened up a lot of cool doorways of things that only we could do. But it also opened up a couple of Pandora's boxes of real design and production problems that we knew we had to approach carefully.
 
So almost every decision I made about the narrative design was trying to work with the technology and figuring out how we could highlight the parts we wanted to and either avoid or at least contain potential problems that were coming up.
 
THE DIALOGUE STORY IS THE GAMEPLAY
We built Starship Commander using the Unreal engine because it had two features that we really wanted. First was the blueprint system. If you guys aren't familiar with blue, It is kind of like having twine embedded into the engine, and I say kind of because it behaves very similarly. Similarly, it's about creating a flow of logic. Once you're used to nodes and branches and flow, you can learn blueprint very easily. The trick is that while twine artists draft really for story, logic, and story flow blueprint is really for game logic and gameplay flow. It's about what buttons the player is pressing and how that changes all the math and the numbers we're keeping track of and what that means for our animations and sound effects and enemy AI and all of that. So you build it in a similar way, but it's for a different type of logic. And usually those two logics are kept separate from each other.
 
In most games the combat or whatever it is you're doing, usually doesn't affect the story. So we'll pause the gameplay and nothing's going to attack you, and you can decide what dialogue options you want to make.
But in our case the only way that we had for the player to interact with the game was to speak to a character or their ship… So we needed one spot where we could build both of those in the same place and see the entire flow all at once. And blueprint was the answer to that for us.
 
PLATFORM CHOICES
The other thing that Unreal did that we really wanted is it allowed us to build our own tools.  We actually initially attempted to build our own voice and speech recognition system. before eventually deciding, let's just use something someone else has already built, that's cleaner and quicker.  But in that case, Unreal allowed us to build in those outside sources to the code to the engine and a way that was effortless.
 
As soon as the Oculus microphone picked up what people were saying, we would send it out to Microsoft and Facebook's wit.ai program, we use those two services because they had the least amount of internet latency. And then it would send back text.
 
NATURAL DIALOGUE AND MANAGEABLE INTENTS
 
We really wanted the dialogue to feel natural, which meant there were  Hundreds of thousands of potential things that anybody could say at any point in time, so we needed to funnel those down to a limited number of things we could actually script.
 
So our programmers built this tool kit and this sort of flow of logic that we called the intent system. It works this way. any individual idea or concept of something that the player could say, was labeled an intent. So up there, you're looking at a question, Where are we going? As an intent? It could also be, who are you? It could be a comment, like, I think that's a bad idea. But any individual thing that somebody said and that we wanted to have a unique response to was an intent. Inside each intent. There was a list of utterances and utterances were essentially all of the different ways somebody could phrase that intent. So if we were going to use Who are you? as an example, some of the utterances inside that might be Tell me about yourself or What's your story? Or Tell me your story? Those are all different ways of phrasing that same idea. So ultimately, utterances became all of the different ways a player could activate this one intent.
 
DEFINING CONTEXTS
And the last thing we needed to do was take those and phrase them in the context of a specific conversation. Because what we're saying can mean very different things depending on the context, like whether or not a provocative comment is a joke, or an insult. So, somebody would speak into the microphone. We would send it out to Facebook and Microsoft get back text, the computer would then figure out what model am I in, which is something I would determine for it in blueprints, then it would check all of the intense inside that model, and all of the utterances inside all of those intents and that would run an accuracy test and the utterance that was the most similar to the text that Microsoft sent us back determines which intent would activate. And then we had all of that system hooked up to a custom node here in blueprint that depending on what the player said, would take it off on different branches of logic and that's how our game would react to the players speech.
 
FULL MOTION VIDEO
We created our content using full motion video, partially because it was really quick and cheap and easy to do fast
 
BUILDING STORY TO FIT THE TECH
Once the tech was in place I was hired to build a plot to match.  There were a lot of really cool, exciting things that we could do with our system and we wanted to make sure we had a story that capitalized on it. And there were also a lot of pitfalls that we thought we could use the story to kind of help mitigate. So I ended up coming up with a plot that was not necessarily about how excited I was as a writer to tell the story, but that I thought would fit the technology. And I was aiming to do it by thinking about these four things, which I will go through one at a time.

IF YOU”RE USING BRANCHING DIALOGUE KEEP YOUR EXPERIENCE SHORT
If you have a story that branches heavily a section that takes the average player roughly 15 to 25 seconds to get through demands a lot of writing and recording and programming. So we knew we weren't going to be able to keep this up for a long experience. Instead we're aiming for a short 90 minute to two hour experience.

YOUR BUSINESS MODEL CAN BE A CREATIVE PROMPT.
We wanted players to be comfortable dropping a little bit of money on out game, even though it was really short. SO we made it re-playable because we already had all of this alternate dialogue anyway, just by virtue of the technology.  So, we did things like enabling players to find branches just by saying something specific not because they were presented with a choice. This was simultaneously a design and business decision.

TRACK AND PLAY TEST REAL CONVERSATIONS AND USE CONTEXTUAL CONSTRAINTS TO HELP MANAGE THOSE CONVERSATIONS:
During every play testing session we recorded all conversations and we would be uncovering a lot of utterances that I hadn’t thought of – and that worked, but it was still a time intensive process. So I also needed to find ways to contain the number of intents that we would be plausibly expected to have. So I created specific contexts, around scenes.  Essentially, I decided to write a thriller, where every single scene has this imminent point of conflict and danger with a ticking clock element because in that situation, it does not occur to players to ask, “What's your mom's favourite colour?” This also gave us an easy out anytime someone made a comment or had a question that we hadn't thought of, so a very logical character response would be 'What does that matter right now? Can you please focus on the bomb?'
 
BTW we would play test in New York and because of the way people in New York naturally spoke, a scene would flow well. And then we would take it out to the west coast, and people would have different ways of putting the same idea together and it would get stumped. So actually, a big part of what we discovered with the play-testing process was to take it around to a lot of different local areas. 
 
The biggest problem that we had was ambient noise in the room, disrupting what people said.
There was also delay in the dialogue, we couldn't avoid that but we disguised it a little. Because you are in your ship, and the entire show we disguised it as static. Anytime we needed to buy a second, the screen would fizzle and then they heard you and talked to you. 

USE SURPRISE BRANCHES.  

When your player can say anything at any time, at some point, some player is going to say something that disrupts the plot.  Now on one hand that was a problem, but on the other hand, it can be an opportunity. Because if a player is clever enough to be able to disrupt what's supposed to happen with genuinely good story logic, we wanted to be able to reward them for that by having a concrete change in the story result. But that also put us in a weird situation, where all of a sudden, we were at the risk of players creating endless new branches, so I wanted a plot structure where we could add branches as we needed without having to build a whole lot of new content.
 
We also needed to account for the knowledge gap between the player and the avatar character. In our situation, the player was writing their own lines, without knowing any of their avatar’s backstory. And we needed to make that plausible or help the player figure out how to be this person.

So I created a MacGuffin. This is the anomaly…

At the beginning of the story humanity controls the MacGuffin, but they don't know what it is, or does. And in the opening scene, players role play a scientist trying to figure out what it is. They fly a one person ship into the anomaly in the scene, merging with it, potentially explaining any strange memory loss and also potentially explaining why the player know things about the future because the characters shouldn't. but maybe they've experienced alternate timelines.
 
In the middle of this experiment, it is disrupted, everybody is attacked by a race of war like aliens and the anomaly disappears, it blinks out of existence.  It goes somewhere else within the Milky Way galaxy. They rest of the story is about you the one person with this mysterious connection to the anomaly and your military escort Sergeant Sara Pearson, exploring the Milky Way trying to figure out what the anomaly is, where it's gone, and what the human race want to do with that when they find it.
 
This structure gave us a lot of freedom to improvise later on. ... A mystery is made up of clues. And those clues don't necessarily make sense individually, but when you put them together, they coagulate into the bigger picture of what is happening and why it matters. As long as the clues are structured in a way that's clever, you don't necessarily need to have all of them either. When you start, you start out in the prologue scene every single time you have an experiment with the anomaly. And then depending on the choices you make in that scene, we send you to one of four different tier one scenes. In any individual playthrough you will only ever play any one of these missions and these missions are all isolated from each other. The plot of each mission you go on is dependent on where you are in the Milky Way galaxy, what is happening in this part of the galaxy, and they don't directly relate to one in each other.
 
So once you get to the rescue mission, that mission will proceed as usual, regardless of what missions you've happened to play before it. But the reward for each one of those missions is a clue in the greater story of what's happening with the anomaly, and why.
 
There were different combinations of five clues that I had to make sense and I always had to get the player to the finale of where the anomaly was and why it's there. But once I knew there's only ever going to be one tier one clue. There will only ever be one tier two clue I could create a sense of progression. A good mystery isn't, Oh I have one clue. And it doesn't make sense, only when I have five can it be solved. Instead you want to slowly build and curate a picture of what's happening and then start to correct it. Now that I have two clues. I get the sense that these are connected in this way, but I don't know how yet. And now that I have a tier three clue Oh, that's different than what I thought that it's starting to get clear. And depending on the progression the player took through the game, they would get a slightly different version, a different perspective of what was happening and why you couldn't understand everything about the story until you played it through multiple times.
 
Let's walk through an example. Let's say the player has played one playthrough of the game, and in the first playthrough, they participate in the alien ship mission. And they get that clue about the anomaly. When they restart the game, technically, that fact doesn't exist in the game, but the player still knows it. The story is the same every time the player just gets different pieces of it. So potentially, the clue that the player learned in the alien ship, they might be able to use to change the way the traversal Memorial pans out. They might want to create a surprise branch there. And we could actually then reward them for that by making a branch that goes from the traversal Memorial sit down to Caltech runes, that would otherwise not be connected. So we can rearrange the flow.
 
Also at any point, if we're running out of time, or there's a tech problem that we don't know how to solve, we can just say, you know what, let's get rid of that scene. …and the reality of the story structure is still intact. You could make loads of cuts and as long as you don't make too many cuts from any one branch, the actual story is still playable. So we could sort of decide as we went and adjust ourselves from there.
 
Certain types of stories are more meandering than others and you do not want that. And when I think of sci fi and fantasy, I usually think of the setting as a huge star, but ultimately, we found that meandering was not a fun activity and very difficult to build. So in terms of certain genres of story, I would say, plot driven stories, like thrillers specifically, or maybe a very strong character driven story like a romance might work really well for this approach.
 
 
BUT PLAY TESTS SURPRISED US
 
As it turns out the players have a much better experience when you actually do have a lot of control, and you really can guide them through something specific.
 
One thing we had learned from actually our very initial brief tech demo scene, was that we really needed to have a way of allowing the conversation to continue regardless of what the player said if they happen to get stuck, because in a system that is this open ended, we really didn't have a plausible way of suggesting to the player what they should say to another human being right now without breaking the fourth wall in a way we didn't want to do.
 
But when it is that open ended, and when you are waiting for the player to say something specific, like I'm ready or Let's go, ultimately, it's not going to occur to a lot of people to do that.
 
There's a lot of people that are going to get stuck not know what to say and loop around and endless spiral. So we tried to have a way of progressing automatically. At some point if enough time had passed we would have the character, “You know what, the anomalies out there, we got to keep going and we got to move”. Down here, that's a different face of the conversation with different contexts and different reactions to what you might say. We did that in a very linear way.
 
But it is very, very difficult to write compelling dialogue when you only know what one half of the conversation is going to be.
 
I didn't know what my character were reacting to. I didn't really know what the player just said, I knew the idea. I knew the intent. But I didn't know the words. I didn't know the tone. I didn't know the utterance. And so all of the dialogue that I was putting out was very generic. It was very vanilla. I wasn't in love with these people. It wasn't good writing.
 
Eventually I realised that this was that this was a gameplay problem, and not a writing problem. So it wasn't that the writing was bad. It's that the flow of the activity was not fun.  I took the time to do some extra drafts and try to find voice. But what actually I should have been doing what would have been more productive would have been treating my initial draft in twine, not like a draft of a script, but like a paper prototype for the gameplay to try and figure out if the actual activity of this specific conversation and the way it moves is fun.
 
When we did start play testing the scenes, we discovered that players are actually very bad at coming up with things to say. Most of the things that they said were incredibly short, very simple. A couple of words of one word answers.. 90% of players just did whatever they were told, without getting into it. Only about 10% of players actually explored.
​
Unlimited freedom shuts down our ability to actually make a decision.
 
So in open ended sections in the script, we would say, do you have any questions? Do you want to know about me? Do you want to know about your ship? Do you want to know about this planet? The aliens? What do you want to know about ask? People go? I don't know. I don't have any questions. And we keep moving.
 
But the concept of asking anything turned out to be an inherently unfun activity. The more intents we have, the more utterances we have to have for each one. The more directions we go, the more general we have to be.  But as it turns out, that the freedom wasn't what players wanted, they wanted to be listened to, very, very specifically.
 
 Instead, the consistently successful part of our play test was a very brief moment early in the briefing scene, where Sergeant Pearson is filling out some of your information and she would ask How old are you? And we had written out a response of every age from like five to 95.  Nobody came in with an answer. We weren't prepared.  But what worked about that moment is because the constraints of the scene were very tight, so we could build out 90 different personalised answers to this one question.
And in that moment, players felt listened to.  And more importantly, because we knew what was going to happen we had a funny little joke for every age. If you came in and said, I'm 43. We have a funny little joke about 43 year olds for you. Now that did a couple of things. This call and response dialog did a couple of things for us. But most importantly, it gave us a feedback loop.
 
The idea is that it's the physical concrete process by which people play a game, which usually means that they push a button. That action has an effect inside the world of the game, but the loop isn't complete until the player is informed about the effects of their action. Through feedback, the player presses a gob or presses a button and attacks the Goblin.  Once we have an animation where the Goblin staggers back and a little sound effect where it yells and a blood texture shows up on the floor that's when the player really understands what they did and are equipped to now take another action.
 
This is where controlling the context of each scene became very important. Your character could legitimately take the reins to focus in on this one thing that's very important that we have to do. But then because we're not doing anything else, we can create all of these different options and listen to the specific things that each player says, each utterance can become its own intent, and then we can respond to exactly what they said and the exact words that they used, in a way that's immediately engaging and rewarding. Once I did this, I immediately knew how to write again. I immediately had the ability to look at exactly what a player had said and bounce off of it in a way that uses their own words, and I could create dialogue that was engaging. And the writing became a reward for the players creativity. When the player said something specific, a little bit unique to them, we could respond to it in a way that was actually entertaining, we could respond in a way that had value in and of itself. And when we rewarded players for being creative, all of a sudden player started being creative. They started opening up they started talking a little bit more as if they were talking to an actual person.
 
 
So overall lessons:
- Pick technology that you can control pick technology, you can build your own tools,
- Pick a plot where you can tightly control the context of a scene.

Kuelz, David. 2019. Designing Games that Listen. U.S.: Narrascope 2019/YouTube.

 

.Julius Kuschke, Chief Product officer at RTC creates tools for game writers.  RTC makes Artists Draft, a software to create branching stories.  In this talk he reviews highlights from the ways that previous games have ensured what Julius sees as the four secrets of a good dialogue:
1) Pacing
2) Authenticity
3) Conflict
4) Agency

1. Pacing.

Make it sound good, make it sound like music. It needs rhythm.
 
Previously, in a hub dialogue system, you could choose all the options offered in turn,. And that's very unnatural. It's also very much like an interrogation, because only the player is driving the whole dialogue.  This turns NPCs into information wending machines, where the player feels obligated to punch every button, just to get each available tidbit of data. 
 
So, this is why I think waterfall structures are so popular today because they feel so much more natural. Players understand that each option not chosen is gone forever, they don't have a second chance. The conversation will move on.

 
Just as an example for that, let's take a look at Assassin's Creed Odyssey.
“Tell me mysterious. Did you learn anything worthwhile in your dealings with the world?”
“I'm done talking, I should tell you where you stand.”
“They said you'd be different. And blood is blood, I suppose.”
 So, even though it's clearly a waterfall structure, without reoccurring hops, the dialogue often feels slow and artificial. Whenever there's a choice, everything stands still and the game waits for your input.

The most simple solution for this problem is to have time decisions and Telltale Games in particular made that system very popular.
Timers are great to create tension, and they reduce that weird awkward silence in between choices. They can be very difficult for non-native speakers or just slow readers, however. So most games offer a short preview of texts that hopefully can be understood instantly – but that can create more problems again because unclear options making the player character behave in unwanted ways or options leading to the exact same line of dialogue are frustrating, so frustrating even that one of the most downloaded mods for Fallout 4 replaced these short preview texts with a complete line of dialogue.
 
So, just to keep in mind, if you shorten your choice texts to be able to do something like time decisions, clarity is always more important than reading speed. 

• Bioware’s Dragon Age use iTunes and symbols to make the options more clear.

 

• Another option we have is just to be very consistent in how we present our choices. e.g. Bioware positions dialogue options consistently. For example, agreeable options are always in the upper right corner. If we disagree with what the NPC said, they're, etc. 

 

• In Star Wars, The Old Republic response options were arranged differently. according to the players character class.  The smuggler class, for example, always had the most Han Solo option appear at the top of the list. so lazy players could just pick the first option to experience a very classic interpretation of that character. And that's great, I think, because at least I know what to expect when I choose that.

 

• But as with any pattern based system, it can be boring for the player and also feels more artificial. The difficulty is to create a consistent system with enough flexibility to make it feel not to systemic.

 

• Another game that uses consistency to speed up the decision making process is Alpha Protocol. Their stem system placed answers at certain positions based on their attitude, for example, professional or aggressive. But what makes the pacing in Alpha Protocol really special is that the system has no pauses.  So you pick your answers while the NPCs are still talking and that feels extremely natural and fluent. And much more recent example for that is Firewatch. And it's really a great example when when we talk about pacing, because they combined a lot of the things I just mentioned.  They have very short but precise options. They have a timer, and they have no pauses at all. 

 

• Another way of keeping players engaged even though there are pauses in the system is Heavy Rains’ moving thoughts system. By simply displaying options more dynamically, it urges the player to respond quickly, and it keeps up tension.  n.b. moving texts can be very hard to read for a lot of people, so if you're doing something like that, be careful, maybe at least offer an option to disable these animations.

 

•  If actions do not appear in predictable patterns, players are more inclined to stay alert at all times. 

 
 
2. AUTHENTICITY

If we don't think that the characters are believable, chances are very low that we are interested in what they're saying.

Oftentimes when we talk about games, it's more about the illusion of choice. It's not about real choice all the time. …It doesn't have to be real agency either as long as it is believable.

I don't think that we need a completely advanced AI. We just need to give NPC the illusion of agency by creating more believable characters.

One way to do that is to give them their own agenda. Basically, to give them a life of their own and great example for that are the NPC conversations in the camp in Red Dead Redemption 2.
“Come on, Jack.”
“I'm hungry Mama.”
“We're all hungry son. Just try reading later.”
“Okay,”
“Gotta get some to eat Arthur.”
“Sure”
 
So this scene would also happen without the players standing so close. But the last sentence is only triggered if player's are positioned close by and that really impacts.

The feeling that the NPCs are aware of my presence, they know that I'm there. So I as a player make a difference. But what is more important is that I'm not in the center of everything.  The NPCs they have their own worries, they have their own thoughts and feelings, and they just talk to each other.
 
In real life, many things happen at once. You can't be everywhere or experience everything that's just not possible. In The Last Express a game released back in 1997 by Jordan Magna, the creator of Prince of Persia things happen whether you're there or not.  The NPCs have their own schedule, they sleep, they get up, they go to the dining carriage, and they have conversations there. And if you don't go there, they still do that. It still happens, the story progresses anyway. And this really elevates NPCs from being marionettes only reacting to the player to believable acting personalities.
 
A simpler approach to make NPCs feel more alive is to just let them actively seek interaction with a player. And I would call that mixed initiative, with just a simple timer in the background paired with a few variable triggers, it can seem as if the characters have a will of their own because it was not predictable when they will do that.
 
So if you always ignore them, if you always decide against what they are proposing, they can also just leave you. They can tell you at some point, Okay, I'm done with you. If you don't listen to me, I’ll go my own way.
 
But to be honest, there was one thing that could lead to really awkward situations and that was because the characters didn't have a really good knowledge about the game world, or what has happened so far. …but a game that does that so much better is again Firewatch because they have a system where the characters know quite a lot about the game world. They know everything that has happened so far. And that really influences how they behave and what dialogue choices they have. In Firewatch dialogue lines are selected by a system that tries to find the line with the most matching requirements, so they don't have a traditional dialogue tree, they have a completely different system. And how that works is, I think best explained with an example. On day two of the game, the NPC Delilah would start a conversation with a player. But it makes a huge difference how much the player already told her about his wife Julia. If Delilah knows nothing about Julia, you get a pretty generic statement about relationships. But if the player had spoken about Julia and also told them that Delilah wasn’t feeling well  they heard a more fitting response. “What does she have?”  And the magic about the system is that players probably won't even notice its complexity. It's very invisible in a way. But still, it really makes me feel that all the choices I make matter, and they have consequences. And the NPC respond to me in a very natural way. 

3. CONFLICT
Without conflict, there is no drama.

• But how could we create a dialogue system that really enhances that feeling of conflict? There are very few examples I found that I think are interesting. One is the Council, which has some kind of resource management system built into the dialogue system. and it certainly makes you think, do I want to spend my five points now to get that piece of information? Or do I want to save to maybe convince someone later on. So it creates a conflict for the player.
• And in Blindfolds we try to do something kind of similar, so every line of dialogue basically costs time, and there's nothing I can do about it. Everything I say, will move time forward. So I have to be really careful to choose which character I want to talk to and when I want to switch a topic.
• Interrogation are game-like.  You can actually win or lose a conversation. And players probably think twice before choosing one of the options. And they are much more focused on what is being said because in the end, they have to judge between truth and lie.
• If we look at game mechanics to integrate into our dialogues, another great example is just having puzzles inside of the dialogues.
• The deeper and more emotional the relationship between two characters, the more we care about the conflicts arising between them. Probably one of the best examples for a game that uses relationships to create conflict is The Sims, and that's kind of sad, because the in character interactions in that game cannot be really called a dialogue. The romance systems in Bioware games are more concrete. And they usually have some kind of value to track a relationship between an NPC and player character. And just to have the player aware of that value, really strengthens the feeling that my choices in a conversation matter because saying the wrong thing might damage a relationship I care about. But this can also feel artificial and shallow e.g.  the approval system in Dragon Age Origins. Here, the conflict  is undermined by giving players the option to simply increase the approval rating by making gifts to the NPCs. So I could just install an NPC and never listen to them, but give them a kitchen and hey, we're friends now, right?
• In real life relationships, the challenge is to truly listen to people and to remember what someone cares about in order to say something that they appreciate. In Tokyo Mickey Memorial for example, neglected infrequently dated characters would eventually become angry and gossip to their friends. severely reducing love meters across the board. In the middle of the game, when the number of known love interests was high, these bombs became the primary concern,  forcing careful planning and strategic round robin dating. I would like to see more jealousy systems in games, because it creates conflict…but forcing the player as a solution to undertake this round robin dating strategy is a bit silly, but if that system can offer meaningful choices that could be really, really interesting.
• Of course, there are several other ways to integrate game mechanics into dialogue systems and even simple things such as a skill check in an RPG can enhance player engagement because it makes that choice somehow stick out, it makes it feel more valuable, more rewarding. BUT the greater the sense that a player can win a conversation, the more we engage the analytical calculating parts of the brain, which presents a barrier to empathy. If we want to tell a good story, we have to be careful what we're doing there. If it's too systemic, if it's too artificial, then players will not really care about what's going on anymore.
• So we have to find a good middle ground. Adding elements of conflict to a dialogue system can work, but only if it really fits the setting, like the interrogation example in a crime story,

 

4. AGENCY
Agency is the level of control the player has over the game world. It's the feeling that my choices matter. Usually NPCs have no agency. They respond in pre-programmed ways, and this makes them feel like marionettes, which destroys the illusion of interacting with another character.
 

• So I think the most prominent example to challenge this is probably hinting at consequences as done by Telltale. “Clementine will remember”, this phrase almost became iconic for story driven games.
• ​Hinting at consequences also shows that you need to deliver on the agency you're promising. And I think that was kind of the problem, at least for me in the later Telltale Games. And a lot of people in the press, as I think saw that equally. For example, Game Informer said at one time, I can't help but feel a bit deceived. The Walking Dead had me believing that I could save characters from death. After multiple playthroughs I realized characters fates are etched in stone. 
• Allowing the player to choose the topics they're interested in can enhance the players feeling of agency and that's why we try to combine dialogue systems. If you initiate a conversation with a character, first of all, you get a very traditional app based UI where you can just choose the topic you would like to talk about. And as soon as you choose that, it switches completely to a very natural waterfall system.
•  Agency is also about being in control. Again, Firewatch is a good example.  You can always move and look around even hear dialogue. It even allows you to interrupt an ongoing conversation by switching between topic, … So you wander around, you're talking to Delilah. And you come across that tree. And you could just say, Oh, I found some claw marks. And that really changes the topic of the dialogue.  Whenever a conversation turns to a new topic, a bookmark is set. If the dialogue is interrupted, now the system will try to resume the dialogue at a later point. And there is a specific entry point for each bookmark. This entry point comes with a special line of dialogue used to pick up a topic in a more natural way
• 
  

Truly engaging dialogue systems are still the exception. But as these examples show the design of a dialogue system has a significant impact on pacing, authenticity, conflict and agency and probably a whole lot of other good things as well.

Kuschke, Julius. 2019. Narrascope 2019 - Julius Kuschke - How Dialogue Systems Make or Break Player Engagement. U.S.: Narrascope 2019.