In this presentation Aki Jarvinen, Senior Experience Researcher, Ph.D, Digital Catapult, author of the report The Immersive Audience Journey summarises the report’s main findings and explains why he encourages creators to think about the immersive experiences beyond the story, "Framing immersive experiences under the banner of 'storytelling' in the age of ‘peak TV’ and omnipresent story content is not necessarily a working strategy. Emphasising the experiences seems to differentiate immersive. Immersive producers need to think how to blend production activities into marketing, and adopt holistic thinking around audience engagement. Community building and pre event launch activities can significantly increase awareness and facilitating post event activities contribute to loyalty".
 
Download the full report here
 
The original need for this research emerged from the question of audience. There was a recognition that we lacked insight regarding what is immersive audience, or what are immersive audiences.
 
What is immersive?
 
Whereas producers tend to associate the term with particular technologies, audiences tend to associate it with stepping into a responsive and alternative reality that has been artificially created for them.
 
Immersive technologies are typically good at creating these kind of experiences. So immersive is many things, and can be created using a variety of technologies. 
 
Our analysis highlights a common, high level customer journey map approach which has traditionally been employed in marketing, service and design thinking contexts, but we want to encourage more thinking about this throughout the production process. 

If immersive producers are able to leverage this more holistic approach, we think that it contributes to a more structured planning of the project which puts it in a better place to succeed. There's so many things grabbing our attention, so alternative distribution approaches are important. One definition from service design literature says that a human centered tool, like the journey map not only includes steps where a customer is interacting with the company, but also reveals all the key steps of the experience. So journey maps help us define gaps in customer experiences, and explore potential solutions.
 
The Unfolding Immersive Audience Journey in Phases:

Phase 0: Segmenting Immersive Audiences
 
What we are finding is that the immersive audience is a collection of subsets of audiences, ranging from festival goers who happen across an immersive production exhibited in that festival, whether it's online or location based. Immersive audiences include gamers who are into VR. They include casual AR mobile AR audiences, but also more niche cohorts that might be more profitable and engaged like secret cinema enthusiasts. immersive theater enthusiasts, even VR documentary enthusiasts. The key thing to understand is that the context defines the audience for any given production. So for instance, if it's a location based experience where is it delivered, exhibited, but also on which digital platforms, as the device might dictate and shape your audience, but also social dynamics. So is it something that enables friends and families to go together?
 
Already we are seeing some clever solutions for post Covid-19 engagement that facilitate for instance, a family or a household come in at the same time to a location based solution.
 
Phase 1: Reaching Audience Awareness

This is the phase where a potential audience member becomes aware that an immersive production exists, which requires promotion, marketing, and community building. And the key takeaway, I think, from this phase is that immersive productions are challenging to market by traditional means.
 
In the age of peak TV, where there is an overwhelming amount of streaming television storytelling format content, framing your experience, as the peak in immersive storytelling is not necessarily the most strategically sound choice.
 
Instead you need to emphasize the unique component of your immersive experience. And that tends to be immersion, whether it's the sense of presence that you feel through, let's say, VR, or the physical set, with projections and props. However, the paradox here is that creating awareness around this isn't very easy to do with traditional marketing means. It’s  hard to communicate what is special about an immersive experience via print, or video marketing.  You need to be creative.

My one takeaway would be that you should show audiences engaging with the experience in a unique way in the promotion video.  Don’t just show the visuals of the experience because that also factors to the next phase which is consideration.
 
Phase 2: Audience considerations
 
After becoming aware, your potential audience starts to weigh up whether to attend, or participate, engage, and pay for your experience.  Managing their expectations and almost literally telling them the degree of expected interaction is very important. There is also a duty of care, to show that you will be on-boarding them through this and taking care of them during it as well. This may require that you lower the threshold of attendance explicitly in certain cohorts, and this relates to technology acceptance, so how comfortable people are putting on a headset or engaging with a technology that they're not familiar with in their everyday sort of media, and technology repertoire.  Multiple things go into those considerations, including the genre and type of content and experience.  For example, to engage a broader audience in the thrill seeking aspect of that experience, you might engage your comic lead in your communication and that just might work for you that they can connect with that certain niche.
 
Putting your production out there takes time, money and resourcing.  You need to have people on hand to help participants go through that experience. This is one thing that I feel requires further development on the digital side of things, because people are just not comfortable trying to engage with new equipment like headsets on their own. So how could you facilitate that process? How do you make it as easy as possible to get on board. Also, can you draw attention away from the technology towards the experience itself?  That's another element that you want, might want to do.
 
Phase 3: Evaluating Audience Experience
 
The decision for an audience member to attend or not to attend might contribute directly to your bottom line and your opportunity to continue the work. Therefore, you need to be aware of the different dynamics that go into these decisions. For instance, the size of a market that you are targeting is one important factor. At Digital Catapult we're trying to get more actual data about Immersive Audiences size, demographics and limits to help guide startups.
 
Important factors include price and location. A location can be a platform too. So if an AR mobile experience is only available to Android or iOS, then that shapes your audience. A certain kind of headset shapes your audience and price factors into that too, as well as more qualitative characteristics such as technology acceptance in a certain demographic for instance, or the fear of missing out regarding this experience, which might be something that you want to deliberately build. But there are other considerations, like the expected level of interaction.  The whole point of an immersive theatre piece might be to take it out from the theatre to the streets and make it a bit more chaotic if you will. But then some of the audience members complained that they didn't always see all the actors and therefore didn't always know where to direct their attention. So if your target participants are more used to traditional theatre where you have a good view to the stage and all the techniques that have to do with lighting and drawing audience's attention to this on the stage, immersive disruption can violate those expectations. If it does that audiences will leave unsatisfied and won’t recommend the experience to others.

We have a tool available to help you test your interfaces.
 
If the intention is create an experience which gives participants the option to make choices, ideally they will then come out of the experience feeling that it was unique and personalised. But people bring their own personal context, habits and expectation to every experience that will shape how fully they interact with the piece.  So if there's a lot of self-interaction, there might be people who just are not familiar with those conventions of interaction, and they might come out of the experience by feeling that they didn't get as much as somebody else. That might feel like a design failure.
 
User testing is a skill set and a set of know how that small studios don’t necessarily possess, so it tends to get de prioritised in production. In the interviews that we conducted, nobody discounted the need and potential usefulness of user research and testing, but because they weren't well versed in the methodologies, production itself tends to take over, with ad hoc user tests conducted very close to the end. And that also means that their methodology might not be entirely sound, which then might lead to skewed results. For example, the testing cohort might not be representative of the target audience. So it's just another example where time and resource needs to be allocated early on to this and there needs to be somebody leading on it as well.
 
In larger organisations like museums, there might already be a culture of user research. So, they have experts in house who gather user insights from past projects, to inform a new project and then advocate those insights along the journey.
 
And they have a very good sort of set structure of where at which phases is production tested, and how that feeds into the process. So we hope to assist people find these relatively accessible methodologies and tools.
 
But it is very challenging to be able to assess if there's any longer term impact of an experience, particularly in terms of behaviour change over time.
 
Phase 4: Satisfaction and Loyalty

This is another moment of truth, satisfaction and loyalty. It’s very important especially if you hope people will re-engage or recommend the experience to others. So, if you want to create so called organic growth word of mouth around your production, then you should think about how to facilitate that.
 
This is where community management comes into the picture. To create a community around your product, where people share stories about using the product and experiencing it. And they've tried to leverage this by engaging the community and listening to them, and so on so forth. But again, somebody has to do, perhaps even a small team, and allocating that responsibility can be challenging, but down the line, it does pay off in most cases.
 
Another thing to consider is merchandising. This approach has been tried and tested in the amusement park industry. Merchandise can reinforce a participatory experience.  By buying a T shirt, an object, a piece of memorabilia that they can take home, audience members might also contribute to that word of mouth.
 
And the other benefit from thinking about and planning for merchandising is that it creates another complimentary revenue stream for your ticket or download revenues. Again, this is something that needs to be planned in order to be effectively executed.
 
TEMPLATES

This audience journey is an approach that can be applied in practice, and can inform your thinking about your production in a holistic manner. So therefore, we have included two templates to the report, and also they will be available to download separately as printouts.
 
Template 1:
The audience journey template which illustrates the various touch points to consider from the point of view of the responsibilities and stakeholders involved in different phases of the journey.
 
Template 2:
In a 2nd template the user journey is also mapped from an emotional perspective, to help you think in a more deliberate way about how to take your audience from being indifferent, or casually interested, let's say, to being excited and waiting to see your production and then also possibly eager to re-engage or hear about what you're going to do next.

Download the full report here

Jarvinen, Aki. 2020. The Immersive Audience Journey: Insights and perspectives on immersive art, culture, & entertainment. edited by Digital Catapult. U.K.: Digital Catapult/YouTube.

.Julius Kuschke, Chief Product officer at RTC creates tools for game writers.  RTC makes Artists Draft, a software to create branching stories.  In this talk he reviews highlights from the ways that previous games have ensured what Julius sees as the four secrets of a good dialogue:
1) Pacing
2) Authenticity
3) Conflict
4) Agency

1. Pacing.

Make it sound good, make it sound like music. It needs rhythm.
 
Previously, in a hub dialogue system, you could choose all the options offered in turn,. And that's very unnatural. It's also very much like an interrogation, because only the player is driving the whole dialogue.  This turns NPCs into information wending machines, where the player feels obligated to punch every button, just to get each available tidbit of data. 
 
So, this is why I think waterfall structures are so popular today because they feel so much more natural. Players understand that each option not chosen is gone forever, they don't have a second chance. The conversation will move on.

 
Just as an example for that, let's take a look at Assassin's Creed Odyssey.
“Tell me mysterious. Did you learn anything worthwhile in your dealings with the world?”
“I'm done talking, I should tell you where you stand.”
“They said you'd be different. And blood is blood, I suppose.”
 So, even though it's clearly a waterfall structure, without reoccurring hops, the dialogue often feels slow and artificial. Whenever there's a choice, everything stands still and the game waits for your input.

The most simple solution for this problem is to have time decisions and Telltale Games in particular made that system very popular.
Timers are great to create tension, and they reduce that weird awkward silence in between choices. They can be very difficult for non-native speakers or just slow readers, however. So most games offer a short preview of texts that hopefully can be understood instantly – but that can create more problems again because unclear options making the player character behave in unwanted ways or options leading to the exact same line of dialogue are frustrating, so frustrating even that one of the most downloaded mods for Fallout 4 replaced these short preview texts with a complete line of dialogue.
 
So, just to keep in mind, if you shorten your choice texts to be able to do something like time decisions, clarity is always more important than reading speed. 

• Bioware’s Dragon Age use iTunes and symbols to make the options more clear.

 

• Another option we have is just to be very consistent in how we present our choices. e.g. Bioware positions dialogue options consistently. For example, agreeable options are always in the upper right corner. If we disagree with what the NPC said, they're, etc. 

 

• In Star Wars, The Old Republic response options were arranged differently. according to the players character class.  The smuggler class, for example, always had the most Han Solo option appear at the top of the list. so lazy players could just pick the first option to experience a very classic interpretation of that character. And that's great, I think, because at least I know what to expect when I choose that.

 

• But as with any pattern based system, it can be boring for the player and also feels more artificial. The difficulty is to create a consistent system with enough flexibility to make it feel not to systemic.

 

• Another game that uses consistency to speed up the decision making process is Alpha Protocol. Their stem system placed answers at certain positions based on their attitude, for example, professional or aggressive. But what makes the pacing in Alpha Protocol really special is that the system has no pauses.  So you pick your answers while the NPCs are still talking and that feels extremely natural and fluent. And much more recent example for that is Firewatch. And it's really a great example when when we talk about pacing, because they combined a lot of the things I just mentioned.  They have very short but precise options. They have a timer, and they have no pauses at all. 

 

• Another way of keeping players engaged even though there are pauses in the system is Heavy Rains’ moving thoughts system. By simply displaying options more dynamically, it urges the player to respond quickly, and it keeps up tension.  n.b. moving texts can be very hard to read for a lot of people, so if you're doing something like that, be careful, maybe at least offer an option to disable these animations.

 

•  If actions do not appear in predictable patterns, players are more inclined to stay alert at all times. 

 
 
2. AUTHENTICITY

If we don't think that the characters are believable, chances are very low that we are interested in what they're saying.

Oftentimes when we talk about games, it's more about the illusion of choice. It's not about real choice all the time. …It doesn't have to be real agency either as long as it is believable.

I don't think that we need a completely advanced AI. We just need to give NPC the illusion of agency by creating more believable characters.

One way to do that is to give them their own agenda. Basically, to give them a life of their own and great example for that are the NPC conversations in the camp in Red Dead Redemption 2.
“Come on, Jack.”
“I'm hungry Mama.”
“We're all hungry son. Just try reading later.”
“Okay,”
“Gotta get some to eat Arthur.”
“Sure”
 
So this scene would also happen without the players standing so close. But the last sentence is only triggered if player's are positioned close by and that really impacts.

The feeling that the NPCs are aware of my presence, they know that I'm there. So I as a player make a difference. But what is more important is that I'm not in the center of everything.  The NPCs they have their own worries, they have their own thoughts and feelings, and they just talk to each other.
 
In real life, many things happen at once. You can't be everywhere or experience everything that's just not possible. In The Last Express a game released back in 1997 by Jordan Magna, the creator of Prince of Persia things happen whether you're there or not.  The NPCs have their own schedule, they sleep, they get up, they go to the dining carriage, and they have conversations there. And if you don't go there, they still do that. It still happens, the story progresses anyway. And this really elevates NPCs from being marionettes only reacting to the player to believable acting personalities.
 
A simpler approach to make NPCs feel more alive is to just let them actively seek interaction with a player. And I would call that mixed initiative, with just a simple timer in the background paired with a few variable triggers, it can seem as if the characters have a will of their own because it was not predictable when they will do that.
 
So if you always ignore them, if you always decide against what they are proposing, they can also just leave you. They can tell you at some point, Okay, I'm done with you. If you don't listen to me, I’ll go my own way.
 
But to be honest, there was one thing that could lead to really awkward situations and that was because the characters didn't have a really good knowledge about the game world, or what has happened so far. …but a game that does that so much better is again Firewatch because they have a system where the characters know quite a lot about the game world. They know everything that has happened so far. And that really influences how they behave and what dialogue choices they have. In Firewatch dialogue lines are selected by a system that tries to find the line with the most matching requirements, so they don't have a traditional dialogue tree, they have a completely different system. And how that works is, I think best explained with an example. On day two of the game, the NPC Delilah would start a conversation with a player. But it makes a huge difference how much the player already told her about his wife Julia. If Delilah knows nothing about Julia, you get a pretty generic statement about relationships. But if the player had spoken about Julia and also told them that Delilah wasn’t feeling well  they heard a more fitting response. “What does she have?”  And the magic about the system is that players probably won't even notice its complexity. It's very invisible in a way. But still, it really makes me feel that all the choices I make matter, and they have consequences. And the NPC respond to me in a very natural way. 

3. CONFLICT
Without conflict, there is no drama.

• But how could we create a dialogue system that really enhances that feeling of conflict? There are very few examples I found that I think are interesting. One is the Council, which has some kind of resource management system built into the dialogue system. and it certainly makes you think, do I want to spend my five points now to get that piece of information? Or do I want to save to maybe convince someone later on. So it creates a conflict for the player.
• And in Blindfolds we try to do something kind of similar, so every line of dialogue basically costs time, and there's nothing I can do about it. Everything I say, will move time forward. So I have to be really careful to choose which character I want to talk to and when I want to switch a topic.
• Interrogation are game-like.  You can actually win or lose a conversation. And players probably think twice before choosing one of the options. And they are much more focused on what is being said because in the end, they have to judge between truth and lie.
• If we look at game mechanics to integrate into our dialogues, another great example is just having puzzles inside of the dialogues.
• The deeper and more emotional the relationship between two characters, the more we care about the conflicts arising between them. Probably one of the best examples for a game that uses relationships to create conflict is The Sims, and that's kind of sad, because the in character interactions in that game cannot be really called a dialogue. The romance systems in Bioware games are more concrete. And they usually have some kind of value to track a relationship between an NPC and player character. And just to have the player aware of that value, really strengthens the feeling that my choices in a conversation matter because saying the wrong thing might damage a relationship I care about. But this can also feel artificial and shallow e.g.  the approval system in Dragon Age Origins. Here, the conflict  is undermined by giving players the option to simply increase the approval rating by making gifts to the NPCs. So I could just install an NPC and never listen to them, but give them a kitchen and hey, we're friends now, right?
• In real life relationships, the challenge is to truly listen to people and to remember what someone cares about in order to say something that they appreciate. In Tokyo Mickey Memorial for example, neglected infrequently dated characters would eventually become angry and gossip to their friends. severely reducing love meters across the board. In the middle of the game, when the number of known love interests was high, these bombs became the primary concern,  forcing careful planning and strategic round robin dating. I would like to see more jealousy systems in games, because it creates conflict…but forcing the player as a solution to undertake this round robin dating strategy is a bit silly, but if that system can offer meaningful choices that could be really, really interesting.
• Of course, there are several other ways to integrate game mechanics into dialogue systems and even simple things such as a skill check in an RPG can enhance player engagement because it makes that choice somehow stick out, it makes it feel more valuable, more rewarding. BUT the greater the sense that a player can win a conversation, the more we engage the analytical calculating parts of the brain, which presents a barrier to empathy. If we want to tell a good story, we have to be careful what we're doing there. If it's too systemic, if it's too artificial, then players will not really care about what's going on anymore.
• So we have to find a good middle ground. Adding elements of conflict to a dialogue system can work, but only if it really fits the setting, like the interrogation example in a crime story,

 

4. AGENCY
Agency is the level of control the player has over the game world. It's the feeling that my choices matter. Usually NPCs have no agency. They respond in pre-programmed ways, and this makes them feel like marionettes, which destroys the illusion of interacting with another character.
 

• So I think the most prominent example to challenge this is probably hinting at consequences as done by Telltale. “Clementine will remember”, this phrase almost became iconic for story driven games.
• ​Hinting at consequences also shows that you need to deliver on the agency you're promising. And I think that was kind of the problem, at least for me in the later Telltale Games. And a lot of people in the press, as I think saw that equally. For example, Game Informer said at one time, I can't help but feel a bit deceived. The Walking Dead had me believing that I could save characters from death. After multiple playthroughs I realized characters fates are etched in stone. 
• Allowing the player to choose the topics they're interested in can enhance the players feeling of agency and that's why we try to combine dialogue systems. If you initiate a conversation with a character, first of all, you get a very traditional app based UI where you can just choose the topic you would like to talk about. And as soon as you choose that, it switches completely to a very natural waterfall system.
•  Agency is also about being in control. Again, Firewatch is a good example.  You can always move and look around even hear dialogue. It even allows you to interrupt an ongoing conversation by switching between topic, … So you wander around, you're talking to Delilah. And you come across that tree. And you could just say, Oh, I found some claw marks. And that really changes the topic of the dialogue.  Whenever a conversation turns to a new topic, a bookmark is set. If the dialogue is interrupted, now the system will try to resume the dialogue at a later point. And there is a specific entry point for each bookmark. This entry point comes with a special line of dialogue used to pick up a topic in a more natural way
• 
  

Truly engaging dialogue systems are still the exception. But as these examples show the design of a dialogue system has a significant impact on pacing, authenticity, conflict and agency and probably a whole lot of other good things as well.

Kuschke, Julius. 2019. Narrascope 2019 - Julius Kuschke - How Dialogue Systems Make or Break Player Engagement. U.S.: Narrascope 2019.