“If interactive narrative is ever going to approach the emotional power of movies and drama, it will be as a three-dimensional world that opens itself to the body of the spectator but remains the top-down design of a largely fixed narrative script”

• Marie-Laure Ryan, Avatars of Story

After a review of narrative theory in AR environments - considering the difference between a story (sequence of events) and narrative (the way those events are presented), as well as the likely blurring of fiction (the story) and reality (the mobile environment) in AR, and the broad treatment of the term text in this article to include all forms of communications including hardware and buildings, versus their definition of a medium, which extends beyond the hardware to include "a set of conventions, practices and design approaches that authors make use of to create a familiar and meaningful experience for the user" - the authors survey features of early AR fiction experiences, distinguishing between
1) Situated (local, quick) augmented experiences,
2) Location based narratives (using a few, sparsely located portals in a wider area)
3) World-level AR experiences sited across an entire neighbourhood, or city, or globally that tend to run for longer periods.

1) Situated augmented experiences
- AR/Fac¸ade: A portable version of a previous, breakthrough, interpersonal audio-visual dialogue interaction with a warring couple, AR/Fac¸ade  allowed users to inhabit the same physical/virtual space as the drama’s main characters, Tripp and Grace, while wearing an AR headset and a portable computer.  In order to maintain the illusion the virtual characters could not respond with commands like 'invalid input'.
- Three Angry Men (TAM): An interactive experience that allows users to explore the scene from different physical points of view (which trigger character behaviour changes, rather than plot variations).
- "inbox": An AR installation that allowed users to enter a shipping container and trigger short stories about the shipping industry by engaging AR markers in the space.
2) Location based narratives
- M-Views: Users were encouraged to walk around the MIT campus and encounter a distributed and modular, variable order cinematic narrative, embedded about the campus.
- Murder on Beacon Hill: A murder mystery tour of downtown Boston
- GEIST: A similar approach is used to tell the history of 17th century Heidelburg
- Hopstory: Added the option for users to act in their own timeline and move to different locations throughout the building, adding a layer of time and evolution
- The Westwood Experience: The creators used real live actors and physical setups to increase the immersion, alongside computer vi- sion methods for landmark locations.  At certain points the actors broke out of character to explain technical aspects of this Nokia research project.
- The Oakland Experience: A mostly linear audio only tour of a cemetary, with branching mini-stories around single graves.
- [8]: Combined a positional tracking system with a directional microphone to create an unfolding narrative in the changing landscape surrounds of a motor car drive-through (whilst there were some branches, mostly the drive had to be linear, without room for variation)
3) World level augmented narratives
- Alternate Reality Games e.g. 'Conspiracy for Good': Involved both online and offline presence with live actors.  The master narrative was fixed, but players had the option to change the advent, or order of the next story 'beat' which supported the impression that they were changing the story.

These examples often emphasised exploration of real world spaces in order to support both a sense of interactivity and narrative progression.

In order to help guide users through this exploration AR markers were used, as well as non-marker signposts e.g buildings and objects that might be more compatible with the fictive immersion.




Shilkrot, R., Montfort, N. and Maes, P., 2014, September. nARratives of augmented worlds. In 2014 IEEE International Symposium on Mixed and Augmented Reality-Media, Art, Social Science, Humanities and Design (ISMAR-MASH'D) (pp. 35-42). IEEE.

This study of the Interactive Alice installation (an experience crafted across 6 stages, each inspired by the Alice in Wonderland novel), suggests that immersion (the sense of being in the story world) is influenced more by the duration/intensity, rather than type of interaction. 
 
LITERATURE REVIEW
The paper starts with a review of recent research, which suggests that 1) An increased sense of presence, does not necessarily increase engagement (in terms of the user’s active interest in and interaction with the application), 2) That a user’s experience is changed by what they do during an event, shaped in turn by the type of interactions available to them.  3) At the same time, responsive technologies like sensors can heighten the sense of interaction and engagement, even in a situation with limited choices in terms of the way that the story unfolds.  4) Also, the sequencing of unfolding events is important, as it creates a dramatic and emotional rhythm/effect.
 
STUDY METHODS
This study involved a survey of the user experience of 18 – 33 year olds in the 3rd stage of the Interactive Alice installation entitled Eat Me, Drink Me which is situated in a white cube where users land after ‘falling’ through the rabbit hole.   The room appears to be locked with different sized doors (projected onto the cube, which is a sort of VR cave with reactive audio and video effects).  There are also sensors embedded in the floor, and movement sensitive physical props (such as a bottle of liquid that says ‘drink me’) are also on display.  Some rooms were more responsive and more interactive than others.  The simulated responses to the participant’s efforts to open a door also varied according what each particiant had previously eaten, or drunk.  Responses included sentences like “Sorry! You’re way too big.”, or  “No use. Haha! I forgot to tell you. I’m locked.” 
 
When the participant was the right height and had already found the key the virtual door opened, revealing the exit.
 
Through a combination of observation and survey the researchers measured for the participant’s sense of presence, agency (the ability to take meaningful action), satisfaction and also observed different sets of user behaviours in each different context.
 
RESULTS
Results showed that whilst short interactions in and of themselves created a sense of purpose, and satisfaction, nevertheless the type of interaction users experienced did not seem to influence their sense of satisfaction, or presence in the interaction. 
 
The researchers noticed clear differences in the users’ behaviours. ‘The participants that were immersed in a non-responsive environment (NIE) were more active and tried out more interaction possibilities (touch, walk, look around). The participants who experienced minimum stimuli in a non-interactive environment (NIMS) were less active, more observant. The participants in the non-interactive environment (NIE) seemed more likely to become confused and frustrated, while the participants in the interactive environment (IE) seemed satisfied every time they found something to do and experienced a feedback result.
 
 
Please cite this article as: M. Nakevska, A. van der Sanden, M. Funk, J. Hu, M. Rauterberg, Interactive Storytelling in a Mixed Reality Environment: The Effects of Interactivity on User Experiences, Entertainment Computing (2017),
doi: http://dx.doi.org/10.1016/j.entcom.2017.01.001

This study evaluates 4 different interface options for augmented reality applications including keyboard/mouse, visual markers, a joy pad and multi-touch pad.   Test tasks including ‘selection’ of a virtual object, ‘translation’ of the selected object on a 2D desktop space, ‘rotation’ of the selected object, and whole ‘scaling’ of the selected object.
 
Results:
Most test users felt most comfortable, and accomplished tasks more efficiently using a keyboard and mouse pair.  These tools also had lower error rates. Visual markers and touchpad were least preferred by many users, in part due to limited line of sight and out of camera view issues, as well as finicky touchpad controls. Familiarity was assumed to be a key concern, but the researchers felt that this could be overcome once the learning process was more intuitive (e.g. players quickly picked up gesture interactions using Wii controllers).


Park, J., 2012, August. Evaluation of interaction tools for augmented reality based digital storytelling. In International Conference on Hybrid Information Technology (pp. 263-270). Springer, Berlin, Heidelberg.

An edited book that is a companion publication to the 2018 Brazil exhibition of standout recent interactive and immersive factual narratives in different environments and across platforms.  


Introduction to interactive factuals:

The interactive format is linked to the emergence of web 2.0.  Influential works include _Gone Gitmo (2009) a participatory Guatanamo Bay Detention Center environment in Second Life, Highrise (2009 – 2017), a group of documentary projects about high-rise living noted for its aesthetic innovation (where site design became as informative as text and video) and it’s collaborative, perpetually in development over time production model.  Other notable projects include Alma: A Tale of Violence (2012) a mobile delivery documentary about the violence of Guatemalan street gangs; Snow Fall (2012), an interactive rich-media news report about an avalanche told through an exploration of the location through a choice of skier viewpoints, which also had a narrative path driven by a vertical scroll that inspired the term ‘scrollytelling’; Prison Valley (2010), a docu-game where you play as a prisoner; The Quipu Project (2015), a social impact participatory storytelling project about enforced sterilisation that allowed indigenous South Americans to record and upload their own stories by phone; Do Not Track (2015) a personalised web series about privacy and the data economy. Digital Me (2015) that creates a personalised reflective document of people’s hybrid online/offline identities. 
No format prevails, and hybrids proliferate. 
The Guardian and NYT are both active in this field, as are broadcasters BBC (England), RTVE (Spain), SBS (Australia), Arte (France) and The National Film Board Canada. 
Developmental breakthroughs include relatively cheap headsets, like the Oculus Rift, plus solutions to the challenge of motions sickness, and the emergence of cross-pollinating creative production hubs and festivals such as HotdocX (Montreal, Canada), idocs (Bristol, UK), the IDFA Doclab in Amsterdam, as well as the Tribeca Cinema 360 and Sundance Frontier Lab Festivals (U.S.).  The first production studios emerged out of these labs, along with the Interactive Documentary workshop (Switzerland).  The main distribution platforms for VR content are SteamVR, Oculus and Within. In Brazil, the first interactive documentaries emerged about 2010, are socially engaged and delivered across platforms. 

Immersive Narratives
The exhibition includes the 360 visual experience, Through The Masks of Luzia linked to a Cirque Du Soleil production which allow participants to experience becoming gigantic, as well as tiny; Dreams of O, which combined underwater and slow-motion 360 video; and the immersive VR manga, Tales of Wedding Rings.

The future?
Other formats will continue alongside VR/AR, so complementary streams of media are also likely to continue their importance.

• The duration and complexity of VR experiences is likely to increase. 
• The future status of VR (which offers simulations and interactions via a variety of sensory stimuli, mostly audio-visual) is uncertain because the sale of VR headsets is now going down, not up. 
• But AR is growing, along with emergence of mixed reality headsets. Gaudenzi predicts new works with can be consumed as either AR, or VR. 

Production tips 

1. Put the user at the centre of your creative process
2. Do your research and know what has already been done in the field, as well as where you are placed in terms of style and trends.
3. Embrace multi-disciplinary teams and iterative production
4. Consider co-creation with your subjects
5. Have a clear impact strategy e.g. user impact, social impact, social campaigns – and design action points accordingly
6. Ensure coherency by aligning platform, audience, impact, partners and resources

Paz, A., Gaudenzi, S., Salles, J., Maciel, K.A., Gifreu-Castells, A., Cruz, J. and De La Vega, X., 2019. BUG-Interactive and Immersive Narratives.