We don't normally summarise course content.  Yet, this particular course offers a clear and accessible introduction to augmented reality technologies, so it seems helpful to flag it here.  The main points covered within the course are listed below:

The basics of augmented reality

• Humankind’s first foray into immersive reality through a head-mounted display was the “Sword of Damocles,” created by Ivan Sutherland in 1968.
• HMD is the acronym for “head-mounted display.”
• The term “Augmented Reality” was coined by two Boeing researchers in 1992.
• A standalone headset is a VR or AR headset that does not require external processors, memory, or power.
• Through the combination of their hardware and software, many smartphones can view AR experiences that are less immersive than HMDs.
• Many of the components in smartphones—gyroscopes, cameras, accelerometers, miniaturized high-resolution displays—are also necessary for AR and VR headsets.
• The high demand for smartphones has driven the mass production of these components, resulting in greater hardware innovations and decreases in costs.
• Project Tango was an early AR experiment from Google, utilizing a combination of custom software and hardware innovations that lead to a phone with depth-sensing cameras and powerful processors to enable high fidelity AR.
• An evolution of Project Tango, ARCore is Google’s platform for building augmented reality experiences.

AR functionality

• In order to seem real, an AR object has to act like its equivalent in the real world. Immersion is the sense that digital objects belong in the real world.
• Breaking immersion means that the sense of realism has been broken; in AR this is usually by an object behaving in a way that does not match our expectations.
• Placing is when the tracking of a digital object is fixed, or anchored, to a certain point in the real world.
• Scaling is when a placed AR object changes size and/or dimension relative to the AR device's position. For example, when a user moves away or towards an AR object, it feels like the object is getting larger or smaller depending on the distance of the phone in relation to the object. AR objects further away from the phone look smaller and objects that are closer look larger. This should mimic the depth perception of human eyes.
• Occlusion occurs when one object blocks another object from view.
• AR software and hardware need to maintain “context awareness” by tracking the physical objects in any given space and understanding their relationships to each other -- i.e. which ones are taller, shorter, further away, etc.

Inside-out vs. outside-in tracking

• There are two basic ways to track the position and orientation of a device or user: outside-in tracking and inside-out tracking.
• Outside-in tracking uses external cameras or sensors to detect motion and track positioning. This method offers more precision tracking, but a drawback is the external sensors lower the portability.
• Inside-out tracking uses cameras or sensors located within the device itself to track its position in the real world space. This method requires more hardware in the AR device, but offers more portability.
• On the AR headset side, the Microsoft HoloLens is a device that uses inside-out tracking. On the VR headset side, the HTC Vive is a device that uses outside-in tracking.
• On the AR mobile side, the Google Pixel is a smartphone that uses inside-out tracking for AR.

Fundamentals of ARCore

• ARCore integrates virtual content with the real world as seen through your phone's camera and shown on your phone's display with technologies like motion tracking, environmental understanding, and light estimation.
• Motion tracking uses your phone's camera, internal gyroscope, and accelerometer to estimate its pose in 3D space in real time.
• Environmental understanding is the process by which ARCore “recognizes” objects in your environment and uses that information to properly place and orient digital objects. This allows the phone to detect the size and location of flat horizontal surfaces like the ground or a coffee table.
• Light estimation in ARCore is a process that uses the phone’s cameras to determine how to realistically match the lighting of digital objects to the real world’s lighting, making them more believable within the augmented scene.
• Feature points are visually distinct features in your environment, like the edge of a chair, a light switch on a wall, the corner of a rug, or anything else that is likely to stay visible and consistently placed in your environment.
• Concurrent odometry and mapping (COM) is a motion tracking process for ARCore, and tracks the smartphone’s location in relation to its surrounding world.
• Plane finding is the smartphone-specific process by which ARCore determines where surfaces are in your environment and uses those surfaces to place and orient digital objects. ARCore looks for clusters of feature points that appear to lie on common horizontal or vertical surfaces, like tables or walls, and makes these surfaces available to your app as planes. ARCore can also determine each plane's boundary and make that information available to your app. You can use this information to place virtual objects resting on flat surfaces.
• Anchors “hold” the objects in their specified location after a user has placed them.
• Motion tracking is not perfect. As you walk around, error, referred to as drift, may accumulate, and the device's pose may not reflect where you actually are. Anchors allow the underlying system to correct that error by indicating which points are important.

Constraints with current AR

• Currently AR has a lack of user interface metaphors, meaning that a commonly understood method or language of human interaction has not been established.
• The purpose of the interface metaphor is to give the user instantaneous knowledge about how to interact with the user interface. An example is a QWERTY keyboard or a computer mouse.
• The details of what makes AR challenging from a technical standpoint are complex, but three influential factors are power, heat, and size.
• AR requires high processing power, batteries generate heat, and a current challenge is fitting all the necessary components into a small enough form factor to wear on your face comfortably for extended periods of time.
• Not everything in AR has to be 3D, but the vast majority of assets, applications, and experiences will require at least a little 3D design.
• Currently, there is a limited base of people with 3D design and interaction skills, such as professional animators, graphic designers, mechanical engineers, or video game creators. For AR to grow, the adoption of 3D design theory, skills, and language needs to become much more widespread. Later on in this course, we’ll be discussing a few programs that are helping overcome this challenge, like Sceneform or Poly API.
• Computer vision is a blend of artificial intelligence and computer science that aims to enable computers (like smartphones) to visually understand the surrounding world like human vision does. This technology needs to improve in terms of object detection and segmentation to make AR processes more effective.

Use cases and current powers/limitations of AR

• ARCore can be used to create dynamic experiences for businesses, nonprofits, healthcare, schools, and more.
• ARCore’s strengths are its phone-based spatial mapping capabilities and addressable user base. Approximately 85% of phones around the world run on the Android operating system.
• At the beginning of 2018, ARCore is already available on 100 million Android-powered smartphones and that number continues growing. ARCore requires a lot of processing power, so not all older Android models have the necessary specifications yet. ARCore is also available in China.
• Limitations to consider with contemporary AR technology include: low-light environments, a lack of featured surfaces, and the availability of powerful mobile processors in new phones.

Basic AR interaction options

1.     Drag and Drop
2.     Voice
3.     Tap
4.     Pinch and Zoom
5.     Slide
6.     Tilt

Think like a user

• User flow is the journey of your app's users and how a person will engage, step by step, with your AR experience.
• Planning your user flow needs to take into account the scene, the user interactions, any audio cues, and the final user actions.
• A user flow can be created with simple sketches and panels all collected into one cohesive diagram.
• UX and UI are complementary fields of product design, and generally speaking UX is the more technical of the two.
• When considering UX/UI, one good rule of thumb to remember with AR is to avoid cluttering the screen with too many buttons or elements that might be confusing to users.
• Choosing to use cartoonish designs or lighting can actually make the experience feel more realistic to the user, as opposed to photorealistic assets that fail to meet our expectations when they don't blend in perfectly with the real world.
• Users might try to “break” your experience by deliberately disregarding your carefully planned user flow, but your resources are better spent on improving your app’s usability rather than trying to prevent bad actors

Next steps on the AR journey

• Advanced 3D design tools like Maya, Zbrush, Blender, and 3ds Max are powerful professional tools.
• Google’s Poly can be a good starting resource for building your first ARCore experience.
• Poly by Google is a repository of 3D assets that can be quickly downloaded and used in your ARCore experience.
• The recommended guide for your AR experience is a design document that contains all of the 3D assets, sounds, and other design ideas for your team to implement.
• You may need to hire advanced personnel to help you build your experience, such as: 3D artists, texture designers, level designers, sound designers, or other professionals.

A closer look at mechanics of ARCore

• Surface detection allows ARCore to place digital objects on various surface heights, to render different objects at different sizes and positions, and to create more realistic AR experiences in general.
• Pose is the position and orientation of any object in relation to the world around it. Everything has its own unique pose: from your mobile device to the augmented 3D asset that you see on your display.
• Hit-testing lets you establish a pose for virtual objects and is the next step in the ARCore user process after feature-tracking (finding stationary feature points that inform the environmental understanding of the device) and plane-finding (the smartphone-specific process by which ARCore determines where horizontal surfaces are in your environment).
• Light estimation is a process that allows the phone to estimate the environment's current lighting conditions. ARCore is able to detect objects in suboptimal light and map a room successfully, but it’s important to note that there is a limit to how low the light can be for the experience to function.
• Occlusion is when one 3D object blocks another 3D object. Currently this is only possible with digital objects, and AR objects cannot be occluded by a real world object. For example, in an AR game the digital object would not be able to behind a real couch in the real world.
• Assets in multi-plane detection are scaled appropriately in relationship to the established planes, though only need to be placed on them (via anchor points) when it causes them to function like their real-world counterparts.
• Immersion can be broken by users interacting with AR objects as if they were physically real. Framing can be used to combat these immersion-breaking interactions.
• Spatial mapping is the ability to create a 3D map of the environment and helps establish where assets can be placed.
• Feature points are stationary and are used to further environmental understanding and place planes in an experience. ARCore assumes planes are unmoving, so it is inadvisable to attempt to anchor a digital object to a real world object that is in motion. In general, it’s best not to place an object until the room has been sufficiently mapped and static surfaces have been recognized and designated as feature points.

Using Poly and Unity to create ARCore assets

• Unity is a cross-platform game engine and development environment for both 3D and 2D interactive applications. It has a variety of tools, from the simple to the professionally complex, to allow for the streamlined creation of 3D objects and environments.
• Poly toolkit for Unity is a plugin that allows you to import assets from Poly into Unity at edit time and at runtime.
• Edit-time means manually downloading assets from Poly and importing them into your app's project while you are creating your app or experience.
• Runtime means downloading assets from Poly when your app is running. This allows your app to leverage Poly's ever-expanding library of assets.

Even despite Google's undeniable motivation to promote its own products via this course, the course explanation of the above summary points is helpful and clear, enhanced by videos, diagrams and graphics.

Coursera course content is free for University students.  Certification is optional for an extra fee.  

VR, Google AR &. "Introduction to Augmented Reality & AR Core." Coursera, accessed 22.9.20. https://www.coursera.org/learn/ar.

Augmented reality (AR), creators can use the space around participants to visualize media, enable participants to organise those aspects with their own, and share it all with peers face-to-face. This is a Spatial Interface.

META an augmented reality glasses company researched  SPATIAL INTERFACE DESIGN GUIDES in collaboration with a diverse team of neuroscientists, led by Professor Stefano Baldassi (previously of Stanford University), UX designers and developers, advisers from industry and academia, and Meron Gribetz (founder and CEO of Meta), studying the way that the brain understands natural 3D interfaces. 

Guideline 1
Think Spatial: Place Tools and Content in Space
Replace flat layouts of windows, menus, and buttons, with a Spatial Interface that arranges tools and content in 3D space around the user. 
Do
Arrange volumetric tools and content in space.
Don’t
Cram traditional GUI elements (windows, icons, menus, etc.) into 2D panels.
UI Design Suggestions
1. Reconsider traditional windows and screen-based conventions
For example, don’t place a “start menu” in space, since it was designed for a small screen and presents a cluttered mess of buttons, abstract icons, and tools. 
2. Separate elements into two buckets: volume and content
Text and video are inherently flat, and should be contained within tangible 3D structures with their own sense of affordance. 3D models, on the other hand, may exist as free-standing objects of their own and their design should suggest their use. 
3. Distinguishing between tools and content
Content is an experience in and of itself; it conveys information or some kind of sensory experience. Tools, on the other hand, serve the purpose of creating, modifying or in some way interacting with content.  The "three-dimensional tool" needs to be based on the user's intuition, and the "content" may have an abstract and flat shape, or it may have a realistic and three-dimensional shape.  For example, text and video are flat, but need to be embedded in some three-dimensional structure. On the other hand, since the 3D model is originally three-dimensional, it can exist as an independent object.
4. Avoid expandable or hidden menus in AR
Instead of imitating the flat world of traditional traditional UIs into a three-dimensional world as they are, it is necessary to get inspiration from actual work sites such as art studios and workshops. Instead of thinking about menus and buttons, you should think of tools that the user can actually grasp.

Guideline 2
Minimize Abstractions: Design Tools with Volume and Affordances
Now that we understand how to arrange objects in space, how should we design the objects themselves? Replace abstract representations (like flat icons) with volumetric tools featuring physical characteristics that suggest their use without explicit instruction, leveraging the neuroscience of affordance.
Do
Use realistic volumetric designs for tools and content, creating intuitive affordances for the user.
Don’t
Represent objects with abstractions such as iconography

UI Design Suggestions
1. Design with affordances in mind
Design tools with affordances, which are physical characteristics that suggest how the tool should be grasped or used. For example, a holographic eraser might have grooves to invite grasping it on one side, and a flat surface for erasing on the other side.  Objects and tools designed with affordance in mind are better than abstract icons. Can be understood quickly and deeply.  
2. Build on the user’s prior knowledge of a tool instead of defining your own​
Design based on prior knowledge, to enhance usability.  For example, don’t reinvent the paintbrush in some radically new way.   
3.Use tools only if direct hand manipulation is insufficient or biomechanically challenging
Direct hand interaction is preferable for most simple tasks, such as moving, rotating, or scaling an object. 
4. Avoid use of buttons in AR as much as possible
For instance, rather than presenting a “send” button when the user writes an email, provide a mailbox that the user can drop the email into.
5. Compensate for lack of haptics with other sensory cues
For instance, consider synchronized audio cues when the user’s hand interacts with tools and content, or build hybrid tools that mix digital interfaces with real-world objects, or use physical markers that permit holographic drawing in space. 

Guideline 3
You are the OS: Organize Holographic Files and Tools in the User’s Environment
Replace the traditional abstract and complex hierachy of file systems with spatial organization of files that makes sense for people, it makes them easier to engage with.
Do
Use the physical environment to create organization and structure.
Don’t
Use the abstract, nested file systems of traditional UIs.

UI Design Suggestions
1 Use space as an organizational tool
For example, if the user places a holographic object in a specific area of a physical desk, their spatial memory of its placement makes it easy and faster to search later.
2 Use volumetric containers and holographic furniture
Also, the object should use a three-dimensional container rather than an abstract folder or file system. Unless these containers have a deep hierarchical structure, files can be easily retrieved ergonomically. 
3 Don’t overload the user’s working memory with nesting
These containers should not be nested. Nesting overloads the user's spatial memory and makes the file unintuitive.  If the application does need nesting, then do not exceed one level. Despite being commonplace in traditional UIs, a holographic drawer should not also feature hidden “sub-drawers” within it. 
4. Preserve the user’s spatial memory
When the user is selecting an object, other unselected objects should not be hidden Yes. This is because hiding impairs the integrity of the user's spatial memory. Highlight the selected object instead of hiding it.
5 Miniaturize content and tools to optimize space
On the other hand, miniaturizing content and tools (or vice versa) is effective in maximizing the user's workspace without disrupting spatial memory.

For example, family photos are routinely kept by the thousands in labeled albums, which can then be organized on shelves. This does not represent a violation of those principles because the photo albums are already exposed, and once opened, the user immediately arrives at the content they’re looking for—hence one level of hierarchy deep. This use of furniture, simple grouping, and shallow nesting provides AR developers and designers with an effective model for organizing huge amounts of holographic content without the cognitive overhead of abstract file systems. 

Guideline 4
Touch to See: Use the Hands to Interact Directly
The brain naturally tends to recognize objects near the hand. The tools and the content they operate on should not be physically or spatially separated.  That is, the user should not operate the content from a distance or mediate abstract objects such as gestures, buttons, and menus.
Do
Use direct hand manipulation to use tools and interact with content.
Don’t
Use remote gestures and abstract pointers.

UI Design Suggestions
1. Touch content to act on it
Tools, actions, and the content on which they operate should never be physically or spatially separated. 
2. When grabbing¨ pushing¨ or expanding
It is necessary for the object to reflect the user's physical movements and movements such as grasping and spreading. It should not be associated with gestures that the user must first learn the action and function of, such as opening a fist or waving to spread an object.  
3. Avoid gestures that require specialized knowledge or memorization
While power user shortcuts are enticing ways to speed up interaction with an interface, they leave behind most of humanity (the non-power users). Instead of discrete gestures, use affordances to encourage the user to interact with objects in more natural ways.
4. Persistence and the usage of tools
Tools intended for persistent use (such as a paint brush, a conductor’s wand, or a flashlight) should realistically respond to the user’s manipulation, instead of requiring a discreet hand gesture to activate or deactivate. For instance, the brush should draw when pressed against a surface, rather than in response to a separate trigger like a gesture or button press. 
5. Proximity feedback is your friend
Provide proximity feedback when the user uses or touches an object or content. By doing this, it is possible to compensate for the lack of tactile sensation and emphasize collisions and movements.  Glows and subtle audio cues are a good way to accent a collision or movement while compensating for the lack of haptics. 

Guideline 5
Do Not Disturb: Do not interrupt the User’s Workflow
Spatial computing demands spatial notifications. Rather than interrupting the user’s workflow with pop-ups, allow users to designate a separate container to passively collect incoming notifications. This allows the user to remain focused on their task, only stopping to check for updates when desired.
Do
Establish a passive notification receptacle: non-intrusive, segregated from the task at hand by default, and easy to move.
Don’t
Interrupt the user’s task flow with active notifications and pop-ups.

UI Design Suggestions
1Separate designated areas
Allow the user to designate a container or space to collect incoming notifications away from their workspace, instead of interrupting them directly with pop-ups or alerts. This preserves the user’s control over their experience by allowing them to focus on the task at hand, only breaking their concentration for updates when desired. Ideally, such a box would be located outside the user’s field of view and at some reasonable distance, ensuring disruptions are avoided. As a default, prioritise the user’s concentration, only allowing disruptive notifications when requested explicitly by the user.
2. Design tools for specific purposes
Certain tasks require bottom-up disruptions to break top-down flow (like when your timer beeps), and should be incorporated into tools designated for that specific purpose, but even these tools should mimic real world tools like alarm clocks and egg timers. Most importantly, the user should always understand how to turn off the sounds and visuals associated with alerts.
3 Allow for tailoring and flexibility
Create flexibility by allowing the user or developer to designate multiple boxes, and tailor them to receive certain kinds of notifications. For instance, one box might be associated with all social media apps from different services or people, while another might be for email only. A third box might handle all incoming phone calls, making it easy for the user to selectively allow calls to disrupt their work (by bringing this particular box closer to the workspace). Boxes can also be connected to specific people or groups, allowing for further organization of incoming notifications. Furthermore, the design of the box itself should use affordances to suggest its use, such as a lid that can be closed to mute notifications, or opened to enable them.
4 Consider gradual notifications over abrupt ones
Even if users choose to place notifications within their workspace, they should consider using gradual visual and audio cues instead of explosive alerts.

Guideline 6
Avoid Surprises and Magic Tricks: Pair Actions with Intuitive Outcomes
Avoid “magical” events that appear unrelated to user behavior or earlier causes, or events that ignore the laws of physics as these only confound the user.
Do
Create predictable connections between a user’s actions and the result.
Don’t
Use nonsensical, disorienting quirks and surprises.

UI Design Suggestions
1Avoid “magic” by tailoring outcomes to match the user’s actions
Here, “Magic ” refers to anything that confuses the intuitive sense of cause and effect. A magic wand, for example, is confusing because it looks more like a stick and there is no affordance associated with a particular task. Also, people can't predict the consequences of using it. On the other hand, most users will immediately know how to use an eraser; its use can be understood and its effects predicted with no additional explanation.
2. Use realistic physics to create intuitive object interactions and behavior
It also confuses the user when the object does not respond in a reasonable way.
For example, even if you gently push a holographic ice cube and slide it on a desk, do not fly across the room at 900 m / h. It is necessary to have a clear relationship between actions and effects.
3 Enforce causality with clear associations between action and effect
A sound triggered by a user’s action should occur immediately in real-time and emanate from the appropriate location. Likewise, visual feedback such as glowing highlights, shadows, or physical deformations should be proportionate and spatially close to the event that triggered them.

Guideline 7
The Holographic Campfire: Don’t Obscure Hands and Faces with the UI
Our ancestors have evolved to face each other, both in pairs and in tribes. Therefore, we are very sensitive to the gaze of the eyes and the facial expressions of others, and the dedicated areas of the brain continuously perceive those sensations.
Our perception of the line of sight extends to its surroundings and is immediately noticed when someone else is looking at something of interest.
Avoid blocking eye contact, create shared spaces that encourage a mutual gaze, and ensure hands are visible while collaborating. 
Do
Replace UIs that disrupt eye contact with a shared space. Also, collaborate by directly manipulating shared content.
Don’t
Obscure or occlude the face of others with interface elements, or use single modality collaboration, such as face-to-face collaboration without the hands or voice chat without video.

​UI Design Suggestions
1. Promote unobstructed views
Avoid interfaces that occupy the entire field of view and obscure facial expressions—two significant components of personal communication. Within a shared physical space, interfaces should be arranged around users, not between them. 
2. Support collaborative experiences when working in the same location
For example, a collaborative design tool should encourage all participants to manipulate the same object in the same place, as opposed to private instances visible to each individual separately.  
3. Show hands when collaborating
The interface must clearly indicate the position of the user's hands.  When a user's hands are within the sight of other users, the ability to learn from collaboration and demonstration increases.
4.When collaborating remotely¨ use 3D video of participants instead of avatars or virtual characters
Artificial expressions such as still images, icons, and virtual characters only reduce the accuracy of communication and improve cognitive burden and social anxiety.

Guideline 8
Public by Default: Shared Understanding Reduces Anxiety Among Users
We have evolved to constantly observe the attitudes, movements, and behaviors of those around us in order to attempt the intentions of others, and to guess the minds of others.  By default, other people's content should be viewable. All users should be able to see the same holographic environment, just as they see the same physical environment in the real world.
Do
Make tools and content public by default. Maintain the ability to make
things private if needed.
Don’t
Offer private, asymmetric UIs (like the ones seen in Google Glass).

UI Design Suggestions
1. By default¨ ensure content can be viewed by anyone else wearing a headset
All users should be exposed to the same holographic surroundings, just like we all see the same physical surroundings in the real world. When a user's content is hidden from other users in AR, the action appears to be in an empty space . To confuse. Furthermore, devices like Google Glass have demonstrated our distrust of private interfaces because they allow aggressive or invasive actions that can be performed secretly, even when face-to-face. Such practices may divide users, rather than encourage cooperation, and may contribute to social anxiety.
2. When privacy is necessary¨ use a blocking object like curtains or dividers
Do not simply make sensitive content invisible from the perspective of unauthorized users, as this creates a disparity between users that can lead to confusion and distrust. Instead, just as in real life, establish privacy with holographic curtains or dividers. This approach ensures a consistent experience across all users without confusion or mixed signals, and eliminates the disorienting sight of gestures performed on empty space.
3. Privacy is a function of etiquette¨ not a feature of the interface
Augmented reality is most powerful when treated as a single, shared space. Don’t relegate privacy to an on/off switch that segregates user experiences. Confusion arises when this otherwise collaborative experience is unknowingly changed from one user to another for the sake of privacy or discretion. For example, a user who wishes to view financially sensitive information should do so within an interface that has privacy features, such as a curtain, divider, or display that can be tilted away from unwanted viewers. Users should design their own privacy within the rules of the shared space, as opposed to breaking the rules.

Guideline 9
Augmented, Not Mixed, Reality: Enhance the User’s Perception with Relevant Information
Rather than block out users’ reality (as in virtual reality) or distort the user’s reality (as in mixed reality), provide access to metadata about the world placed near by, without masking it—add something useful to people's understanding of their environment. Create an informative, powerful, and unobtrusive layer of digital information on top of the real
world.
Do
Present information and tools that reflect the user’s reality and deepen their understanding of it.
Don’t
Block or distort reality, whether fully or partially.

UI Design Suggestions
1. Complement the real world
When images and sentences added in the user's field of view by the AR (Augmented Reality) interface are added to the real world without discomfort, the user can understand them intuitively and effectively.  Design interfaces to complement real-world objects, instead of masking or transforming them. For instance, a panel with a Wikipedia page about a flower placed near, but not occluding, the flower would be considered augmented reality. Consider a user that is able to touch a flower and learn about the amount of DNA s/he shares with it. This simplistic example demonstrates how one can enhance their connection with their surrounding environment.
2. The experience of turning the gym floor back to the surface of the pool, in contrast, causes unnecessary confusion and misleads the user about how it works.
This is because it changes or distorts the physical properties of the actual object.
3. Add value and insight for the user
Rich content should be displayed to provide information with as much clarity and depth as possible. Text, video, volumetric content, and the Web can all play a role in empowering the user with deeper understanding and connection to their world.
4 A note about use cases in gaming, In games, we recommend allowing virtual characters to pass through the actual user's environment, rather than "breaking walls" or covering them with overly distorted visuals. On the other hand, for certain uses, such as interior decoration, momentary changes in the environment are tolerated. This use case leads to changes and improvements in the physical properties of the real world.

The report is further summarised in these two blog posts: 

• https://realityrookie.com/2017/03/16/use-neuroscience-to-design-the-ar-interfaces-of-the-future/
• https://qiita.com/A_kkie/items/37f5496838012791ae77

For the full report, together with example images and related neuroscience references, see:
https://pages.metavision.com/meta-augmented-reality-design-guidelines/

ATA 2018 Audience of the Future Presentation.  
 

• Prof Andrew Chitty (Director, Audience of the Future Programme) spoke with Ruth Sessions (Director of Production, Factory 42), Emily Smith (Director of Marketing and BD, Atlantic Productions), Paul Moore (Head of School Communication & Media, Ulster University), Greg Furber (Creative Director, Rewind).Alchemy VR (moved from documentary into museum interpretation.

​
Alchemy VR: Our first experimental virtual reality was back in 2015, where we partnered with the Natural History Museum.  We worked on an experience where we walked people through the Cambrian ocean, a 500 million year old environment based on fossil data held at the museum. David Attenborough took you through that amazing world which we delivered on location in the Natural History Museum.  Visitors would come in, wear a headset, and they'd get the VR experience then they'd see the actual specimens. And it was a great example of a use case of VR, because how else could visitors go back in time to that period and understand what it's like to be in that environment.
Visitor feedback:
86% of visitors learnt something new about the natural world
68% of visitors also learnt something new about the museum’s scientific research, which is a really important outcome.
 
On the basis of that success, we again worked with David on a VR experience that takes you down into the Great Barrier Reef and visitors could explore the barrier reef with him, again, a use case for VR that other museum interpretive tools could not deliver.  How can you go down into the barrier reef unless you're extremely rich, and dive that environment and understand about the fragility of it.  Both of those experiences have run commercially in museums right across the world.
 
In other words, VR is an interpretive tool that offers things that other museum interpretation tools cannot do.  It puts visitors in the position of somebody actually experiencing something, or undertaking an otherwise impossible journey.
 
The VR lounge was delivered on site in the Science Museum, and also toured around the UK to a number of venues across two years.
Also, the VR bus takes a cut down version of the experience into hard to reach communities and schools to engage people across the nation.
 
And another experience that we've worked on involves more of a 4k experience and supported efforts to defend the Amazon for Greenpeace who wanted to create a piece that engaged people with the plight of the Maduro tribe who are about to be wiped out by the building of a dam.  We partnered with another company to create an experience whereby, as virtual reality visitors had fully visceral physical experience and sensory experience delivered in really beautiful pods. And we took a perfume out to the Amazon to capture the smells of the Amazon. Those were delivered in the pod. So as you were walking through the Amazon with Munduruku people, you'd smell what they would smell as point in experience where you have a cup of coffee with the people and you suddenly smelt coffee and felt the heat of the coffee cup in your hand. So really beautiful, visceral experience and it ran across a number of museums across South America. And best of all, the dam was not built.
 
The entertainment sector I wanted to talk about is Disney's Pandora. This experience was built around the Avatar film. As Jake played by Sam Worthington in the movie, you fly on a banshee. And this whole experience, the VR ride essentially is one part of a massive, immersive experience. The environment consists of built mountains and beautiful streams that visitors walk through.  Visitors engage in the physical experience, long before they come face to face with an avatar in a jar. They're given their own avatar through which they then enter the virtual reality.  So, there's elements of personalization and there are also elements of extremely theatrical environments. You're got wind, you've got mist you really feel like you're on the back of a banshee.
 
We're also doing big warehouse scale gaming VR, which have included lots of zombie experiences.  Participants engage with each other in a game environment in the physical space. Noma ran an experiment called virtually dead, which essentially emerged as a kind of theatrical experience with a virtual reality experience. Participants first had to train with actors before they came face to face with the zombies, both as actors and then in the real world in an East London warehouse. This is a small kind of startup essentially tickets sold out immediately. And they started going for five times their market price on the internet.
 
 
AR examples.
The Science Museum did the first AR museum experience, I think.  They used AR to essentially invigorate a gallery, which was very tired from the 1970s to add vision of skin on bones in order to enable visitors to understand more about the creatures in a more exciting way. AR is now also being used to gamify a kind of commercial visitor attraction. Cedar Point used AR, to essentially build a game for visitors around the characters in the theme park with a lot of social interaction and storytelling. So virtual creatures appeared in the real world and visitors formed groups to play games against each other in the theme park. Greg from rewind has also got a wonderful example from his own work, a NASA experiment with their Jet Propulsion Lab and the pre commercial HoloLens in which shows Buzz Aldrin playing that at the Kennedy Space Center. They created a Mars environment where multi visitors at the same time could engage in a virtual Mars environment. 
 
 
Factory 42: Hold the world involves David Attenborough, and gives visitors access to treasures of the Natural History Museum.  In VR visitors can pick up and turn the objects around to examine them much closer…and watch the objects come to life.
The key to Hold The World is that it gives the user the chance to do something that they simply can't do in real life, which is crucial.  Also, once visitors put on the headset it’s worth it because they have a one on one audience with David Attenborough. The experience also takes visitors behind the scenes in the Natural History Museum to three extraordinary rooms that the public is not able to enter.
 
We sort of think of it it's hard to explain, really, but it's, it's somewhere between a TV documentary and a computer game.
 
Eight organizations worked together to make this possible:

• So the first technology we used was a holographic capture. We capture David as a hologram. So the technologies volumetric capture So 106 cameras surrounded him in the green screen studio, which captured every angle of his performance. So the output of that is a hyper realistic 3d hologram, that even your close up really feels like you're sitting across the desk from him.
• We also use photogrammetry. We use photogrammetry to scan the three the three behind the scene environments in the Natural History Museum.

We also used game engine technology, high end visual effects, and several other technologies as well, which combined with a script from a BAFTA winning director.
 
We advise three broad principles to success in this space.
1) A balance between storytelling and consumer insight and technology. So a lot of what we see out there right now doesn't really have that balance. Too much is led by the technology, and not what the consumer might want to experience. … the key thing is, and this is really obvious, but it's sometimes easy to forget, is for the technology to enable, rather than to lead people and actually what's the story you want to tell? And then how can the technologies help you tell the story?
2) I think the second thing you need to succeed is resilience and an appetite for a tough challenge. I mean, this stuff is hard to do, particularly at scale. The technology doesn't always work. It's changing quickly, and finding the right people and good people who know how to work with it is also challenging.  It took us probably about a year to work out how we'd make Hold the world actually before we started in production.
3) Collaborate effectively, because no one's got all the answers. So bringing the cultural world and the tech world together. Both have to adapt the way they work. And that's a key learning from my own personal experience on making Old World was we spoke different languages. I mean, we really did it as if we should have had a sort of simultaneous translator with us at all times. But sides of the team. It wasn't two sides. But both people from both backgrounds in it, we've moved. We grew in understanding how each other works and things. And I think we all emerged wiser from that.  These are complicated relationships with multiple stakeholders

Change is happening quickly.
 
Hololens (and now Unity MARS) - The reality of what the technology can do is absolutely mind blowing. It can scan a room like this can map every surface it can place objects and things on those surfaces and explore an entire world. The building blocks from that are things like unity AI technologies that are in the hands of any talented development team working within this space. And those things are transferable across the entire pipeline.
 
So something that HoloLens today could soon enable us to see smoke come out of trains as we walk around them.   We can bring up the data that people are squinting at.  We can bring in objects you could not otherwise have that space.  
 
If you think of Red Bull Races.  Actually, what we're doing is bringing in real time race to the telemetry data, we're bringing in multiple video feeds, we're using the HoloLens as a control unit that controls multiple screens in the space, so you can switch between different pilots racing. Red Bull air race is the fastest motor sport on the planet, because even though a plane might be doing 200 miles an hour, they're getting pulled up on a slight technicality because their wings weren't perfectly aligned at a certain point or they lost by nought point nought one of a second. Also, when you can't see two planes on a racetrack at the same time, the danger, the excitement and the power that's going out in the field out there doesn't translate. So with technologies like this, we're able to show the truth of what's going on. We can use the data to help us tell the full, dramatic story what's happening out in the racetrack.
 
 
How do you please more than one visitor in the same experience? Is that visitor coming along to get a visual experience? Are they going to have a kind of public experience of some kind? Are they going to have a visceral experience? Is it going to be emotive? Who are you actually appealing to? Now it seems to me it has to be a global audience, given the scale. So how do you square that circle of having somebody down the street walk in, and somebody's logging in remotely from China, applying a very different cultural understanding to whatever experience you're giving them. The technology has to deliver a culturally variable message. Also, a great experience is not really about single experiences.  Coming to a museum isa community experience. Putting on a headset optimizes that experience, but it also takes you out of that community, plus it makes you look like a dick (e.g. you might want to consider using mobiles, or glasses instead). But you see what I'm getting at, the experience is being dictated by the Technology, not necessarily the other way around. The curiosity should not be driven by whatever the technology happens to be.
 
Imagine we’ve got the Giant's Causeway. It's very boring. But actually, if you were trying to make a visitor experience out of that, you can ask a lot of questions about whether it is a geological site. Yes. Is it a contested site? Absolutely. Because the other folkloric story is that a giant called Finn McCool made this place. And if you think it was Finn McCool, that means that you're coming from a Catholic background, and that makes you a Republican. And if you're a unionist, the Giant's Causeway is where you realized that you are connected and not separated by water to your ancestors in Scotland. Contested space! So you've made a visitor experience, and three people have just come through the door. And they've all got different understandings of what that is. How do you represent those narratives? Because it's those narratives that are important. And more crucially, how do you represent those narratives to a Chinese person who has their own narratives? So if I was putting the visitor experience together, I would be thinking about a template of applied understandings and ideas which can move to anywhere in the globe. So you've created a template which you can then sell to South America or you sell to us Africa where they can tell their stories with their cultural resonances. But you've created a vehicle through which they can do that using the technology.
 
Constructing a mixed reality, sensation, experience, which allows people to bring their reality and construct their own narrative coming out of that experience space is quite a challenge. You've got your narratives, where do you set that into?  You may create a landscape, a soundscape, a cityscape, but there has to be a context in which those experiences happen. And those narratives can communicated regardless of the technology, so don't fixate on the technology.
 
You want everyone to know the same piece of information and to be able to share an experience. To help people have their own private experience and also find out the shared base-line, does there need to be a group activity, or a walking tour together, or simply a way to share with families and friends? The answer depends upon who you're trying to reach and what you're trying to help them understand.

I teach museum personnel on a master's program, and it's my job to encourage museum workers to take on digital tools and they are absolutely resistant to that space because they believe passionately in the authenticity of the object. And that's the kind of problem you're up against How do you convince people that those technologies enhance the authenticity of the object?