STATE OF THE ART IN XR HERITAGE EXPERIENCE
PANEL_ STATE OF THE ART IN IMMERSIVE HERITAGE VISITOR EXPERIENCES
ATA 2018 Audience of the Future Presentation.
Prof Andrew Chitty (Director, Audience of the Future Programme) spoke with Ruth Sessions (Director of Production, Factory 42), Emily Smith (Director of Marketing and BD, Atlantic Productions), Paul Moore (Head of School Communication & Media, Ulster University), Greg Furber (Creative Director, Rewind).Alchemy VR (moved from documentary into museum interpretation.
Alchemy VR: Our first experimental virtual reality was back in 2015, where we partnered with the Natural History Museum. We worked on an experience where we walked people through the Cambrian ocean, a 500 million year old environment based on fossil dataheld at the museum. David Attenborough took you through that amazing world which we delivered on location in the Natural History Museum. Visitors would come in, wear a headset, and they'd get the VR experience then they'd see the actual specimens. And it was a great example of a use case of VR, because how else could visitors go back in time to that period and understand what it's like to be in that environment.
Visitor feedback:
86% of visitors learnt something new about the natural world
68% of visitors also learnt something new about the museum’s scientific research, which is a really important outcome.
On the basis of that success, we again worked with David on a VR experience that takes you down into the Great Barrier Reef and visitors could explore the barrier reef with him, again, a use case for VR that other museum interpretive tools could not deliver. How can you go down into the barrier reef unless you're extremely rich, and dive that environment and understand about the fragility of it. Both of those experiences have run commercially in museums right across the world.
In other words, VR is an interpretive tool that offers things that other museum interpretation tools cannot do. It puts visitors in the position of somebody actually experiencing something, or undertaking an otherwise impossible journey.
The VR lounge was delivered on site in the Science Museum, and also toured around the UK to a number of venues across two years.
Also, the VR bus takes a cut down version of the experience into hard to reach communities and schools to engage people across the nation.
And another experience that we've worked on involves more of a 4k experience and supported efforts to defend the Amazon for Greenpeace who wanted to create a piece that engaged people with the plight of the Maduro tribe who are about to be wiped out by the building of a dam. We partnered with another company to create an experience whereby, as virtual reality visitors had fully visceral physical experience and sensory experience delivered in really beautiful pods. And we took a perfume out to the Amazon to capture the smells of the Amazon. Those were delivered in the pod. So as you were walking through the Amazon with Munduruku people, you'd smell what they would smell as point in experience where you have a cup of coffee with the people and you suddenly smelt coffee and felt the heat of the coffee cup in your hand. So really beautiful, visceral experience and it ran across a number of museums across South America. And best of all, the dam was not built.
The entertainment sector I wanted to talk about is Disney's Pandora. This experience was built around the Avatar film. As Jake played by Sam Worthington in the movie, you fly on a banshee. And this whole experience, the VR ride essentially is one part of a massive, immersive experience. The environment consists of built mountains and beautiful streams that visitors walk through. Visitors engage in the physical experience, long before they come face to face with an avatar in a jar. They're given their own avatar through which they then enter the virtual reality. So, there's elements of personalization and there are also elements of extremely theatrical environments. You're got wind, you've got mist you really feel like you're on the back of a banshee.
We're also doing big warehouse scale gaming VR, which have included lots of zombie experiences. Participants engage with each other in a game environment in the physical space. Noma ran an experiment called virtually dead, which essentially emerged as a kind of theatrical experience with a virtual reality experience. Participants first had to train with actors before they came face to face with the zombies, both as actors and then in the real world in an East London warehouse. This is a small kind of startup essentially tickets sold out immediately. And they started going for five times their market price on the internet.
AR examples.
The Science Museum did the first AR museum experience, I think. They used AR to essentially invigorate a gallery, which was very tired from the 1970s to add vision of skin on bones in order to enable visitors to understand more about the creatures in a more exciting way. AR is now also being used to gamify a kind of commercial visitor attraction. Cedar Point used AR, to essentially build a game for visitors around the characters in the theme park with a lot of social interaction and storytelling. So virtual creatures appeared in the real world and visitors formed groups to play games against each other in the theme park. Greg from rewind has also got a wonderful example from his own work, a NASA experiment with their Jet Propulsion Lab and the pre commercial HoloLens in which shows Buzz Aldrin playing that at the Kennedy Space Center. They created a Mars environment where multi visitors at the same time could engage in a virtual Mars environment.
Factory 42: Hold the world involves David Attenborough, and gives visitors access to treasures of the Natural History Museum. In VR visitors can pick up and turn the objects around to examine them much closer…and watch the objects come to life.
The key to Hold The World is that it gives the user the chance to do something that they simply can't do in real life, which is crucial. Also, once visitors put on the headset it’s worth it because they have a one on one audience with David Attenborough. The experience also takes visitors behind the scenes in the Natural History Museum to three extraordinary rooms that the public is not able to enter.
We sort of think of it it's hard to explain, really, but it's, it's somewhere between a TV documentary and a computer game.
Eight organizations worked together to make this possible:
So the first technology we used was a holographic capture. We capture David as a hologram. So the technologies volumetric capture So 106 cameras surrounded him in the green screen studio, which captured every angle of his performance. So the output of that is a hyper realistic 3d hologram, that even your close up really feels like you're sitting across the desk from him.
We also use photogrammetry. We use photogrammetry to scan the three the three behind the scene environments in the Natural History Museum.
We also used game engine technology, high end visual effects, and several other technologies as well, which combined with a script from a BAFTA winning director.
We advise three broad principles to success in this space.
1) A balance between storytelling and consumer insight and technology. So a lot of what we see out there right now doesn't really have that balance. Too much is led by the technology, and not what the consumer might want to experience. … the key thing is, and this is really obvious, but it's sometimes easy to forget, is for the technology to enable, rather than to lead people and actually what's the story you want to tell? And then how can the technologies help you tell the story?
2) I think the second thing you need to succeed is resilience and an appetite for a tough challenge. I mean, this stuff is hard to do, particularly at scale. The technology doesn't always work. It's changing quickly, and finding the right people and good people who know how to work with it is also challenging. It took us probably about a year to work out how we'd make Hold the world actually before we started in production.
3) Collaborate effectively, because no one's got all the answers. So bringing the cultural world and the tech world together. Both have to adapt the way they work. And that's a key learning from my own personal experience on making Old World was we spoke different languages. I mean, we really did it as if we should have had a sort of simultaneous translator with us at all times. But sides of the team. It wasn't two sides. But both people from both backgrounds in it, we've moved. We grew in understanding how each other works and things. And I think we all emerged wiser from that. These are complicated relationships with multiple stakeholders
Change is happening quickly.
Hololens (and now Unity MARS) - The reality of what the technology can do is absolutely mind blowing. It can scan a room like this can map every surface it can place objects and things on those surfaces and explore an entire world. The building blocks from that are things like unity AI technologies that are in the hands of any talented development team working within this space. And those things are transferable across the entire pipeline.
So something that HoloLens today could soon enable us to see smoke come out of trains as we walk around them. We can bring up the data that people are squinting at. We can bring in objects you could not otherwise have that space.
If you think of Red Bull Races. Actually, what we're doing is bringing in real time race to the telemetry data, we're bringing in multiple video feeds, we're using the HoloLens as a control unit that controls multiple screens in the space, so you can switch between different pilots racing. Red Bull air race is the fastest motor sport on the planet, because even though a plane might be doing 200 miles an hour, they're getting pulled up on a slight technicality because their wings weren't perfectly aligned at a certain point or they lost by nought point nought one of a second. Also, when you can't see two planes on a racetrack at the same time, the danger, the excitement and the power that's going out in the field out there doesn't translate. So with technologies like this, we're able to show the truth of what's going on. We can use the data to help us tell the full, dramatic story what's happening out in the racetrack.
How do you please more than one visitor in the same experience? Is that visitor coming along to get a visual experience? Are they going to have a kind of public experience of some kind? Are they going to have a visceral experience? Is it going to be emotive? Who are you actually appealing to? Now it seems to me it has to be a global audience, given the scale. So how do you square that circle of having somebody down the street walk in, and somebody's logging in remotely from China, applying a very different cultural understanding to whatever experience you're giving them. The technology has to deliver a culturally variable message. Also, a great experience is not really about single experiences. Coming to a museum isa community experience. Putting on a headset optimizes that experience, but it also takes you out of that community, plus it makes you look like a dick (e.g. you might want to consider using mobiles, or glasses instead). But you see what I'm getting at, the experience is being dictated by the Technology, not necessarily the other way around. The curiosity should not be driven by whatever the technology happens to be.
Imagine we’ve got the Giant's Causeway. It's very boring. But actually, if you were trying to make a visitor experience out of that, you can ask a lot of questions about whether it is a geological site. Yes. Is it a contested site? Absolutely. Because the other folkloric story is that a giant called Finn McCool made this place. And if you think it was Finn McCool, that means that you're coming from a Catholic background, and that makes you a Republican. And if you're a unionist, the Giant's Causeway is where you realized that you are connected and not separated by water to your ancestors in Scotland. Contested space! So you've made a visitor experience, and three people have just come through the door. And they've all got different understandings of what that is. How do you represent those narratives? Because it's those narratives that are important. And more crucially, how do you represent those narratives to a Chinese person who has their own narratives? So if I was putting the visitor experience together, I would be thinking about a template of applied understandings and ideas which can move to anywhere in the globe. So you've created a template which you can then sell to South America or you sell to us Africa where they can tell their stories with their cultural resonances. But you've created a vehicle through which they can do that using the technology.
Constructing a mixed reality, sensation, experience, which allows people to bring their reality and construct their own narrative coming out of that experience space is quite a challenge. You've got your narratives, where do you set that into? You may create a landscape, a soundscape, a cityscape, but there has to be a context in which those experiences happen. And those narratives can communicated regardless of the technology, so don't fixate on the technology.
You want everyone to know the same piece of information and to be able to share an experience. To help people have their own private experience and also find out the shared base-line, does there need to be a group activity, or a walking tour together, or simply a way to share with families and friends? The answer depends upon who you're trying to reach and what you're trying to help them understand.
I teach museum personnel on a master's program, and it's my job to encourage museum workers to take on digital tools and they are absolutely resistant to that space because they believe passionately in the authenticity of the object. And that's the kind of problem you're up against How do you convince people that those technologies enhance the authenticity of the object?