SPATIAL INTERFACE DESIGN
SPATIAL INTERFACE DESIGN: AUGMENTED REALITY AND NEUROSCIENCE
Augmented reality (AR), creators can use the space around participants to visualize media, enable participants to organise those aspects with their own, and share it all with peers face-to-face. This is a Spatial Interface.
META an augmented reality glasses company researched SPATIAL INTERFACE DESIGN GUIDES in collaboration with a diverse team of neuroscientists, led by Professor Stefano Baldassi (previously of Stanford University), UX designers and developers, advisers from industry and academia, and Meron Gribetz (founder and CEO of Meta), studying the way that the brain understands natural 3D interfaces.
Guideline 1
Think Spatial: Place Tools and Content in Space
Replace flat layouts of windows, menus, and buttons, with a Spatial Interface that arranges tools and content in 3D space around the user.
Do
Arrange volumetric tools and content in space.
Don’t
Cram traditional GUI elements (windows, icons, menus, etc.) into 2D panels.
UI Design Suggestions
1. Reconsider traditional windows and screen-based conventions
For example, don’t place a “start menu” in space, since it was designed for a small screen and presents a cluttered mess of buttons, abstract icons, and tools.
2. Separate elements into two buckets: volume and content
Text and video are inherently flat, and should be contained within tangible 3D structures with their own sense of affordance. 3D models, on the other hand, may exist as free-standing objects of their own and their design should suggest their use.
3. Distinguishing between tools and content
Content is an experience in and of itself; it conveys information or some kind of sensory experience. Tools, on the other hand, serve the purpose of creating, modifying or in some way interacting with content. The "three-dimensional tool" needs to be based on the user's intuition, and the "content" may have an abstract and flat shape, or it may have a realistic and three-dimensional shape. For example, text and video are flat, but need to be embedded in some three-dimensional structure. On the other hand, since the 3D model is originally three-dimensional, it can exist as an independent object.
4. Avoid expandable or hidden menus in AR
Instead of imitating the flat world of traditional traditional UIs into a three-dimensional world as they are, it is necessary to get inspiration from actual work sites such as art studios and workshops. Instead of thinking about menus and buttons, you should think of tools that the user can actually grasp.
Guideline 2
Minimize Abstractions: Design Tools with Volume and Affordances
Now that we understand how to arrange objects in space, how should we design the objects themselves? Replace abstract representations (like flat icons) with volumetric tools featuring physical characteristics that suggest their use without explicit instruction, leveraging the neuroscience of affordance.
Do
Use realistic volumetric designs for tools and content, creating intuitive affordances for the user.
Don’t
Represent objects with abstractions such as iconography
UI Design Suggestions
1. Design with affordances in mind
Design tools with affordances, which are physical characteristics that suggest how the tool should be grasped or used. For example, a holographic eraser might have grooves to invite grasping it on one side, and a flat surface for erasing on the other side. Objects and tools designed with affordance in mind are better than abstract icons. Can be understood quickly and deeply.
2. Build on the user’s prior knowledge of a tool instead of defining your own
Design based on prior knowledge, to enhance usability. For example, don’t reinvent the paintbrush in some radically new way.
3.Use tools only if direct hand manipulation is insufficient or biomechanically challenging
Direct hand interaction is preferable for most simple tasks, such as moving, rotating, or scaling an object.
4. Avoid use of buttons in AR as much as possible
For instance, rather than presenting a “send” button when the user writes an email, provide a mailbox that the user can drop the email into.
5. Compensate for lack of haptics with other sensory cues
For instance, consider synchronized audio cues when the user’s hand interacts with tools and content, or build hybrid tools that mix digital interfaces with real-world objects, or use physical markers that permit holographic drawing in space.
Guideline 3
You are the OS: Organize Holographic Files and Tools in the User’s Environment
Replace the traditional abstract and complex hierachy of file systems with spatial organization of files that makes sense for people, it makes them easier to engage with.
Do
Use the physical environment to create organization and structure.
Don’t
Use the abstract, nested file systems of traditional UIs.
UI Design Suggestions
1 Use space as an organizational tool
For example, if the user places a holographic object in a specific area of a physical desk, their spatial memory of its placement makes it easy and faster to search later.
2 Use volumetric containers and holographic furniture
Also, the object should use a three-dimensional container rather than an abstract folder or file system. Unless these containers have a deep hierarchical structure, files can be easily retrieved ergonomically.
3 Don’t overload the user’s working memory with nesting
These containers should not be nested. Nesting overloads the user's spatial memory and makes the file unintuitive. If the application does need nesting, then do not exceed one level. Despite being commonplace in traditional UIs, a holographic drawer should not also feature hidden “sub-drawers” within it.
4. Preserve the user’s spatial memory
When the user is selecting an object, other unselected objects should not be hidden Yes. This is because hiding impairs the integrity of the user's spatial memory. Highlight the selected object instead of hiding it.
5 Miniaturize content and tools to optimize space
On the other hand, miniaturizing content and tools (or vice versa) is effective in maximizing the user's workspace without disrupting spatial memory.
For example, family photos are routinely kept by the thousands in labeled albums, which can then be organized on shelves. This does not represent a violation of those principles because the photo albums are already exposed, and once opened, the user immediately arrives at the content they’re looking for—hence one level of hierarchy deep. This use of furniture, simple grouping, and shallow nesting provides AR developers and designers with an effective model for organizing huge amounts of holographic content without the cognitive overhead of abstract file systems.
Guideline 4
Touch to See: Use the Hands to Interact Directly
The brain naturally tends to recognize objects near the hand. The tools and the content they operate on should not be physically or spatially separated. That is, the user should not operate the content from a distance or mediate abstract objects such as gestures, buttons, and menus.
Do
Use direct hand manipulation to use tools and interact with content.
Don’t
Use remote gestures and abstract pointers.
UI Design Suggestions
1. Touch content to act on it
Tools, actions, and the content on which they operate should never be physically or spatially separated.
2. When grabbing¨ pushing¨ or expanding
It is necessary for the object to reflect the user's physical movements and movements such as grasping and spreading. It should not be associated with gestures that the user must first learn the action and function of, such as opening a fist or waving to spread an object.
3. Avoid gestures that require specialized knowledge or memorization
While power user shortcuts are enticing ways to speed up interaction with an interface, they leave behind most of humanity (the non-power users). Instead of discrete gestures, use affordances to encourage the user to interact with objects in more natural ways.
4. Persistence and the usage of tools
Tools intended for persistent use (such as a paint brush, a conductor’s wand, or a flashlight) should realistically respond to the user’s manipulation, instead of requiring a discreet hand gesture to activate or deactivate. For instance, the brush should draw when pressed against a surface, rather than in response to a separate trigger like a gesture or button press.
5. Proximity feedback is your friend
Provide proximity feedback when the user uses or touches an object or content. By doing this, it is possible to compensate for the lack of tactile sensation and emphasize collisions and movements. Glows and subtle audio cues are a good way to accent a collision or movement while compensating for the lack of haptics.
Guideline 5
Do Not Disturb: Do not interrupt the User’s Workflow
Spatial computing demands spatial notifications. Rather than interrupting the user’s workflow with pop-ups, allow users to designate a separate container to passively collect incoming notifications. This allows the user to remain focused on their task, only stopping to check for updates when desired.
Do
Establish a passive notification receptacle: non-intrusive, segregated from the task at hand by default, and easy to move.
Don’t
Interrupt the user’s task flow with active notifications and pop-ups.
UI Design Suggestions
1Separate designated areas
Allow the user to designate a container or space to collect incoming notifications away from their workspace, instead of interrupting them directly with pop-ups or alerts. This preserves the user’s control over their experience by allowing them to focus on the task at hand, only breaking their concentration for updates when desired. Ideally, such a box would be located outside the user’s field of view and at some reasonable distance, ensuring disruptions are avoided. As a default, prioritise the user’s concentration, only allowing disruptive notifications when requested explicitly by the user.
2. Design tools for specific purposes
Certain tasks require bottom-up disruptions to break top-down flow (like when your timer beeps), and should be incorporated into tools designated for that specific purpose, but even these tools should mimic real world tools like alarm clocks and egg timers. Most importantly, the user should always understand how to turn off the sounds and visuals associated with alerts.
3 Allow for tailoring and flexibility
Create flexibility by allowing the user or developer to designate multiple boxes, and tailor them to receive certain kinds of notifications. For instance, one box might be associated with all social media apps from different services or people, while another might be for email only. A third box might handle all incoming phone calls, making it easy for the user to selectively allow calls to disrupt their work (by bringing this particular box closer to the workspace). Boxes can also be connected to specific people or groups, allowing for further organization of incoming notifications. Furthermore, the design of the box itself should use affordances to suggest its use, such as a lid that can be closed to mute notifications, or opened to enable them.
4 Consider gradual notifications over abrupt ones
Even if users choose to place notifications within their workspace, they should consider using gradual visual and audio cues instead of explosive alerts.
Guideline 6
Avoid Surprises and Magic Tricks: Pair Actions with Intuitive Outcomes
Avoid “magical” events that appear unrelated to user behavior or earlier causes, or events that ignore the laws of physics as these only confound the user.
Do
Create predictable connections between a user’s actions and the result.
Don’t
Use nonsensical, disorienting quirks and surprises.
UI Design Suggestions
1Avoid “magic” by tailoring outcomes to match the user’s actions
Here, “Magic ” refers to anything that confuses the intuitive sense of cause and effect. A magic wand, for example, is confusing because it looks more like a stick and there is no affordance associated with a particular task. Also, people can't predict the consequences of using it. On the other hand, most users will immediately know how to use an eraser; its use can be understood and its effects predicted with no additional explanation.
2. Use realistic physics to create intuitive object interactions and behavior
It also confuses the user when the object does not respond in a reasonable way.
For example, even if you gently push a holographic ice cube and slide it on a desk, do not fly across the room at 900 m / h. It is necessary to have a clear relationship between actions and effects.
3 Enforce causality with clear associations between action and effect
A sound triggered by a user’s action should occur immediately in real-time and emanate from the appropriate location. Likewise, visual feedback such as glowing highlights, shadows, or physical deformations should be proportionate and spatially close to the event that triggered them.
Guideline 7
The Holographic Campfire: Don’t Obscure Hands and Faces with the UI
Our ancestors have evolved to face each other, both in pairs and in tribes. Therefore, we are very sensitive to the gaze of the eyes and the facial expressions of others, and the dedicated areas of the brain continuously perceive those sensations.
Our perception of the line of sight extends to its surroundings and is immediately noticed when someone else is looking at something of interest.
Avoid blocking eye contact, create shared spaces that encourage a mutual gaze, and ensure hands are visible while collaborating.
Do
Replace UIs that disrupt eye contact with a shared space. Also, collaborate by directly manipulating shared content.
Don’t
Obscure or occlude the face of others with interface elements, or use single modality collaboration, such as face-to-face collaboration without the hands or voice chat without video.
UI Design Suggestions
1. Promote unobstructed views
Avoid interfaces that occupy the entire field of view and obscure facial expressions—two significant components of personal communication. Within a shared physical space, interfaces should be arranged around users, not between them.
2. Support collaborative experiences when working in the same location
For example, a collaborative design tool should encourage all participants to manipulate the same object in the same place, as opposed to private instances visible to each individual separately.
3. Show hands when collaborating
The interface must clearly indicate the position of the user's hands. When a user's hands are within the sight of other users, the ability to learn from collaboration and demonstration increases.
4.When collaborating remotely¨ use 3D video of participants instead of avatars or virtual characters
Artificial expressions such as still images, icons, and virtual characters only reduce the accuracy of communication and improve cognitive burden and social anxiety.
Guideline 8
Public by Default: Shared Understanding Reduces Anxiety Among Users
We have evolved to constantly observe the attitudes, movements, and behaviors of those around us in order to attempt the intentions of others, and to guess the minds of others. By default, other people's content should be viewable. All users should be able to see the same holographic environment, just as they see the same physical environment in the real world.
Do
Make tools and content public by default. Maintain the ability to make
things private if needed.
Don’t
Offer private, asymmetric UIs (like the ones seen in Google Glass).
UI Design Suggestions
1. By default¨ ensure content can be viewed by anyone else wearing a headset
All users should be exposed to the same holographic surroundings, just like we all see the same physical surroundings in the real world. When a user's content is hidden from other users in AR, the action appears to be in an empty space . To confuse. Furthermore, devices like Google Glass have demonstrated our distrust of private interfaces because they allow aggressive or invasive actions that can be performed secretly, even when face-to-face. Such practices may divide users, rather than encourage cooperation, and may contribute to social anxiety.
2. When privacy is necessary¨ use a blocking object like curtains or dividers
Do not simply make sensitive content invisible from the perspective of unauthorized users, as this creates a disparity between users that can lead to confusion and distrust. Instead, just as in real life, establish privacy with holographic curtains or dividers. This approach ensures a consistent experience across all users without confusion or mixed signals, and eliminates the disorienting sight of gestures performed on empty space.
3. Privacy is a function of etiquette¨ not a feature of the interface
Augmented reality is most powerful when treated as a single, shared space. Don’t relegate privacy to an on/off switch that segregates user experiences. Confusion arises when this otherwise collaborative experience is unknowingly changed from one user to another for the sake of privacy or discretion. For example, a user who wishes to view financially sensitive information should do so within an interface that has privacy features, such as a curtain, divider, or display that can be tilted away from unwanted viewers. Users should design their own privacy within the rules of the shared space, as opposed to breaking the rules.
Guideline 9
Augmented, Not Mixed, Reality: Enhance the User’s Perception with Relevant Information
Rather than block out users’ reality (as in virtual reality) or distort the user’s reality (as in mixed reality), provide access to metadata about the world placed near by, without masking it—add something useful to people's understanding of their environment. Create an informative, powerful, and unobtrusive layer of digital information on top of the real
world.
Do
Present information and tools that reflect the user’s reality and deepen their understanding of it.
Don’t
Block or distort reality, whether fully or partially.
UI Design Suggestions
1. Complement the real world
When images and sentences added in the user's field of view by the AR (Augmented Reality) interface are added to the real world without discomfort, the user can understand them intuitively and effectively. Design interfaces to complement real-world objects, instead of masking or transforming them. For instance, a panel with a Wikipedia page about a flower placed near, but not occluding, the flower would be considered augmented reality. Consider a user that is able to touch a flower and learn about the amount of DNA s/he shares with it. This simplistic example demonstrates how one can enhance their connection with their surrounding environment.
2. The experience of turning the gym floor back to the surface of the pool, in contrast, causes unnecessary confusion and misleads the user about how it works.
This is because it changes or distorts the physical properties of the actual object.
3. Add value and insight for the user
Rich content should be displayed to provide information with as much clarity and depth as possible. Text, video, volumetric content, and the Web can all play a role in empowering the user with deeper understanding and connection to their world.
4 A note about use cases in gaming, In games, we recommend allowing virtual characters to pass through the actual user's environment, rather than "breaking walls" or covering them with overly distorted visuals. On the other hand, for certain uses, such as interior decoration, momentary changes in the environment are tolerated. This use case leads to changes and improvements in the physical properties of the real world.
The report is further summarised in these two blog posts:
For the full report, together with example images and related neuroscience references, see:
https://pages.metavision.com/meta-augmented-reality-design-guidelines/