No matter how much experience you have with video games you probably have heard of the health bar. It is the red bar in most video games in a top corner of the screen that indicates how much damage your character can take before you have to use another life. Visual cues like health bars, player scores, collected coins, and really any relevant information, collectively comprise what’s known as the Heads-Up Display (HUD). While these UI (user interface) systems were developed to give players a competitive incentive to play, the recent focus for game designers has been moving towards a more immersive system. First-person video games primarily are concerned with immersion and usability. We can look to these systems for inspiration in designing UI systems for XR (extended reality) solutions.
In order to borrow from video game strategies, it’s best to understand the history, jargon, and evolution of design styles and methods. HUD was originally named in a master’s thesis, Beyond The HUD: User Interfaces for Increased Player Immersion in FPS Games by Erik Fagerholt and Magnus Lorentzon. The thesis explained the most commonly applied methods fall into four flavors: Non-Diegetic, Diegetic, Spatial, and Meta UI. Diegetic interacts with the objects in the game world to signal communication. A common example is when the character has to interact with an in-game object to see information, like a watch on their wrist, or a GPS device. The Non-Diegetic manifests as bars or text on screen to display information but does not interact with the game world — the health bar on most classic games is a good example. Spatial UI exists in the game world but is not something that is directly related to the game’s story or interactions, for example, a piece of text on the side of the building that tells the player the objective, but the in-game character doesn’t notice. Meta UI, on the other hand, breaks the visual barrier and creates a sub-UI for the player to navigate easily, like a menu that spawns to show a character’s inventory.
Nicole Winestock, a product designer and writer for the UX collective explains how the Doom VR game uses the main character’s helmet to take the classic non-diegetic UI information, like health bars, and make it feel immersive as if you’re seeing it in his helmet. “While in old-school non-VR video games there isn’t truly a ‘right or wrong way,’” she said. “I do strongly recommend using a diegetic UI when designing for VR to increase the immersion for the player”.
Mike Alger, a senior UX designer for Google, explains that designing VR UI is like developing a new operating system, and to reach full maturity may take up to 15 years. “For it to feel natural, you shouldn’t have to teach someone how to use it, like a straw,” Alger said in a manifesto video about his thesis. He breaks down the ideas of Diegetic UI and explains how he is experimenting with a virtual watch UI system that would allow the user to both see the website they are on, navigate dialog boxes in VR by looking at their wrist and using their natural habits of looking at the watch for information.
“As VR grows it will have to build on other systems that we see already work, and web designers will become experience designers, much like interior designers or architects.”
We are seeing VR, AR, and XR take what has been learned from video games to make more immersive UI. With new ideas for how we apply these methods in VR emerging every day, an intuitive system will evolve as a standard propelling XR into the mainstream.
