We collected every email and every tweet from the 2020 Democratic primary. Here's what we found...

How Does Virtual Reality Technology Work? (+Where It's Headed)

Andrej Kovačević
Andrej Kovačević  |  November 25, 2019

It might seem like virtual reality (VR) technology has only been around for a few short years.

However, the VR systems and headsets we know today have been under development for decades. The earliest progenitor of today's VR systems actually dates back to 1957 with a patent filed by Martin Heilig for a head-mounted stereoscopic television device.

In the years since, VR technology has been making slow but steady progress. At first, developers lacked the computing power to make a true, immersive VR experience. Then, once they had it, the race was on to make it portable and affordable for the average consumer.

That's where we are today. Companies like HTC, Oculus, Valve, and Sony now offer commercially viable VR hardware that's continuing to improve by leaps and bounds. For that reason, people all around the world are now familiar with VR and understand what it is. Most don't, however, have a firm grasp on the specifics of the technology.

A technical guide to virtual reality

To remedy that, here's a basic technical guide to virtual reality technology. You'll learn how it works, what it takes to make it work, and where the technology might go next. Let's dive in.

The scientific basics of virtual reality

At its core, VR technology has only one purpose: to simulate settings and environments realistically enough to fool the human brain into accepting them as reality. From a scientific standpoint, that all begins by understanding how our brains interpret the things we see to develop a mental picture of the world around us.

Without getting into too much detail, the simplest explanation is that our perception of reality is based on rules we develop using our experiences as a guide. For example, when we see the sky, it tells us which direction is "up". When we see objects we can identify, we can use their size relative to one another to judge distance. We can also detect light sources by picking up on the shadows cast by the objects around us.

VR designers can use those conventional rules to create virtual environments that conform to our mental expectations of reality. When they do, the result is a seamless experience that we interpret as "real".

The technical basics of virtual reality

Today's commercial VR systems are all competing to determine which can provide the best possible user experience in a virtual setting. In truth, none of them are capable of a completely immersive experience, for one very simple reason: the technology hasn't caught up with the capabilities of human vision – yet. Here's a breakdown of where today's VR headsets are, and where they're trying to reach.

Field of view

From a technical point of view, one of the biggest hurdles is the fact that humans are capable of a much wider field of view (FOV) than today's headsets can provide. An average human can see the environment around them in a roughly 200 to 220-degree arc around their head. Where the eyesight from our left and right eyes overlap there is a roughly 114-degree arc, where we can see in 3D.

Today's headsets focus their attention on that 114-degree 3D space to deliver their virtual environments. No headset, though, can yet accommodate the full FOV of the average human. Right now, though, today's VR hardware designers are aiming to create devices that will allow for a 180-degree FOV, which is considered ideal for a high-performance VR simulation.

Frame rate

In the world of VR, there is perhaps no greater topic of disagreement than over how to deal with the frame rates of virtual environments. That's because there's no real scientific consensus on how sensitive human vision is in that regard. From a physical standpoint, we know that human eyes can see up to the equivalent of 1000 frames per second (FPS). The human brain, however, never receives such detail via the optic nerve. There have been studies that have suggested that humans can discern frame rates up to 150 FPS, but beyond that, the information is lost in translation on the way to the brain.

For a movie you see in a theater, the frame rate is 24 FPS. That, however, isn't designed to simulate reality. For VR applications, most developers have found that anything less than 60 FPS tends to cause disorientation, headaches, and nausea in the user. For that reason, most developers aim for a VR content "sweet spot" of about 90 FPS and some (like Sony) won't certify software to run on their devices if they fall below 60 FPS at any point while in use. Going forward, though, most VR hardware developers are going to start pushing for a frame rate of 120 FPS or more, as that will provide a more true-to-life experience for most applications.

Sound effects

Another crucial technical aspect of VR is the way that designers use sound effects to convey a sense of three-dimensional space to the user. Today, cutting-edge VR relies on a technology called spatial audio to create a simulated audio landscape that matches the visuals created by VR.

Anyone who has ever sat in a well-designed concert hall should be familiar with how the sounds we hear can vary based on where we're located within a space and even which way we turn our heads. Spatial audio is a technique whereby VR designers can produce binaural (stereo) audio through a set of headphones that mimics that exact sensation.

There are a variety of current implementations, but they all share some similar characteristics, including: 

  • Controlling volume
  • Using left/right delay to convey direction
  • Using head tracking to map auditory space
  • Manipulating reverberation and echo to simulate environmental factors

It's also important to remember that for a VR headset, the audio effects described here must be computed in real-time to account for the movement of the user. When it comes to this, today's VR hardware is still just beginning to scratch the surface of what's possible.

Head and position tracking

The real magic of VR doesn't come from how convincing the visuals or sound are (although those are critical foundational elements), it comes from the fact that users can move within a virtual space that adjusts to their position. It's what separates a VR headset from a simple set of video viewing glasses.

Right now, there are two types of head and position tracking in use for VR applications – measured in degrees of freedom - 3DoF and 6DoF. Mobile VR headsets like the Samsung Gear VR, Google's Daydream View, and the Oculus Go use 3DoF, which means they are capable of rotational tracking only. They know when you turn your head left and right, look up or down, or tilt your head to one side or another. If you move your whole body, though, they won't pick that up.

Headsets that use 6DoF, by contrast, can track the wearer's position within the room, as well as the direction their head is pointed. That means 6DoF headsets can allow for full autonomous movement through a 3D space, which is a far more convincing VR experience. The way it's done varies from platform to platform, but major methods tend to include camera-based tracking in concert with infrared light beacons.

Where virtual reality is headed

As advanced as today's VR technology is, it's bound to get a whole lot better in the coming years. As developments continue, we should start to see hardware with an enhanced, more lifelike FOV, and better 3D audio to match. That alone makes the near-term future of VR exciting.

We're also on the cusp of seeing new improvements to VR that are going to make the experience vastly better than what you can get from today's hardware. One of those is the use of haptic feedback devices like the HaptX Gloves, which provide realistic touch sensations for the objects users interact with in VR. Another is a graphics technique known as foveated rendering, which takes advantage of the human eye's limited focal point to deliver ultra-high definition images only where our eyes are focused, thus lowering the computing power required to create the image.

What's more important, though, are the new ways that VR is likely to be used. Parallel advances in machine learning technology in the education field are going to make immersive distance learning a reality for the first time. Surgeons will benefit from advanced VR training to improve patient outcomes. Those in need of treatment for PTSD and related disorders will finally have a way to heal.

The bottom line here is that VR technology is only just beginning to realize its potential in a variety of fields. As the technology grows, so too will the applications that talented software developers, researchers, and business leaders dream up for it. From that standpoint, it's fair to say that we are much closer to the beginnings of the story of virtual reality than we are to the conclusion – and there's going to be a whole lot more amazing developments to come. 

Interested in learning more about existing virtual reality software and related technology? See all available options to take your knowledge to the next level – only on G2. 

Find the best Virtual Reality software on the market. Discover Now, Free →

Andrej Kovačević
Author

Andrej Kovačević

Andrej is a digital marketing expert, editor at TechLoot, and a contributing writer for a variety of other technology-focused online publications. He has covered the intersection of marketing and technology for several years and is pursuing an ongoing mission to share his expertise with business leaders and marketing professionals everywhere.