With the ever expanding world of VR taking hold in everyday life, it’s always interesting to understand how these technologies are working. The interesting thing about it all, while it might seem like space age technology, it is all built on technologies we’ve been using for decades, from using the TV remote to filming in the dark. Now today, I’ll be talking specifically about how all of these technologies track your position or movement. I will be discussing the more popular devices that are being used today, including headsets, and accessories.
So let’s start with the headsets, as they’re the more widely used currently today. Now there’s a pretty broad range of devices, including things just running off of your cell phone, and up to full warehouse scale systems, and it all just seems to work automagically, well I’m here to shed some light, though from a very high level, on how all of these magic techs work.
Head Mounted Displays
Google Cardboard & Gear VR
So you might think that both of these things are different, why am i grouping them together? Well both of these techs work in the same way. So they take advantage of technology that is in almost everyone phone today, but wasn’t created specifically for use in VR. I’m talking about the Gyroscope, and the accelerometers. So instead of knowing exactly where your phone is in the world, these techs keep track of the relative orientation of the device, so this is what is used to make your phone display in landscape or portrait, as it detects how the phone is rotated on each axis. Most of the movement you are able to do in let’s say a 360deg video when using Mobile VR, would be handled by the accelerometer. So instead of knowing the phone’s exact position, it will track the relative acceleration, or delta-velocity (Change in velocity over time, on any given axis), and this will change your rotational position as you turn your phone to look around.
Now the Oculus takes things one step further. Since the Mobile VR is based on VR as a second thought, Oculus, and its HMD(Head Mounted Display) has VR as its main mission; It’s a dedicated machine. So the Oculus HMD takes the technology used in phones, the gyroscope, accelerometer, and magnetometer to help track the positioning as motion as accurately as possible. Now we already know how these technologies are used for VR, but Oculus has something different, it has a tracking camera. Though this isn’t an ordinary camera, they use the same technology that would be found in your TV remote, and is radiating everywhere, all the time, but just can’t see it. I’m talking about Infrared Radiation (Which is completely safe). So the camera can see in IR(Infrared) and will also cast it, with special lights, but this isn’t made to see you, it’s made to look for the HMD on your head, but more specifically the Markers found on the HMD, as the positions of those specific markers, tells the software that processes all of the incoming information where your head is, in that space. So with the mobile VR, you were limited to one point in space, but could look around; with the Oculus you can look around, but now the position of your head can now be processed.
Now we have something more, the Vive is another dedicated head, but again it takes things yet another step forward. So the Vive is still using those same technologies of the Gyroscrope, accelerometer, magnetometer, as well as the IR Camera. So where’s the twist with the Vive, you might be wondering, well instead of just a single IR camera, the Vive uses two, and the same technology that Nasa uses to track their rockets when launching their astronauts. That technology is a consumer ready version of Cinetheodolites, which uses multiple cameras, and their static positions, to know exactly where an object is in 3D space, but instead of a rockets, it’s the HMD and the two controllers, all of which use markers, and the three positional instruments giving you the most accurate tracking possible. The bonus of all that extra hardware, is that now instead of sitting at a desk like the Oculus, the Vive allows you to walk around in a space, this is referred to as Room scale VR.
Now how can things get any better than these VR machines, well currently available and in development, there’s something referred to as Warehouse scale which takes the the technologies used for the HTC Vive and scales it up ten times, meaning a lot more walking space, and portable VR Units. With a large amount of cameras, the software can track a lot more Units, with more accuracy, and within a large area.
This is a hardware and software combo, that tracks hands and their movements, including each individual finger. So the hardware is a couple infrared emitters, and a camera. So unlike the HMDs mentioned before, there are no markers to refer to, or special instruments to know how much your hand has rotated, so how does it track your hands? The software processes the input from the camera within the leap, and creates bones, and models within that space for what the camera sees. Now this is limited by the view range of the camera, and what it physically sees, so the camera is best used for spread fingers with simple movements, meaning that it doesn’t understand fists very well. Another issue is that your hands have to be in front of the camera to be processed, so it can make things uncomfortable or drop you from the immersion.
Manus VR Gloves
This is another hand tracking solution available for VR, but is very different from the Leap Motion. While the leap motion is based around IR, the Manus Gloves just use simple motors. So the gloves do not currently have any built in positional tracking capabilities, but can work the HTC Vive, and their controllers to track the positions, but that makes things a little clunkier. Though the fact that there is not any camera means that the hands can be tracked anywhere, and since the gloves use motors to detect the finger movements, it supports a lot more complex gestures.