Imagine you’re having a good time playing Beat Saber and suddenly your saber flies away. Or imagine trying to line up a crit shot on Population: ONE and your scope starts to shake uncontrollably. Having your devices uncouple during VR games is something everyone despises. With the Meta Quest 3, a lot of people tend to track better now in bright rooms. This is directly related to the technology in the Quest 3s controllers along with the interior tracking.
I have spent many hours in virtual worlds since VR began selling to customers, and in all these years, the way tracking has progressed is fascinating. Moving from external base stations to the freedom brought by modern standalone headsets has been a journey. Now, the Touch Plus controllers, or as they are officially called, the Meta Quest 3 controllers, is a true embodiment of this evolution. In this guide, we will explore the mechanics of these controllers, the science of lighting, and in great detail, the reasons the Quest 3 is much better than its predecessors.
Purchasing the headset and its controllers will make more sense through understanding the chronology of VR tracking. The original HTC Vive and many other high-end VR systems during its time used “outside-in” tracking. This is done by setting up sensors, called “lighthouses”, in the four corners of a given space. This approach saw the lighthouses illuminating the space in infrared light, as the headset along with the controllers determined their position based on the sensors. Although this technology was extremely efficient, it came at a high price and was complex to set up since the user was confined to a certain area of movement.
With the first Quest, Meta (previously known as Oculus) pioneered ‘inside-out’ tracking. This changed the game entirely and flipped the system on its head. Instead of external devices tracking the system, the external cameras on the headsets track the controllers and the environment. This self-contained system offered the users immense freedom as they were now able to carry their VR experiences anywhere. The magic light, cameras, and intelligent software symphony nears perfection in the Quest 3 system.
How does the headset and the controllers communicate and what do I need to know to unpack the magic?
What seems as magic of seamless one to one movement in VR is a high speed conversation between your headset and controllers. This conversation is dependent on two technologies, infrared LEDs, and inertial measurement units (IMUs), which need to function in perfect synergy.
What users do not see, the infrared light of the LEDs, is crucial to the tracking system. If you were to look at your Quest 3s controllers through an infrared camera, they illuminate differently. The almost blank plastic shells contain a galaxy of small infrared (IR) LEDs. These LEDs are crucial to the functionality of the tracking systems as they pulse in rapid dashes and form patterns distinctive to every controller.
Picture it this way: the cameras on the Quest 3 headset are like stargazers. The IR LEDs on your controllers are the special constellations. The software on the headset is taught to identify the specific shapes of these constellations. The headset is able to determine your controllers’ position and rotation in 3D space by determining the position, scale, and orientation of these shapes in the field of view. This is the most accurate form of tracking, and it’s the most effective way to find your controllers in 3D space.
The Headset’s Eyes: The Cameras and Computer Vision
The Quest 3 headset has four front-facing cameras and additional side cameras that detect infrared light. These cameras are highly sensitive and specialized. They do not serve the same functions normal cameras do. They are intended to provide a crystal-clear view of the LEDs on the controllers.
The cameras are not the only things that work in this process. This is where sensor fusion is important. Each Touch Plus controller has what is called an Inertial Measurement Unit, or IMU, which contains an accelerometer and a gyroscope.
The accelerometer is able to detect linear movement, which is to say movement in a linear Direction (up/down, left/right, forward/back).
The gyroscope is able to measure rotational movement, which in this case is tilt, pan, and roll.
IMUs do a great job of figuring out fast movements, but these systems do tend to ‘drift’ over time, which isn’t good. The cameras, in contrast, do a great job of marking absolute positions, but their update rate is much slower. The real breakthrough of Quest 3 is combining both. The IMU delivers instantaneous data for fast movements, and the cameras provide constant drift correction in the real world to every controller position. These two are what make the tracking feel super fast and precise.
The Main Event: Enhancing Tracking Accuracy in Brightly Lit Rooms
We have reached the most important part of the discussion. The two reasons why a well-lit room improves the performance of Quest 3 controller are: first, the headset better tracks the room and second, the room improves the SNR for tracking the controllers.
Telling Signal from Noise
In order for inside-out tracking to function correctly, the headset must know its position in the room at all times. It uses a technique called SLAM (Simultaneous Localization and Mapping) for this. The headset’s cameras start scanning the room and mapping different static and unique feature points like the corner of a desk, a pattern on the rug, or a picture frame on the wall. It creates a 3D model to position itself in the room.
In a dark or poorly lit room, the cameras will have a difficult time identifying and locking onto these feature points. The surrounding area becomes a low-contrast, blurry soup, and the headset loses its ability to track its position. In these cases, the headset position gets lost, and the position of your controllers becomes a guess, often based on the last position they were lit. This usually coincides with the times you experience world-wobble or the time you feel like your controllers are floating away.
A room that is sufficiently bright and well beyond even lighting presents the cameras a rich high-contrast and simple identifiable points. Consequently, the headset is able to stably and robustly self-map. In turn, this enables solid tracking of the controllers relative to that position. The headset “hears” the controller’s IR LEDs as the “signal” and it is the ‘environmental noise’ the headset has to separate the headset from. While direct sunlight is a challenge, since the sun is a significant source of IR radiation and can overpower the controller LEDs, an ordinary and well-lit indoor area is the ultimate setting.
The Quest 2 Comparison: Why the Quest 3 Has an Edge
With every update made, it is clear that the improvements to the tracking performance of the Quest 3 come from more powerful hardware as opposed to the Quest 2. One of the most visually striking changes is the removal of the large tracking rings. On the Quest 2’s touch controllers, the rings contained IR LEDs and placed them away from the user’s hands. They served their purpose well, but also tended to be large and easily clashed.
The Quest 3s controllers have IR LEDs embedded into the front and back of each controller. While the headset’s Snapdragon XR2 Gen 2 chip now offers significantly more computation power, Meta’s engineers have developed advanced predictive algorithms to make up for the missing ring. Even more, an IR LED, which enhances the ability to aim down sights, was placed under the controller, just above the trigger/grip section. This tiny addition helps a lot with specific functions, like during aiming or bringing hands down close to the headset, or when the capture from the camera facing down gets blocked by hands. This, and the overall better camera sensors on the headset, helps a lot with the Quest 3 tracking consistency.
Beyond The Light: Other Aspects that Impact The Quest 3s Controllers Performance
Other than lighting, there are other elements in the environment that can affect the VR experience. Knowing these factors can help you design a better play space.
Reflective Surfaces and Their Tracking Pitfalls
The SLAM system does not handle reflective surfaces well. Big mirrors, glass tabletops, or glossy floors can create feature points that are either duplicate or distorted, which leads the headset to believe that it is moving even when it isn’t. This causes jittery headset and controller tracking. If tracking is jittery, mirrors can be covered or the user can move away from reflective surfaces.
The Importance of a Feature-Rich Environment
Like a dim room is detrimental to tracking, a room that is stark naked is equally bad. A featureless white wall has little to no points that the cameras can focus on. A room with a bit of furniture and varied textures is actually ideal. These help the tracking system to have a stable map of the user’s surroundings, resulting in a smoother experience during use.
The Impact Battery Life
Battery Life impacts and influences quite a few features, and the exact amount of charge remaining for each barrel can and does make a difference. Weaker IR LEDs and suppressed pulses being tracked by cameras becomes an issue. If after a long session, the headset detected and tracked the user less accurately, all that was needed was probably a newly charged battery.
Advances in VR Controller Technology
It’s safe to say that the Quest 3’s most unique features, such as advanced haptics and TruTouch, as well as sleek controller tracking, fully incorporate modern engineering. Innovations are however, still being developed. Having fingers tracked in real time and precisely mapped using a headset camera is still an illusion, but advancements in controller-less hand tracking make it possible. Although the precision is not yet comparable to handheld controllers in gaming, it represents a time in the future where hand gestures can replace controllers entirely.
Controllers in the future might attach their own cameras for self-tracking which would solve occlusion problems completely, or have highly advanced haptics systems that can recreate touch and resistance. There is still a long way to go, but the Touch Plus controllers for now have a wonderfully intuitive and reliable way to connect to the virtual worlds that we love to navigate and explore.
To sum things up, the Touch Plus controllers for the Quest 3 are a technological marvel, and their performance is not due to one element, rather a perfectly crafted balance of hardware and software. There is a powerful balance between the computer vision in the headset, the infrared LEDs in the controllers, and the internal sensors in the controllers that makes the experience immersive and reliable. Making sure that your play space has enough illumination will give this system the data it needs for optimal performance, so that the user can completely enjoy the experience without the intrusive thoughts of the technology used.
Questions and Answers
What exactly makes Quest 3s controllers so advanced as compared to Quest 2?
The difference is the design, technology, and how much the controllers can process. Touch Plus Quest 3 controllers remain smaller and more advanced since they do not have a tracking ring and the LEDs tracking the ring are built into the body of the rest of the controller. Also, they have some additional infrared LEDs that are put in pretty new places, like under the main body, which helps to get rid of tracking loss when the controllers are really close to the headset or in really weird positions. The controller is even more powerful due to the XR2 Gen 2 processor in the headset which uses more advanced tracking systems.
Are there any advantages to using my Quest 3 controllers in a dark room?
It may look like darkened rooms will result in increased visibility of the controllers’ infrared LEDs and other lights to the sensors, but it’s actually more detrimental to the performance tracking system as a whole. So bin the goggles. The tracking system in them has to deal with the controllers’ IR lights and the geometric contours of the mapped room. If there’s no light, the goggles are unable to see a reference room, and thus won’t configure something called SLAM. If the goggles the user is wearing cannot pinpoint a reference point, the chances of gauging the location of the controllers with any accuracy will drop, causing a massive drift and loss of radius tracking.
Why does my controller float away or get stuck in VR even in a well-lit room?
Most people call it controller drift or flyaway. It often results from something called occlusion. You get occlusion when the cameras on the headset lose sight of the controllers’ infrared LEDS. You lose sight of them when you put a controller behind your back, too close to your body, or when one of your hands blocks the other. While the cameras are blind to the controllers, the system revolves exclusively around the internal IMU sensors for a short duration. If occlusion persists for too long, it is the IMU drift that takes over and the controller slowly shifts to a virtual position that is way out of alignment. The system then waits for the controller to be in the cameras’ sight to re-capture and set its position.
What happens if I don’t clean the cameras on the Meta Quest 3 headset?
Of course you should. The rest of the “inside out” tracking system will depend on the clear and unobstructed views the small cameras on the front and side of your headset constantly have of the world and your controllers. Fingerprint smudges and other “light entering” diffusers like dust and hair can confuse, if not capture the attention, of the tracking algorithms. This confuses the ability of the system to see the IR lights on the controllers and/or the environmental features. You should clean the camera lenses to remove dirt or dust. Use a dry cleaner cloth without any large dust particles to guarantee the utmost tracking and camera performance.
Mehar Mozan