Mark Zuckerberg has so many prototypes of VR headsets to show us

Meta’s Reality Labs division has unveiled new prototypes in its roadmap to lightweight, hyper-realistic virtual reality graphics. The breakthroughs are far from ready for consumers, but the designs – codenamed Butterscotch, Starburst, Holocake 2 and Mirror Lake – could lead to a sleek, brightly lit headset that supports finer details than the current Quest 2 display.

Meta CEO Mark Zuckerberg and Reality Labs chief scientist Michael Abrash, along with other Reality Labs members, presented their work in a virtual roundtable last week. The event focused on designs that Meta calls “time machines,” voluminous proofs of concept intended for testing a specific function, such as a super-bright backlight or a super-high-resolution screen. “I think we are now in the midst of a big step forward towards realism,” Zuckerberg told reporters. “I don’t think it will be that long before we can shoot scenes with basically perfect fidelity.” Display technology isn’t the only unsolved piece of that puzzle, but it’s one area where Meta’s intensive VR hardware research gives it an edge.

Zuckerberg reiterated plans to ship a high-end headset codenamed Project Cambria in 2022, after its initial announcement last year. Cambria supports both full VR and mixed reality, thanks to high-resolution cameras that can relay a video feed to an internal screen. It also comes with eye tracking, a key feature for future Meta headsets. From there, Zuckerberg says Meta is planning two lines of VR headsets: one that will remain inexpensive and consumer-oriented, like today’s Quest 2, and one that will feature the company’s latest technology aimed at a “prosumer or professional” market. That matches reports that the company is already planning updates to Cambria and the Quest 2, although those prototypes were not discussed during the call.

Meta’s VR headsets sit alongside a separate line of smart augmented reality glasses, which aim to project images onto the real world rather than block it with a screen. Meta recently scaled back the launch of its first-generation AR glasses, and overall, VR displays have reached consumers much faster than AR holograms. But Meta’s prototypes show how far the company thinks it can go.

Butterscotch is an attempt at an almost retina-quality headset display — something you can find in high-end headsets from companies like Varjo, but not in the current Meta lineup. The design is “nowhere near shippable” and required the Meta Quest 2’s 110-degree field of view to be roughly halved. But it offers about 2.5 times the resolution of the Quest 2’s (kind of) 1832 x 1920 pixels per eye, allowing users to read the 20/20 vision line on an eye chart. Zuckerberg says it offers about 55 pixels per degree of field of view, slightly less than Meta’s 60-pixel-per-degree retina standard and slightly lower than Varjo’s 64 pixels per degree†

Starburst is even less shippable than Butterscotch, but tests an equally impressive upgrade. The bulky design uses a powerful lamp – which requires handles to support its weight – and produces High Dynamic Range (HDR) lighting with 20,000 nits of brightness. “This is hugely impractical to think of as a first-generation product direction, but we’re using it as a test bed for further research and studies,” Zuckerberg says. “The goal of all this work is to help us identify which technical paths will allow us to make meaningful enough improvements that we can approach visual realism.”

Holocake 2 moves in the opposite direction, exploring Meta’s options to make VR headsets thinner and lighter. It succeeds a 2020 design based on holographic optics, a light-bending technique that allows a nearly flat panel to replace a thick refractive lens. The result could be as thin as sunglasses, but Meta is still developing a self-contained light source that would power them — almost certainly a laser, not the OLEDs commonly used today. “We’re going to have to do a lot of engineering to get a consumer-viable laser that meets our specifications—it’s safe, cheap, and efficient, and it fits into a sleek VR headset,” says Zuckerberg. “Frankly, to this day the jury is still looking for a suitable laser source.”

The presentation also discussed Half Dome, a long-running series of prototypes which can shift the focal planes depending on what users are looking at. These varifocal optics started out in 2017 as a clunky mechanical system and later transitioned to a range of liquid crystal lenses, and according to internal Meta research, they can create a more convincing (and physically comfortable) illusion of depth in VR.

Meta described Half Dome’s technology in 2020 as “almost ready for prime time,” but today Zuckerberg was more measured. “This stuff is pretty far out,” he said in response to a question about the “prime time” comment. “We’re working on it. We really want to get it in one of the upcoming headsets. I’m confident we will someday, but I’m not going to announce anything in advance today.”

Reality Labs will be discussing more research at the SIGGRAPH trade show in August, including how to more accurately capture real-world images for mixed reality.

The designs above exist as actual hardware that Zuckerberg briefly showed off at the event. But Meta also unveiled a prototype called Mirror Lake, which is essentially ambitious and was never built. The design looks more like ski goggles than Meta’s bulky Quest hardware, and it’s said to incorporate Holocake 2’s thin optics, Starburst’s HDR capabilities, and Butterscotch’s resolution. “It shows what a complete next-gen display system could look like,” Abrash says.

On top of these features, Mirror Lake would include an outward-facing screen that projects an image of the user’s eyes, reducing the sense of physical separation for people outside the headset. Meta showed this slightly creepy feature in a prototype last year, and it may not be the only company interested in the concept: Apple has reportedly been considering a similar feature. for his rumored headset† The idea is tailor-made for a mixed reality world where Meta has staked much of its future — but today, the company is highlighting the incremental steps along the way.