Noise around AR, VR hits new peak

During a two-day event at SPIE Photonics West, leading experts from industry and academia gathered to talk about the direction of VR, AR, and its cousin technologies.

On Sunday, creators and critics of AR, and its cousins Virtual and Mixed Reality, sounded off on successes and misfires at Sunday's second annual Industry Days, one of the best-attended events of Photonics West.

As the AR/VR/MR 2019 speakers noted, Sunday's much-augmented Superbowl LIII was a timely case study. Its painted-on lines of scrimmage, first-down lines, replays and ever-present info boxes immersed millions in Augmented Reality.

"If you're a Superbowl fan, you came to right place," said Bernard Kress of Microsoft HoloLens, the event moderator. "We are making the Superbowl more fun, more immersive. We are going to provide the world with this new immersive display to allow millions of people to experience the Superbowl in ways they never have."

But sharp criticism came from Thad Starner, a professor at Georgia Tech, who argued that the industry is ignoring what users actually do, and like. For example, users don't want the widest field of view, don't want to play "head ping pong," moving left and right before a big screen. "Your eyes want to be watching within 30 degrees for the action."

For videos, he said users prefer the small, portable screen in their pocket, like a smart phone, to a giant-size home theater screen."Portability beats FOV." And users don't opt for 3D features. With a choice of 3D and 2D, Starner said, "Most users set the device to 2D mode. In this industry, that's heresy."



Headsets are too heavy on the nose to wear them more than a short time, he said. Starner wears his own all day, he said, because "I believe in wearing them." But he asked his audience of 400 or so, and no one else was wearing one. "Wear one all day every day for six weeks. See how usable it is," he challenged them.

Meanwhile, he said, there are cases for using heavy AR/VR headsets - fixing complex equipment, maintenance and repair work, and yes, entertainment. Otherwise, he said, design of many sets "is based on science fiction and aspiration, rather than achievable goals and real experiences."

The next big thing
In 2018, sales of smartphones and tablets declined. In his opening remarks on Monday, Kress asked the crowd with a metaphorical "What's the next big thing? You tell me." No one responded explicitly, but the implied answer was: virtual, augmented, and mixed reality.

Kress described where the progression of the technologies through the hype cycle, as determined by technology firm Gartner in August 2018. Mixed reality is currently coming down from a peak of inflated expectations, Kress said, pointing at the graph. AR is currently in the "trough of disillusionment"-and VR isn't on the graph at all, implying, perhaps, that the tech is mature. Kress recapped some of the big news in the industry in the past year, including new announcements from companies such as DreamWorld and DigiLens, as well as the shuttering of AR headset company Meta.

Tish Shute, of the Chinese company Huawei, said spatial computing - for example, in automobiles - was a key AR/VR ingredient, but it's too early to predict its future. "It's 9 a.m. in the age of light," she said, as we are moving from an electronic to a multi-user photonic society. "You change what It means to be human when you change the nature of communication. Personal drones can see and hear you. Your house becomes your health monitor. A dialogue without words is unfolding between driver and vehicle." The future of computing, she said, goes well beyond screens. She predicted that "the photonic city" will merge light, data, and communications.



Behnam Bastani of Oculus at Facebook described the two "pillars" of AR and VR as realism and comfort. In other words, the images must look good-realistic-and the devices must be comfortable to wear and complement natural human ability. This is no easy task. For example, engineers might try to make an image easier to comprehend by blurring background objects, so the viewer knows where to focus. But these blurring techniques can introduce artifacts in video.

Mark Bolas of Microsoft talked about the boundaries between perception and reality, and how humans use images reflexively to maintain their equanimity. He pointed out that human vision employs a variety of tricks, such as accommodating for depth of field both vertically and horizontally. For AR and VR to look natural, they should consider human biology and psychology, said Bolas.

Finding better ways to bend photons
Some speakers discussed technical advances in the field. Federico Capasso of Harvard University is developing metalens designs-flat, lightweight lenses that bend light because of tiny nano-sized structures, designed via computer simulation. Capasso's team has managed to design so-called achromatic lenses that focus all colors at the same point. They are also designing so-called hybrid metalenses, where they adapt commercially available optics to potentially lower costs.

Jim Melzer of Thales talked about how AR can be used to improve people's performance in high-intensity jobs, such as pilots, firefighters, and anesthesiologists: to use the tech to enhance their "situation awareness," he called it. For example, his team ponders how to design head-mounted devices that present relevant information. One important constraint to keep in mind, for example, is that if a person keeps their head stationary, their movements are limited to just 15 degrees.

Mary Lou Jepsen, the CEO of Openwater, discussed her company's vision for creating a lightweight, low-cost replacement for MRI based on infrared light for medical imaging. MRI, whose machines can cost in the millions, is cost-prohibitive in a lot of areas, she said. Their machine eschews magnets for infrared light and acoustic sensors to measure optical signals transmitting through the brain and body. The measurements produce holographic signals that encode image information.



The diverse crowd included attendees from a variety of industries and academia. One attendee worked in the semiconductor industry and wanted to learn how their machines might be re-appropriated for the mixed reality industry. Some attendees came from medical companies: "Our company develops products for low vision patients," said Taeho Ha of CMLab Corporation in Seoul, South Korea. Ha came to the session to learn about how AR and VR might be able to help such patients.

Across the way, industries brought AR and VR demos to show off their latest products. Magic Leap brought their headset. SyGloss, based in West Virginia, brought a headset demo targeting medical researchers and educators, in which the user could manipulate and virtually cut open a 3-D MRI scan. Leia, a company based in California, demonstrated an app that turns two images from a camera into four different views, rendering it into a three-dimensional-esque image that looks like it pops out of the screen. Gamma Scientific, based in California, brought a robotic eye for testing various devices.

Ultimately, it's important to keep developing the optics that are the basis of the technology, Bolas said. "Let's keep finding better ways to bend those photons," he said.

A version of this article appeared in the 2019 Photonics West Show Daily.