VR headset company Fove is betting on eye tracking to compete |Techilink

SAN FRANSCISO — At a Kickstarter launch party at a swanky downtown hotel, employees and friends of year-old company Fove milled about, ready to talk to everyone about their contributions to a new virtual reality headset. VR headsets are old news at this point: Oculus Rift, Samsung Gear VR, Sony’s Project Morpheus have all made it through the press gamut a few times. But Fove aims to outshine traditional players by coming to the starting line with something none of those incumbents have (at least until now): an eye-tracking system.

Fove says the eye-tracking system will eventually enable foveated rendering – an advanced way to reduce the processing demands of VR headsets by generating a high-resolution image only for the immediate area a player is looking at, eliminating peripheral areas can be viewed. are displayed with less definition.

Fove just reached its $250,000 Kickstarter goal, which it will use to produce an SDK headset with a 5.8-inch screen with a resolution of 2560 x 1440 and a weight of 0.8 pounds. What sets it apart, however, are the infrared sensors that bounce IR light off the user’s retina to measure the distance between the eyes and the direction they’re pointing. Kickstarter backers have been able to secure development headsets for between $300 and $400, with Fove aiming to ship in Spring 2016. The development platform will integrate content from Unity, Unreal Engine and eventually Cryengine.

Ars got a chance to try out a demo version of the headset, which featured eye-tracking capabilities but not foveated rendering, which Fove says is still in development. The housing of the headset is sleek and on my head it felt like an Oculus Rift DK2. It was a bit heavy, but not to the point of discomfort. Once on, the team at Fove let me quickly calibrate the headset by looking at a succession of green dots projected at random points in front of me.

From there, the Unity-based demo was much like the one Ars experienced with German eye-tracking company SMI at the Augmented Reality Expo last year. However, the difference between SMI and Fove is that SMI sells its eye-tracking technology to headset manufacturers and Fove aims to sell an eye-tracking headset directly to consumers.

I got to play a minute or two of a little first-person shooter where you aim your eyes at incoming triangle grenades and then pull the trigger on a controller to detonate them. The experience felt seamless and natural to me. I didn’t have to get used to the feel of a thumbstick to move, and there seemed to be only a slight delay between looking at a target and being able to shoot.

For the show, Fove’s team then turned off eye tracking and head tracking – after 30 seconds of trying to play the same game in a more static world, I felt nauseous and dizzy. (By comparison, Oculus Rift has had head position tracking since DK2, but eye tracking is an added bonus in Fove.)

A daring plan

It’s risky trying to enter a hardware market with a number of incumbents already on their way to producing a market-ready consumer version of a similar product.

Still, Fove’s founders, CEO Yuka Kojima and CTO Lochlainn Wilson, are no strangers to the video game industry. Kojima was a game producer at Sony Computer Entertainment in Japan before founding Fove. Wilson previously worked with an imaging start-up in Australia. The two met at the annual Game Developers Conference in San Francisco and remained friends. When Kojima presented her idea to Wilson, he joined and later recruited two of his friends to help Fove build his product in Japan.

Since then, the company has had support and mentoring from Microsoft Ventures Accelerator London and won a spot in the Rothenberg Ventures River VR Accelerator, which earned a $100,000 prize to promote their product. The company has also received support from the University of Tokyo and says Toshiba and Samsung will provide components.

Clearly, Fove’s value proposition is not only in eye tracking, but also in bringing foveated rendering to virtual reality game developers. That feature is still relatively new and untested in video games and virtual reality, but it has a lot of potential for letting game developers shrink the graphics processing power they need by overlaying different levels of resolution in a game world.

Fove’s press materials demonstrate a concept similar to the process outlined by Microsoft’s labs in a 2012 research paper (PDF). In this process, the highest resolution image covers only the small distance from the screen that directly touches the player’s fovea – usually only a few degrees. A concentric circle around that area has a lower resolution, and the periphery outside that circle has an even lower resolution. Despite the degradation of the image as a whole, the human eye won’t tell the difference as long as eye tracking can keep the highest resolution area in front of your retina at all times.

There may be a catch to this attempt to reduce the technical requirements for VR hardware. Oliver Kreylos, a VR researcher at UC Davis, explained to Ars in an email that foveated rendering has to be very good or it could hurt the user experience:

“The problem right now is that foveated rendering only pays off when screens have a very high pixel count, and eye tracking is very accurate and has very low latency. It’s very off-putting for the viewer if her eyes ever look directly at a portion look It’s similar to the common problem of texture popping up in 3D games where players suddenly find themselves looking sharply at blurry, low-resolution textures. If eye tracking isn’t good or fast enough, resolution display area needs to be expanded to cover all possible spots where the fovea power are in the next frame, and because of the significant overhead of foveated rendering, there may not be any performance gains (or even losses) as a result.

At this early stage, it’s unclear how easy that problem will be for Fove to solve, but experts in the field seem to think that working to create hardware, software, and a developer base that can take advantage of eye-tracking could be an step in the right direction. right direction.

Dr. Albert “Skip” Rizzo, a director of Medical Virtual Reality at USC’s Institute for Creative Technologies, recently attended a virtual reality conference where Fove made quite the appearance. “Unfortunately, there was a long queue in front of the Fove, but I’ve heard from people that it’s quite compelling,” he told Ars.

Since the 1990s, Rizzo has been working to bring virtual reality into therapy and help patients with PTSD and brain injury by immersing them in therapeutic simulations. Fove has already tested its products for similar use cases: Long before launching its Kickstarter, the company took its prototype into therapeutic situations, helped students at the University of Tsukuba’s School for the Physically Handicapped play the piano with their eyes, and showed a paralyzed man to pilot a robot that would allow him to communicate with his family.

Rizzo says that even as an adamant proponent of head-mounted displays in therapeutic settings, he’d like to see a clearly defined value proposition before predicting that Fove will be revolutionary in the field. “I think you need to be very clear about what eye-tracking can add to the mix here. My first question is do you need eye-tracking in a head-mounted display to play the piano?” he asked, adding that eye-tracking has been around for a long time and has helped patients who need it without the added expense of a head-mounted display. block out the ambient light,” Rizzo suggested.

A personal project

Kojima told Ars that Fove’s eye tracking will have multiple uses, but that her personal goal in creating the headset was to make interactions with characters more emotionally realistic. Gamers like themselves are “not satisfied with [the] current virtual reality system,” Kojima said, adding that eye contact “makes virtual reality friendlier.”

“I wanted to interact with characters with nonverbal communication, I wanted to dive into the world with more sensitive contact,” Kojima told me at Fove’s launch party last week. She turned to a demo of a game with a character looking at us. When Fove’s platform is complete, looking into the character’s eyes will elicit a smile. Likewise, “when we look at her breasts, she will be angry, it’s more humane,” Kojima laughed.

For Wilson, the project was an opportunity to make VR headsets more competitive in real-world gaming. “We don’t believe current virtual reality is competitive,” Wilson said at Fove’s launch party. “We’d kick ourselves if we played against someone with a keyboard and mouse.” Eye tracking, Wilson believes, makes reaction time in a head-mounted display faster and makes the player better.

Whatever the reason, more than 650 other Kickstarter backers have decided that Fove’s vision is the vision they want to see in virtual reality next year. And VR is so new that underdogs can become major players in that short period of time. Like dr. Rizzo noted, “There’s been an explosion of interest in head-mounted displays. But three years ago it was still a failed technology.”