Tobii and Ubisoft developers discuss how eye tracking works Villain.
Most of today’s press release announcing a March 10 release for the PC port of Assassin’s Creed Rogue is strictly standard. Then you get to the last paragraph and you read that “the Assassin’s Creed Rogue The Kiev PC development team has partnered with Tobii Tech to integrate eye-tracking input as part of the gameplay.” Wait, what?
Fortunately, in their own press release, the folks at Tobii go much deeper into what they call an “infinite screen” experience in Villain. For example, when a player looks to the left side of the screen, an eye tracker can measure that gaze at 50 frames per second and feed it back to the game. That will make the game’s protagonist look to the left and the camera will automatically pan to show what he’s looking at.
A Tobii illustration showing how eye tracking works.
You can still use the traditional mouse look at the same time, but the idea seems to be that you won’t want to do that once you’ve experienced what Tobii calls “the next evolution of human interfaces in gaming.” A short video from the developers demonstrates how the technology will work and shows how the game reacts as the player’s “viewpoint” moves across the screen. “The screen automatically centers around what you’re looking at, essentially giving you an infinite screen where your gaze always determines what’s shown on the screen,” says Anders Olsson, VP of Tobii Tech Software Partners in the video.
As if that weren’t enough, eye-tracking users will find that the game automatically pauses when they look away from the screen and resumes when they look back. Sounds perfect for gamers who find it too hard to reach for the escape key every time they want to look up at a big bang theory replay on the nearby TV.
A $200 chicken-and-egg problem
You’ll need special hardware to activate these features, of course: most notably the $200 Steelseries Sentry Eye Tracker, the first consumer device to integrate Tobii’s technology. Released late last year, the Sentry is currently being marketed as a way for game streamers to virtually “point” viewers to elements on the screen, while retaining full control of the game using the mouse and keyboard. It is also marketed as a tool for professional gamers who want to analyze and improve their performance by seeing what parts of the screen they are looking at, and for how long, during key moments.
Those niche applications are fine for what they are, but Tobii seems particularly excited about the prospect of major, big-budget studio releases increasingly using eye tracking as a new control scheme. “There is strong indirect business value for Tobii as each new application, such as this AAA game integration, will increase the usability of the company’s eye-tracking platforms, driving long-term sales,” says the company in its press release. To support that vision, the next 5,000 Steelseries Sentry units sold will include a bundled copy of Assassin’s Creed Rogue free.
But Tobii sees eye tracking as useful for much more than just camera control. The company suggests that trackers could be used for “multi-dimensional movement,” where a player, for example, shoots the mouse in one direction and throws a grenade in another direction with their eyes. In-game characters can react to eye contact “just like they would in real life,” suggests Tobii, or parts of the game environment can react when a player looks at them. Developers looking to pursue these kinds of ideas and get into what’s still pretty close to ground level can invest in a $139 EyeX development kit to add support to their projects.
At CES 2013, we tested Tobii’s eye-tracking prototype on a Asteroidsstyle game.
While you have Villain since an AAA proof of concept is helpful, it may not be enough to convince gamers to just buy a $200 unproven eye controller. Tobii’s Eye Arcade website lists four other indie games that currently support the eye tracker, including the Asteroid-shooting demo we tested at CES 2013. of the eyes can be a lot like mind control, if done right.
We look forward to seeing how Tobii’s completed eye-tracking technology works in a major studio release when Villain hits next month. Still, we have to wonder if Gear VR-like virtual reality or HoloLens-like augmented reality, more than flat monitors, represent the real future of controlling games with your eyes. But again, these are not mutually exclusive ideas; firms like SensoMotoric Instruments are exploring how eye tracking can enhance the virtual reality experience by improving focus and reducing motion sickness.
It’s definitely something keep an eye on. eh? eh? I’ll show myself.
List image by Flickr / Miran Rijavec