Wed. Sep 28th, 2022
Microsoft has stopped making the Kinect, and that makes me sad

Microsoft no longer builds Kinect devices for consoles or PCs, Fast Co writes. design. Since their introduction in 2010 on the Xbox 360 and through major updates to Xbox One and PC, the sensors combined a depth-sensing camera, a regular video camera, and a microphone array into a device that Microsoft hoped would usher in a new wave of games and apps-packed. voice and motion based controls. Microsoft’s own marketing promised gamers that they would be the controller themselves.

With Kinect, Microsoft seemed to lead the tech world with a rich mix of voice and motion controls that no one else could match, all neatly packaged in an easy-to-use box.

But a world of immersive voice and motion games never really materialized, so a device that once held such promise — like a box in your living room you talk to; as a showcase for machine vision; as the basis for complex multimodal input mixing controllers, movement and voice – being destroyed after never living up to the potential we thought it had. And as someone who still uses a Kinect every day, I’m more than a little saddened.

It’s hard to be different

Why did this happen? Just as Nintendo discovered with the Wii, new control mechanisms such as Kinect’s combination of motion, motion and voice have proved challenging for game developers to exploit successfully.

This is partly because they are a poor match for a world dominated by cross-platform titles built for a generic game console rather than a particular device. Why make a title for the Kinect if that means leaving every PlayStation 3 owner behind, along with every Xbox 360 owner who hasn’t spent $150 on peripherals? Microsoft has made every effort to make the hardware more attractive in this regard. The Xbox One initially shipped with a revised Kinect sensor in the box to ensure developers for the Xbox One could be confident that the hardware would at least available to the target group.

Even this move turned out to be an obligation. The inclusion drove up the cost of the hardware and made the Xbox One a somewhat less attractive development platform. Initially, the Xbox One devoted a certain amount of memory and processing power to processing Kinect input, so even non-Kinect games couldn’t harness the full power of the machine. This might have been good if the bundling had spawned a wave of high-quality, Kinect-exploiting games.

But that didn’t happen because of a deeper and more persistent problem: these new input systems present a certain conceptual difficulty. The metaphors and design elements that work with controllers, mice and keyboards are all well established and well understood by developers and players alike. Similar concepts for Kinect, like for the Wii before it, don’t really exist. The same vocabulary of concepts has not yet been developed. Even if they could guarantee the hardware was there, developers never developed a strong idea of ​​what to do to do with it.

In addition, and especially with the first version for Xbox 360, developers struggled to get the latency — the time between performing an action in real life and seeing it on screen — low enough for games to feel comfortable. . The inability to track individual fingers similarly made it difficult to develop games that required fine control. The Xbox One’s Kinect was much better in both respects, but the lack of involvement with the original meant it didn’t matter anymore; interest from game developers had evaporated.

The demise of the hardware will be mourned by a certain kind of more experimental developer. Microsoft produced Kinect hardware that worked with Windows PCs, along with an SDK to develop software for it, and several researchers have taken advantage of it by using the peripherals for things like building machine vision systems for robots and prototyping and development. of user interfaces.

Laying the foundation

However, the Kinect concepts live on and one could argue that Kinect was the right technology, just delivered in a not quite right way. Microsoft’s HoloLens augmented reality headset, for example, uses a derivative of the Kinect for its depth sensor, and it uses its integrated sensors to scan the room around you and detect objects in it. This, in turn, has given way to systems that use a conventional camera along with accelerometer data to track movement, something that underlies the virtual and augmented reality system in the latest version of Windows 10.

Google has gone through a similar evolution, from special Kinect-like hardware called Project Tango to a camera- and accelerometer-based system. Apple meanwhile bought PrimeSense, the company that made the sensors for the original Xbox 360 Kinect. A highly miniaturized version of that same technology is the power of the Face ID feature coming soon with the iPhone X. Here, the depth perception is used to better track the face rather than creating a map of the environment or tracking your body’s movements. Apple also has an augmented reality framework that again uses regular cameras and accelerometers.

The HoloLens integrates its own Kinect-like sensor.
enlarge / The HoloLens integrates its own Kinect-like sensor.

Microsoft

Voice control is also emerging in the same ambient style offered by Kinect. The Xbox One with Kinect sits silently in the corner of the room, listening for the command to turn on; when on, you can talk to it from the other room to watch a video, start a game or play some music. The Amazon Echo and Google Home devices develop the same core idea and work as headless agents to fulfill your bids. You just speak utterances in the void; the cylinder in the corner of the room hears you and reacts accordingly.

Microsoft never quite allowed Cortana on the Xbox to live up to this same potential. Last week, a self-contained Cortana speaker from Harman Kardon hit the market, and it rivals the offerings of Google and Amazon. Standalone speakers are essential to really reaching into the living room – a cost comparison between the Echo and the Xbox One makes that much clearer. But you feel the Xbox could and should have offered a similar experience. It should have led the way as a voice-activated agent, with Microsoft using the Xbox to ensure that “Hey Cortana” is as iconic and capable as “Alexa” or “OK ​​Google”.

Missed potential

But the Xbox didn’t offer the same experience and with the end of Kinect, it looks like it never will. Unlike those cheaper devices, the Xbox One doesn’t have its own eyes or ears; it relied on the Kinect to provide that information. While the Xbox One S and Xbox One X are both compatible with the Kinect (although they require the use of an adapter to convert from Kinect’s proprietary connector to standard USB), neither are included, and with the discontinuation, neither will they ever. As Kinect hardware becomes more difficult to obtain, it will also become more difficult to give Cortana those ears that she so desperately needs. A Cortana you can’t talk to might as well not exist.

Microsoft’s “solution”, as it is, is to plug a headset into your Xbox controller and use that to issue commands to Cortana. This is a very different experience from the ambient style found with Alexa or Google Home, and if Microsoft thinks it’s a reasonable suggestion to put on a headset to pause Netflix or wake the Xbox so I can watch something while cooking dinner, I have news: it isn’t.

It makes for a very strange gap; while on the one hand Microsoft’s Windows, Bing and Research divisions are investing heavily in Cortana and machine learning to build an increasingly rich, more capable digital agent, the Xbox team has put an end to what may one of the most natural and convenient ways to actually use that agent. I could live without the cameras and motion input — my New York apartment isn’t big enough for that style of gaming anyway, so it’s never been my priority — but for the life of me, I can’t understand why the Xbox One S and Xbox One X don’t have a built-in far-field array microphone, so they can continue to support voice commands from across the room even without a Kinect connected.

The technology isn’t the only part of the Kinect that sticks; so are the conceptual difficulties. Voice control remains frustratingly inflexible, and the success or failure of augmented and virtual reality will depend not only on developers producing compelling software to take advantage of the hardware, but also on producing the metaphors and idioms to make this software familiar and predictable. . Movement control is difficult, especially on the whole-body scale. Compared to our fingers, our bodies move slowly and imprecisely, and room-scale VR games in particular have to be sensitive to this, just like Kinect titles. We have better motion controllers these days, but we still miss the ability to track individual finger movements.

The industry is investing heavily in AR and VR technology, but no one can claim to have solved human interaction. There’s still every chance that the fate of VR, AR, and even voice control could eventually mirror that of the device that was their predecessor in so many ways.

By akfire1

Leave a Reply

Your email address will not be published.