Jony Ive and OpenAI's secret AI device is facing 'critical' challenges

In 2023, we learned that Apple's ex-chief design officer, Jony Ive, is working with OpenAI on an AI-powered hardware project. These rumors were confirmed earlier this year when OpenAI announced that it is acquiring Ive's startup "io" for $6.5 billion in an all-cash deal, with whispers emerging about multiple AI products currently in development, including a pendant, a smart speaker, and a robot. Now, a new report claims that at least one of the projects is facing significant technical challenges.

Financial Times says that a secretive AI project that revolves around a palm-sized screen-less device that can take audiovisual cues from the environment to facilitate users is facing "critical" issues. These problems primarily seem to be on the software and infrastructure side rather than hardware, which would indicate that Ive's team may not be directly responsible for handling these challenges.

There are discussions going on around the personality of the AI, potential privacy issues, and the massive amount of computing that is needed to power the device in mass market conditions. An anonymous source close to Ive noted that:

Compute is another huge factor for the delay. Amazon has the compute for an Alexa, so does Google [for its Home device], but OpenAI is struggling to get enough compute for ChatGPT, let alone an AI device — they need to fix that first.

Meanwhile, one anonymous source from OpenAI downplayed these concerns, calling them a normal part of product development.

This particular product is designed to sit on a desk but also be portable. It reportedly doesn't feature a screen and only uses cameras, a microphone, and a speaker to interact with the environment and the user. It will supposedly be an always-on device, but that is also causing some problems in the development process, as it should only interact with the user when it's actually beneficial to do so, and it should halt the conversation when an interaction has run its course. Both of these are problems that OpenAI's flagship product, ChatGPT, has yet to tackle or solve.

This "smart speaker" of sorts is scheduled to enter the market later next year, provided all goes well. The latest challenges are tricky, but given the rapid pace of development in the AI sector, it wouldn't be surprising to see Ive and Altman's teams working around them. We'll likely find out more over the next few months.