Apple’s long-awaited mixed reality headset is the company’s ambitious attempt to create a 3D version of the iPhone operating system with eye and hand tracking systems that could set the technology apart from competitors’ products. The roughly $3,000 device, likely to be called the Reality Pro later this year, will take a new approach to virtual meetings and immersive video to shake up the VR industry, which is currently dominated by Meta.
Apple is entering an uncertain market with a premium product. The company’s 1,000-strong technology development group has spent more than seven years on the project, and the company expects it to become a new source of revenue, especially as sales growth stalls this year.
But virtual reality turned out to be a problem for the biggest titans of technology. While some predict the industry will top $100 billion by the end of the decade, headsets are still considered a niche product – and Meta has lost billions on its efforts.
Apple’s goal is to imagine something new. According to people familiar with the product, the eye and hand tracking capabilities will be the main selling point of the device. Its core features will include advanced video conferencing and FaceTime-based meeting rooms. The headset will also be able to display immersive video content, serve as an external display for a connected Mac, and duplicate many iPhone and iPad functions.
The headset will receive several external cameras that can analyze the movement of the user’s hands and sensors in the body of the gadget to read eyes. This would allow a person to control the device by glancing at an item on the screen – whether it’s a button, an app icon, or an item in a list – to select it. To activate the task, the user must simply bring the thumb and forefinger together without having to hold anything. This approach differs from other headsets, which typically rely on a hand controller.
Like the latest Meta headset, Apple’s device will use both virtual and augmented reality. With virtual reality, users see images and content inside the glasses. In addition, augmented reality overlays digital content on the image of the real world.
The headset received two ultra-high-resolution displays developed in conjunction with Sony to work with virtual reality and a set of external cameras to provide end-to-end AR mode. This means that users will see the real world through the cameras on the headset. Apple will offer users with glasses special lenses that are in the case itself.
The device will feature a Digital Crown, a wheel similar to the Apple Watch, which will allow you to switch between VR and AR. Apple expects this to make the product stand out from the rest.
FaceTime software will realistically display the user’s face and entire body in virtual reality. These avatars will allow two people – each with an Apple headset – to communicate with the effect of presence. The technology differs from the virtual meeting rooms on the Meta headset, creating a more cartoon-like user avatar.
Due to the massive processing power required for this feature, the headset will only support realistic avatars during one-on-one video chats. This will still allow FaceTime sessions with multiple people, but additional users will be displayed as an icon or Memoji, Apple’s customized emoji.
Apple plans to introduce the device as early as this spring, although the schedule is still subject to change. This will allow the company to discuss the product at its annual software developer conference in June and then release it later this year. Some features could still be canceled or changed, Bloomberg’s sources said.
The company may begin early production of the device in China as early as February, and is considering launching the product only in the US at first. The price is also expected to limit the product’s appeal, and Apple is already working on a cheaper version – which will be released in late 2024 or early 2025 – and will cost closer to $1,500.
Apple expects to sell about a million units of its new headset in the first year. That compares with more than 200 million iPhone units, the Cupertino-based company’s top earner. However, Apple doesn’t plan to turn a profit from the initial version right away – even at the high price – which suggests that the company has long-term plans for the platform.
The main function of the new device will be watching videos with an immersive effect. Apple has discussed developing VR content for the platform with a dozen media partners, including Walt Disney and Dolby Laboratories. The tech giant itself is working on updating its own Apple TV+ materials to work with the headset. In 2020, Apple acquired streaming company NextVR to create sports content in VR.
Apple plans that the headset will have a special video viewing feature that will make viewers feel like they are watching a movie on a giant screen in a different environment, such as a desert or outer space. However, the headset’s speakers may not be able to cope, so users will need to wear AirPods to get the full surround sound effect.
The device will also get productivity features, including the ability to serve as an external monitor for Macs. With this feature, users will be able to see the computer display in virtual reality, but control it with a trackpad or mouse and a physical keyboard.
The headset’s operating system, called xrOS, will have many of the same features as the iPhone and iPad, but in a 3D environment. These include the Safari web browser, Photos, Mail, Messages, and the Calendar app, as well as apps for the company’s services, such as the App Store for installing third-party software, Apple TV+, music, and podcasts. The company is also working on health tracking features.
The experience should be familiar to Apple users. When they put on the headset, the main interface will be almost identical to that of the iPhone and iPad, showing a home screen with a grid of icons that can be rearranged. Users will be able to pin widgets such as weather, calendar appointments, email and stock market indicators to their app icons.
When they need to type, they can use the Siri voice assistant or the iPhone, Mac or iPad keyboard. However, unlike the Apple Watch, having an iPhone is not required for work. The company is developing technology that will allow users to type in the air with their hands, but such a feature is unlikely to be ready for the first launch.
Games are expected to become a popular offering from third-party developers. In 2017, the company released ARKit and other tools to help prepare augmented reality experiences for the iPhone. This provided the basis for programmers to create applications, games and services for the headset.
Apple’s device will include a variation of the M2 chip found in recent Macs, as well as a dedicated processor for graphics and mixed reality. This second chip will be called the Reality Processor, according to trademark applications filed by the tech giant.
But the creation of sufficiently powerful processors caused another problem: the device overheated when it was on the user’s face. To solve this problem, Apple decided to make an external battery that fits in the user’s pocket and connects with a cable. Another change is the inclusion of a cooling fan like on high-end Macs.
The headset can work for about two hours on a single battery like competitors’ products. However, the battery is too big: about the size of two iPhone 14 Pro Max stacked on top of each other, or about 15.2 cm high and 1.27 thick. However, some internal prototypes for software development still have a built-in battery and charge via USB-C.
The Meta, in contrast, places the batteries on the back of the headset in a way that helps balance the device on a person’s head. Apple’s approach may prove less user-friendly, especially if they’re watching a movie with a headset. Some testers complained that the product was bulky.
The relatively short battery life – about 20 hours less than Apple’s latest MacBook Pro – can present other problems. If users want to watch movies or play games for hours, they will have to buy several batteries and change them often.
Apple recognized these challenges internally and tried to set realistic product expectations. One of the benefits of the device, according to the company, is that it can encourage customers to visit Apple retail stores to try the product and purchase another device, such as an iPad or AirPods.
The initial headset will be made of aluminum and glass – and will resemble Apple’s $550 AirPods Max headphones. The product will receive a curved screen on the front panel, speakers on the sides and a special headband that helps to place the device on the head. This will distinguish the product from the mostly plastic design of competitors, who usually “tie” the device to the user with several straps.
Eye and hand tracking systems can become the most original elements of the headset. Apple likes to add an innovative interface that differentiates its products from the competition: on the iPod it was the wheel, on the iPhone and iPad it was a multi-touch approach, and on the Apple Watch it was the Digital Crown. Now the company is hoping that with the headset’s sci-fi interface, its latest product will become a leader as well.
Source: Bloomberg
