At this year’s Facebook Connect, the platform revealed something called the Presence Platform, a new way for developers to create mixed reality experiences for the Oculus Quest virtual reality headset. It includes a suite of AI tools for developers that makes the mixed reality experience more immersive, helping to combine the virtual world with physical reality.

The system includes three main tool sets: Insight SDK, Interaction SDK, and Voice SDK. Insight, which builds on the the Passthrough API that launched this summer, lets developers overlay virtual objects on the Oculus Quest’s passthrough video feed. Facebook is also adding a system called Spatial Anchors that will let developers “lock” a virtual object to a specific physical area so that object will maintain its position in the world. While the Passthrough API launched as an Oculus Quest 2 exclusive, Facebook tells The Verge that the new platform will support both generations of Quest headset.

Scene Understanding is the final piece of Insight SDK, which allows developers to build virtual reality experiences that match the scale of the player’s room. Facebook notes that as an example, “you can attach a virtual screen to the user’s wall or have a virtual character navigate on the floor with realistic occlusion.”

The second tool set, Interaction SDK, is supposed to make it easier for developers to integrate controller and hand-centric interactions. Facebook is making the Unity library available to developers next year, and as a part of it, developers will be able to incorporate ready-made interactive elements, such as “grab,” “poke,” “select,” and “target.” It offers improved hand tracking and gives developers the ability to create their own gestures.

Lastly, Voice SDK Experimental is a tool set powered by Facebook’s Wit.ai natural language platform. This can give users a hands-free virtual experience. It lets developers create Voice Navigation & Search functions, as well as a Voice FAQ that players can access by asking a question. Voice-Drive Gameplay could even give users the ability to utilize their voice while playing a game, such as when talking with an avatar or casting a voice-activated spell.

There’s been a lot of talk about the metaverse lately, considering that Facebook wants to change its name so it can properly become “a metaverse company.” And according to Facebook, the Presence Platform “will be key to feeling connected in the metaverse” — by letting developers blend virtual content with the real world.

Developing…