Creating a Shared AR Experience

Enable nearby players to join the same AR experience.

Anatomy of an ARNetworking Session

For multiple players to interact in the same AR world, they have to share information from the separate ARSessions running on each of their devices. Connecting players together so they can send and receive that information is handled by a MultipeerNetworking instance on each device. Processing that information is in turn handled by an ARNetworking instance on each device.

ARNetworking sessions are constructed using the ARNetworkingFactory class.

using Niantic.ARDK.AR.Networking;

// Create an ARNetworking session.
// This also creates new ARSession and MultipeerNetworking objects,
// since they are required components of ARNetworking.
var arNetworking = ARNetworkingFactory.Create();

An ARNetworking session can also be constructed from existing ARSession or MultipeerNetworking instances.

For example, your application could have a lobby, where players who are connected to a MulitpeerNetworking session can send messages to each other while waiting for a shared AR experience to start. The players could be labeled by colors based on their peer Identifier, so when those players hop into the shared AR experience, you want those identifiers to stay the same. Using that same MulitpeerNetworking session to power the ARNetworking session enables that, because it means players wouldn’t have to disconnect and join a new session.

using Niantic.ARDK.Networking;
using System.Text;

var networking = MultipeerNetworkingFactory.Create();

// Can join either before or after creating an ARNetworking
var sessionIdentifier = Encoding.UTF8.GetBytes("Example");

// The ARSession and MultipeerNetworking objects used to create an ARNetworking
// need to have the same stage identifier, so specify the stage identifier for
// the ARSession constructed here.
var arSession = ARSessionFactory.Create(networking.StageIdentifier);

var arNetworking = ARNetworkingFactory.Create(arSession, networking);

Starting a Shared Experience

To enable a shared AR experience, both the underlying ARSession and MultipeerNetworking components of the ARNetworking object need to be started.

using using Niantic.ARDK.AR.Configuration;

// Create a configuration.
var configuration = ARWorldTrackingConfigurationFactory.Create();

// Enable shared experiences.
configuration.IsSharedExperienceEnabled = true;

// Run the AR session (if you haven't already).

// Join the networking session (if you haven't already).
var sessionIdentifier = Encoding.UTF8.GetBytes("Example");

See the page on Creating AR Experiences for a comprehensive explanation on how to set up an AR experience, and the page on Create a Multiplayer Experience for how to network players together. In short though:

  1. Add an ARSceneCamera prefab to your scene.

  2. Use the same sessionIdentifier value for all players joining the same session.

Using ARNetworkingManager

To simplify the process of setting up an AR networked session, we provide a Manager you can add to your scene. The API reference and in-code comments/tool tips for ARNetworkingManager explains how to use it.

Mapping the Environment

To ensure all clients can interact in a shared AR environment, clients must synchronize using mapping data from the environment generated by ARDK. This involves scanning the environment to create mapping data that all clients can synchronize to. This process is also known as AR localization.

In an ARNetworking session, the PeerState values mapped to each player can be used to keep track of where they are in the localization process.

Development steps and best practices for implementing localization are detailed in Establishing Localization.

Peer Poses

In an ARNetworking session, all peers begin in their own coordinate systems, where the origin (0,0,0) is located where their device was when the ARSession started running. Upon reaching stability, a peer’s pose will snap to be in the host’s coordinate system. .. only:: ardk_internal

When using map layers, peer poses are handled though peer anchors instead.

By default, all stable peers will broadcast their pose to all other peers. Pose broadcasting can be configured based on your application’s requirements.

// To enable

// To disable

// To configure how often poses are broadcast

The latest available pose of each peer can be retrieved through the IARNetworking interface.

var allPeerPoses = arNetworking.LatestPeerPoses;
var myPeerPose = arNetworking.ARSession.CurrentFrame.Camera.Transform;

Alternatively, peer pose updates, including those for the local device, can be subscribed to.

using Niantic.ARDK.Utilities;

IARNetworking _arNetworking;
Dictionary<IPeer, GameObject> _peerGameObjects;

void SubscribeToPeerStateUpdates()
  _arNetworking.PeerPoseReceived += OnPeerPoseReceived;

void OnPeerPoseReceived(PeerPoseReceivedArgs args)
  var peer = args.Peer;
  var pose = args.Pose;

  // Use this information to attach a GameObject to a player,
  // have in-game characters look at a player, have an AI
  // shoot at a player, etc.
  var peerGameObject = _peerGameObjects[peer];
  peerGameObject.transform.position = pose.ToPosition();
  peerGameObject.transform.rotation = pose.ToRotation();

Placing Virtual Content in a Shared World

When using an ARSession, the origin (0,0,0) will be at the original location of the device when the session was run. 1 unit (in Unity space) corresponds to roughly 1 meter in the real world.

When AR spaces are connected through an ARNetworking session, all player will initially have their own coordinate systems, corresponding to their startup locations. An object placed at position (X, Y, Z) in each player’s virtual environment will not appear in the same position for everyone in the real-world environment.

However, upon completing localization, the positions of all non-host devices will “move” to new positions corresponding to their relative locations on the host’s coordinate system. At this point, all clients are using the same coordinate system (the host’s), and placing a virtual object at position (X, Y, Z) will look the same for everyone in the real world as well.

Walkthrough of Developing a Shared AR Experience with ARDK

The following video walks you through creating a shared AR experience in Unity. The video demonstrates how to add ARNetworking, scanning the environment to localize, and more:

See Also


Multiplayer AR