How to Use VPS Colocalization with Netcode

Colocalization is an essential piece of Shared AR that ensures players will see virtual objects in the same real-world location. Using the SharedSpaceManager class, we can automatically create the network connection and allow players in the same VPS-Activated Location to join a multiplayer AR session without needing a separate networking interface.

Prerequisites

1. You will need a Unity project with ARDK installed and a set-up basic AR scene. For more information, see Set Up Lightship ARDK and Setting up a Basic AR Scene.
   1. Your AR scene will also need an ARLocation. To learn how to add one, see How to Place Content in Real-World Locations Using Location AR.
2. This How-To uses the project from Shared AR and Netcode as its base. If you have not completed it, start there.
3. To use VPS Colocalization, make sure the Colocalization Type in the SharedSpaceManager is set to VPS Colocalization.

VPS Colocalization Overview

To set up VPS Colocalization, you will need to do the following:

1. Set the VPS-Activated Location you want to track.
2. Start VPS tracking by calling SharedSpaceManager.StartSharedSpace().
3. Once VPS reaches the "tracking" state, the Shared AR Origin will be aligned to a point in the real world.
4. Using the steps in How to Display Shared Objects with Netocde, spawn objects in your scene.
5. Start Netcode and play!

There are two main ways to target a VPS-Activated Location; by using the Coverage API and selecting a location at runtime, or by setting a VPS-Activated Location in the Unity editor and building your application around it. Selecting a location at runtime requires more UI and code to handle user input but provides flexibility to work at any VPS-Activated Location. Setting up locations at build time is simpler and allows you to design the experience around a specific space, but is not scalable.

note

Even if your Shared AR experience is designed to be played at any VPS-Activated Location, it can be useful to hard-code in a specific location when testing. This lets you avoid any additional issues with the networking UI if all you want is to test your scene!

Set a VPS-Activated Location using ARLocationManager

To start, set up a SharedSpaceManager for VPS Colocalization:

1. In the Hierarchy, select XR Origin, then, in the Inspector, find the SharedSpaceManager Component. Set the Colocalization Type to VpsColocalization.
2. Drag and drop the XR Origin from the Hierarchy to the AR Location Manager Component.

The Shared Space Manager and AR Location Manager components should now look like this:

Testing Colocalization With Test Scans

If there are no VPS-Activated Locations near you, a test scan can be used to test VPS colocalization instead. To set up a test scan for colocalization:

1. Follow the instructions from How to Manage Private Scans to add a scan to your API key.
2. Continue to Downloading Meshes and follow the instructions to download your mesh.
   1. Make sure to verify your Test Scan Manifest's Location Latitude and Location Longitude now. Waiting until after this step may cause issues!
3. Install the mesh in your Unity project by following steps 1-3 in Adding a Real-World Location to Unity.

Setting Up VPS Colocalization in SharedSpaceManager

To set up your project for VPS colocalization:

1. In the Hierarchy, rename NetworkDemoManager to VPSLocationDemoManager.
2. In the Project window, open the Assets folder and open VPSLocationDemoManager.cs. Find the list of [SerializeField] calls, then add this reference to it:

    [SerializeField]
    private ARLocationManager _arLocationManager;

3. Replace the code in void Start() with this snippet:

    // Set room to join
    // This demo only targets a single ARLocation, so we can just use the first location.
    // For applications that choose from a list, use the specific ARLocation that you are localizing against as the room ID.
    var vpsTrackingOptions = ISharedSpaceTrackingOptions.CreateVpsTrackingOptions(_arLocationManager.ARLocations.First());
    var roomOptions = ISharedSpaceRoomOptions.CreateVpsRoomOptions(
        vpsTrackingOptions,
        "optionalPrefixForRoom_", // room name will be this prefix + Base64 Location anchor data
        32, // set capacity to max
        "Room Description Here!");
    _sharedSpaceManager.StartSharedSpace(vpsTrackingOptions, roomOptions);

At this point, VPSLocationDemoManager.cs should look like this:

Click to show the full VPSLocationDemoManager code sample

using System.Linq;
using Niantic.Lightship.AR.LocationAR;
using Niantic.Lightship.SharedAR.Colocalization;
using Unity.Netcode;
using UnityEngine;
using UnityEngine.UI;

public class VPSLocationDemoManager : MonoBehaviour
{
    [SerializeField]
    private Text _statusText;

    [SerializeField]
    private Button _joinAsHostButton;

    [SerializeField]
    private Button _joinAsClientButton;

    [SerializeField]
    private SharedSpaceManager _sharedSpaceManager;

    [SerializeField]
    private ARLocationManager _arLocationManager;

    protected void Start()
    {
        // Hide UI until VPS is in tracking state
        _joinAsHostButton.gameObject.SetActive(false);
        _joinAsClientButton.gameObject.SetActive(false);

        // UI event listeners
        _joinAsHostButton.onClick.AddListener(OnJoinAsHostClicked);
        _joinAsClientButton.onClick.AddListener(OnJoinAsClientClicked);

        // Netcode connection event callback
        NetworkManager.Singleton.OnClientConnectedCallback += OnClientConnectedCallback;

        // Set SharedSpaceManager and start it
        _sharedSpaceManager.sharedSpaceManagerStateChanged += OnColocalizationTrackingStateChanged;

        // Set room to join
        // This demo only targets a single ARLocation, so we can just use the first location.
        // For applications that choose from a list, use the specific ARLocation that you are localizing against
        //  as the room ID
        var vpsTrackingOptions = ISharedSpaceTrackingOptions.CreateVpsTrackingOptions(_arLocationManager.ARLocations.First());
        var roomOptions = ISharedSpaceRoomOptions.CreateVpsRoomOptions(
            vpsTrackingOptions,
            "optionalPrefixForRoom_", // room name will be this prefix + Base64 Location anchor data
            32, // set capacity to max
            "Room Description Here!");
        _sharedSpaceManager.StartSharedSpace(vpsTrackingOptions, roomOptions);
    }

    private void OnColocalizationTrackingStateChanged(
        SharedSpaceManager.SharedSpaceManagerStateChangeEventArgs args)
    {
        // Show Join UI
        if (args.Tracking)
        {
            _statusText.text = $"Localized";
            _joinAsHostButton.gameObject.SetActive(true);
            _joinAsClientButton.gameObject.SetActive(true);
        }
    }
    private void OnJoinAsHostClicked()
    {
        NetworkManager.Singleton.StartHost();
        HideButtons();
    }

    private void OnJoinAsClientClicked()
    {
        NetworkManager.Singleton.StartClient();
        HideButtons();
    }

    private void HideButtons()
    {
        _joinAsHostButton.gameObject.SetActive(false);
        _joinAsClientButton.gameObject.SetActive(false);
    }

    private void OnClientConnectedCallback(ulong clientId)
    {
        _statusText.text = $"Connected: {clientId}";
    }
}

If you build and run the application at this point, you should see buttons for joining as host or client and a text box that displays the localization state.

Showing the Position of Other Devices

Using the Unity Netcode NetworkManager Player Prefab in Owner Authoritative Mode, we can synchronize the positions of peer devices and display player avatars in the correct position.

To set up the Player Prefab:

1. In the Hierarchy, right-click under the AR scene and select Create Empty. Name the new object playerPrefab.

2. Drag playerPrefab from the Hierarchy to the Assets folder in the Project window to make it into a prefab, then delete it from the scene.

3. Select playerPrefab, then, in the Inspector, click Add Component. Add a LightshipNetworkObject Component to the prefab, then add a New Script Component. Name the script PlayerAvatar.

4. Copy this PlayerAvatar code into the script file:

   Click to show the full PlayerAvatar code sample

   // Copyright 2023 Niantic, Inc. All Rights Reserved.
   
   using Unity.Netcode.Components;
   
   using UnityEngine;
   
   public class PlayerAvatar: NetworkTransform
   {
     [HideInInspector]
     private Transform _arCameraTransform;
   
     protected override bool OnIsServerAuthoritative()
     {
       return false;
     }
   
     public override void OnNetworkSpawn()
     {
       if (IsOwner)
       {
         if (Camera.main)
         {
           _arCameraTransform = Camera.main.transform;
         }
       }
   
       base.OnNetworkSpawn();
     }
   
     new void Update()
     {
       if (IsOwner)
       {
         if (_arCameraTransform)
         {
           // Get local AR camera transform
           _arCameraTransform.GetPositionAndRotation(out var pos, out var rot);
           // Since using the ClientNetworkTransform, just update world transform of the cube matching with the
           // AR Camera's worldTransform. it's local transform will be synced.
           transform.SetPositionAndRotation(pos, rot);
         }
       }
   
       base.Update();
     }
   }

This script returns false from OnIsServerAuthoritative() to ensure that updates are driven by the player avatar object owner, not the server. Each player object is created by Netcode when joining and is owned by the user's device.

Finalizing the Player Prefab with Position Sync

The last requirement for a working player prefab is making sure that player avatar positions match the devices they represent. In the PlayerAvatar script, the Main Camera's transform values are used as the device position. To make sure the player avatar and device position line up correctly, we make the player avatar match the Main Camera transform values and automatically sync their positions.

To enable position sync, select playerPrefab from the Assets folder, then enable In Local Space in the PlayerAvatar Component.

At this point, the player prefab is ready to use in a Shared AR experience. To double-check, make sure the prefab's Inspector view looks like this:

Testing the Player Prefab

Before we can test the player prefab, we need to add it to the Network Manager. In the Hierarchy, select NetworkManager, then drag playerPrefab from the Assets folder to the Player Prefab field in the Inspector.

Once you have added the prefab to the Network Manager, build and run this sample on two devices simultaneously. A player prefab should appear on each device in the session.

Next Steps

To learn how to add shared virtual objects to your multiplayer experiences, see How to Display Shared Objects.

For help with common Shared AR issues and debugging suggestions, see How to Debug and Troubleshoot Shared AR.