Building with VPS

**NOTE: This video is currently outdated at a few points as it aligns with ARDK version 2.1. Since then, TrackingStateUpdated has been replaced with TransformUpdated (20:06 and 28:33). The OnWayspotAnchorsAdded method has also been updated (18:32).

The written guide and code snippet below are aligned with ARDK 2.2 and on. If you’re watching the video, we recommend following along with the text below.

**Known Issue: Using async versions of CreateWayspotAnchors() in WayspotAnchorService throws an exception due to bug in internal ARDK code.

About This Guide [00:00]

Building an AR map of the world has been one of Niantic’s priorities from Day 1, since this is key to making the real-world metaverse come to life with information and interactivity. Our Visual Positioning System (VPS) now enables you to place virtual objects in a specific real-world location and have that object persist, so one person can leave an object for someone else to find, bringing real world global game boards to life.

By the end of this guide, you’ll be able to create your own immersive AR experience that uses persistent AR content in the real world that users can find and interact with.

VPS Concepts [00:58]

To understand how VPS works, we’ll start by going over its features. VPS has:

The VPS Coverage API which lets you discover VPS activated Wayspots near a real-world location.
The VPS Wayspot Anchors API for localizing with VPS-activated Wayspots and creating and managing virtual anchors associated with a VPS-activated Wayspot.

For the purpose of this guide we will use the Wayspot Anchors API to create VPS enabled anchors and save or load them on future app sessions.

Wayspots are places and objects that are locally unique, culturally meaningful, historically notable, or just simply worth venturing out to that have been mapped by Niantic’s player community according to Niantic mapping guidelines. A Niantic Wayspot can be VPS-activated, meaning that Lightship VPS will be able to localize with that Wayspot.

In order to help you with your development and testing process, Niantic also has the Wayfarer App that enables you to scan real-world locations and submit them to Niantic to be registered as VPS-activated Wayspots. You can use the app to create private scans for your own development use, or add scans for public VPS-activated Wayspots.

Step 1: Creating the Basic Structure [02:27]

For the purpose of this demo we’ve already taken the basic steps to set up our project in Unity with Lightship ARDK. If this is your first time working with Lightship or would like to refresh these steps, you can check out our Basics video or guide from this Getting Started series.

We will start by removing the Main Camera from the scene and adding the ARSceneCamera prefab that contains all the scripts that we will be using in this demo.

Step 2: Creating the Experience Logic [02:55]

Now we’re going to create the logic and objects that will interact within the space using plane tracking.

Start by adding an empty object into the scene. In this demo the object is called ARGame, but you can name it however you’d like. Then add the ARPlaneManager script from the ARDK package and set the PlanePrefab property with the PlanePrefab from ARDK Library.

Remember to set Detected Planes Types to Horizontal as the default value is set to “Nothing”. You’ll need to call this detected plane type property for later in the tutorial.

This demo uses the Horizontal planes only to keep the project simple, you could set to any values you need.

Step 3: Base Coding Structure and Init Event [03:35]

VPS experiences require some C# coding skills. If you’re not very familiar with Unity, you can check out Unity’s learning area which has tutorials of different levels and focus areas.

Also, this demo saves the data locally just to keep it simple. When you’re creating your project you can also store it in the cloud so you can share it with different users.

Going back to the code, start by adding a new Script component which this demo will call “ARGameLogic”.

Now we are going to create a series of variables to access our session, camera and prefabs.

Once you have created these, we’re going to define them in the properties panel. Our session property is private and will not show in the properties panel. We will be creating it by code through the ARSessionFactory.

We’ll use our MushroomPrefab which contains a simple 3D model, but you can use your own if you’d like. The MushroomPrefab is included in ARDK Templates See: Getting Started with Templates

To create a VPS enabled anchor we need to work on 2 different workflows using our Wayspot Anchor Service: creating an anchor, and then restoring that anchor.

Lightship’s Wayspot Anchor Service allows you to create a wayspot anchor which provides a payload that you can then save locally or on the cloud. At a later time, you can also provide the payload to restore the real world position of that anchor in your current device coordinates.

Code Snippet

Here is the basic ARGameLogic script you will be building. Please see the next steps for a detailed walk through.

using System.Collections;
using System.Collections.Generic;
using System.Linq;
using System;

using Niantic.ARDK.AR;
using Niantic.ARDK.Extensions;
using Niantic.ARDK.Utilities;
using Niantic.ARDK.Utilities.Input.Legacy;
using Niantic.ARDK.AR.HitTest;

using Niantic.ARDK.AR.WayspotAnchors;
using Niantic.ARDK.LocationService;
using Niantic.ARDK.AR.ARSessionEventArgs;
using Niantic.ARDK.AR.Configuration;

using UnityEngine;

public class ARGameLogic : MonoBehaviour
{
    private string LocalSaveKey = "my_wayspots";

    private IARSession session;
    public Camera camera;
    public GameObject objectPrefab;

    private bool InitialLocalizationFired = false;
    private WayspotAnchorService wayspotAnchorService;

    private Dictionary<System.Guid, GameObject> anchors = new Dictionary<System.Guid, GameObject>();

    void Start()
    {
      session = ARSessionFactory.Create();
      ARSessionFactory.SessionInitialized+=OnSessionInitialized;
    }

    void OnSessionInitialized(AnyARSessionInitializedArgs args){
      var configuration = ARWorldTrackingConfigurationFactory.Create();
      configuration.WorldAlignment = WorldAlignment.Gravity;
      configuration.PlaneDetection = PlaneDetection.Horizontal;
      configuration.IsLightEstimationEnabled = false;
      configuration.IsAutoFocusEnabled = false;
      configuration.IsDepthEnabled = false;
      configuration.IsSharedExperienceEnabled = false;
      session.Ran += OnSessionRan;
      session.Run(configuration);
      
    }
    void OnSessionRan(ARSessionRanArgs args){
        
      var wayspotAnchorsConfiguration = WayspotAnchorsConfigurationFactory.Create();
      
      var locationService = LocationServiceFactory.Create(session.RuntimeEnvironment);
      locationService.Start();

      wayspotAnchorService = new WayspotAnchorService(session, locationService, wayspotAnchorsConfiguration);
      // Uncomment following line to clear local data. Run once and comment again.
      // ClearLocalReferences();
    }
    
    void Update()
    {
      if(wayspotAnchorService==null || wayspotAnchorService.LocalizationState != LocalizationState.Localized){
        return;
      }
      if(wayspotAnchorService.LocalizationState==LocalizationState.Localized && !InitialLocalizationFired){
        LoadLocalReference();
        InitialLocalizationFired = true;
      }

      if (PlatformAgnosticInput.touchCount <= 0) return;
      var touch = PlatformAgnosticInput.GetTouch(0);
      if (touch.phase == TouchPhase.Began) OnTapScreen(touch);
    }

    void OnTapScreen(Touch touch){

        var currentFrame = session.CurrentFrame;

        if (currentFrame == null) return;

        var hitTestResults = currentFrame.HitTest (
            camera.pixelWidth, 
            camera.pixelHeight, 
            touch.position, 
            ARHitTestResultType.EstimatedHorizontalPlane
        );

        if (hitTestResults.Count <= 0) return;

        Matrix4x4 poseMatrix = hitTestResults[0].WorldTransform;//.ToPosition();

        AddAnchor(poseMatrix);
    }

    private void AddAnchor(Matrix4x4 poseData)
    {
      var anchors = wayspotAnchorService.CreateWayspotAnchors(poseData);
      OnWayspotAnchorsAdded(anchors);
    }


    private void OnWayspotAnchorsAdded(IWayspotAnchor[] wayspotAnchors)
    {
      foreach (var wayspotAnchor in wayspotAnchors)
      {
        if (anchors.ContainsKey(wayspotAnchor.ID)) continue;
        var id = wayspotAnchor.ID;
        var anchor = Instantiate(objectPrefab);
        anchor.SetActive(false);
        anchor.name = $"Anchor {id}";
        anchors.Add(id, anchor);

        wayspotAnchor.TransformUpdated += OnUpdateAnchorPose;

      }
      if(InitialLocalizationFired) SaveLocalReference();
    }

    private void OnUpdateAnchorPose(WayspotAnchorResolvedArgs wayspotAnchorResolvedArgs)
    {
      var anchor = anchors[wayspotAnchorResolvedArgs.ID].transform;
      anchor.position = wayspotAnchorResolvedArgs.Position;
      anchor.rotation = wayspotAnchorResolvedArgs.Rotation;
      anchor.gameObject.SetActive(true);
    }


    private void SaveLocalReference(){

      IWayspotAnchor[] wayspotAnchors = wayspotAnchorService.GetAllWayspotAnchors();

      MyStoredAnchorsData storedAnchorsData = new MyStoredAnchorsData();
      storedAnchorsData.Payloads = wayspotAnchors.Select(a => a.Payload.Serialize()).ToArray();

      string jsonData = JsonUtility.ToJson(storedAnchorsData);
      PlayerPrefs.SetString(LocalSaveKey, jsonData);

    }


    public void LoadLocalReference()
    {
      if (PlayerPrefs.HasKey(LocalSaveKey))
      {

        List<WayspotAnchorPayload> payloads = new List<WayspotAnchorPayload>();

        string json = PlayerPrefs.GetString(LocalSaveKey);
        MyStoredAnchorsData storedData = JsonUtility.FromJson<MyStoredAnchorsData>(json);

        foreach (var wayspotAnchorPayload in storedData.Payloads)
        {
          var payload = WayspotAnchorPayload.Deserialize(wayspotAnchorPayload);
          payloads.Add(payload);
        }

        if (payloads.Count > 0)
        {
          var wayspotAnchors = wayspotAnchorService.RestoreWayspotAnchors(payloads.ToArray());
          OnWayspotAnchorsAdded(wayspotAnchors);
        }

      }
    }

    [Serializable]
    private class MyStoredAnchorsData
    {
      public string[] Payloads = Array.Empty<string>();
    }


    // HELPER FUNCTION TO CLEAR THE LOCAL WAYSPOT ANCHOR CACHE
    private void ClearLocalReferences()
    {
      if (PlayerPrefs.HasKey(LocalSaveKey))
      {
        wayspotAnchorService.DestroyWayspotAnchors(anchors.Keys.ToArray());
        PlayerPrefs.DeleteKey(LocalSaveKey); 
      }
    }
}

Step 4: Starting a Session and Adding Events [05:14]

Now we’re going to create an AR Session and subscribe to its events.

Let’s begin by creating our session by invoking the Create method from the ARSessionFactory.
We need to subscribe to the OnSessionInitialized event on the ARSessionFactory.

Now that we have a session initialized, we will run the session with the proper configuration for the experience.

To configure the session we need to create a configuration through ARWorldTrackingConfigurationFactory. In this experience we need the WorldAlignment to be set to Gravity and the PlaneDetection to the same value as the PlaneTrakcer. In this case Horizontal.
Everything else should be set to false.
Remember to import Niantic.ARDK.AR.Configuration.

We recommend using the Gravity alignment versus Compass alignment to avoid drift issues when using the VPS tracking system.

Finally, let’s subscribe to the Ran event. We created a function called OnSessionRan for this matter.

Remember to import Niantic.ARDK.AR.AnyARSessionInitializedArgs, Niantic.ARDK.AR.ARSessionEventArgs as well as a Niantic.ARDK.Extensions namespace.

Now let’s continue by adding our Touch Events.

First, we’ll use the PlatformAgnosticInput under Niantic.ARDK.Utilities.Input.Legacy namespace to get the Touch Event and Touch Position relative to the user’s screen.

With that screen position value we’ll use the CurrentFrame hitTest method from the ARSession to get the collision point on the detected planes.

In this project we are using the pose as a Matrix4x4 to send the full position and rotation to the Wayspot Service.

Remember to import the namespace Niantic.ARDK.AR.HitTest

Step 5: Connection to Wayspot Service [12:51]

Creating VPS experiences requires us to rethink the way we normally tackle some flows within our projects.

For example, we won’t be creating our Anchor directly on our scene. Instead we will be sending the position to the Wayspot service and once it has been validated we will properly add the object to the scene.

Let’s create our Wayspot service with the WayspotAnchorService class. Remember to import Niantic.ARDK.AR.WayspotAnchors.

We will create the WayspotAnchorService in our OnSessionInitialized event. This service needs its own configuration as well as a session and a LocationService reference.

We have now created our configuration, created and started the locationService and finally created the WayspotAnchorService as well. Creating this service automatically starts the localization process in the background.

Remember to import Niantic.ARDK.LocationService to access the class and factory implementations.

Lightship VPS uses the device location service, so you’ll need to make sure your app has permissions to access these services. The ARDK provides convenience classes for handling permission requests.

To work with the WayspotService, we need to make sure our service is localized. To that end we will be checking the localized status of the session. Let’s create a boolean variable called InitialLocalizationFired to use as a toggle inside our code and check for the localization status on the update method.

In this code we are just checking if the WayspotAnchorService has been localized to keep the code simpler, but in your production code you should check for the other status: Initializing, Localizing, Localized and Failed. This way you will have a proper app response for each localization state.

We have two checks on the update method. The first one is to simply skip any update logic and interaction while the wayspotService hasn’t been created or localized, and the second one triggers a function once the service has been localized.

We will be using this event a few steps later in this tutorial.

Step 6: Sending the Position to the Wayspot Service [18:08]

Now let’s move into allowing users to add an actual Anchor to our experience.

Earlier in this tutorial we talked about sending the position to the WayspotAnchorService and adding the object to the scene. Now we’re going to create a function called AddAnchor that receives the Matrix4x4 pose and sends it to the Waypost service using the method CreateWayspotAnchors.

This method receives a matrix pose and returns a list of anchors. We then need to call our method OnWayspotAnchorsAdded, with the array of anchors added to the service as a parameter.

We’ll also use a Dictionary to know which objects have been already created. This will prevent the creation of multiple instances of the same object during the session

Lets create a property called Anchors where we will save the current Anchor ID as well as the gameobject linked to it.

Now let’s create our gameobject and position it.

The OnWayspotAnchorsAdded method receives an array of IWayspotAnchor with its corresponding ID for each one.

We’ll use that ID to check on the Anchor’s Dictionary to see if it has been already created. In case it hasn’t been already created, we will proceed to create the object in the scene and add its ID to the Anchor’s Dictionary.

In this demo we are not setting the position ourselves, we are subscribing to a TransformUpdated event to set the position. This ensures that our Anchor will reposition and update upon any change on the service.

We’ll create our objects as hidden and then enable them when we have their correct position.

Now we have our project using the WayspotService to place objects in space using VPS as the decision maker of the experience.

Step 7: Storing Data Locally [24:23]

To be able to interact with VPS over time we need to save the anchor payload of our Wayspot Anchors to then query on the service once we reload the app again.

You can either save that information locally on your device, using an API or even on a cloud database. In our case we will set up a simple PlayerPref variable where we will be storing our data as a serialized string.

We need to define a key name for our PlayerPref. In our case we will set a variable called LocalSaveKey with a custom value my_wayspots.

To be able to store the data serialized we need to define a structure. So, let’s create a class with a property where we will store all the anchors. We’ll call it MyStoredAnchorsData.

Now, let’s create a method called SaveLocalReference to store all the WayspotAnchors we created in the current session.

This method tackles several things. First, it gets all the WayspotAnchors in the current session. Second, it stores them as a Payload in our MyStoredAnchorsData. And finally it saves its serialized content into the PlayerPrefs.

Keep in mind that although we are just saving the anchors for this example, you can also save other info related to the Anchors like model, color, text, and sound id, among others.

Let’s finish the save process by setting up a call once we have successfully added an Anchor to the experience. Let’s go back to the OnWayspotAnchorsAdded and call the SaveLocalReference at the end.

Make sure you call it outside the Foreach loop to do it one single time per Anchor.

Step 8: Loading Locally Stored Data [27:33]

Lets load all the Anchors we have saved once we start our app. For this purpose let’s create a LoadLocalReference method.

This method involves quite a bit of work, but we’ll get there. The first thing we’re going to do is to create our data structure and deserialize the stored information.

Once we have our data, we cycle through it to create a payload list to be queried in the WayspotService.

We are going to call the RestoreWayspotAnchors service to get all the references from VPS. Note: this can only be done once in Localized state. This method handles the creation of events that update the tracking information and alignment into the scene. The TransformUpdated event will now provide updates on these anchors that were restored.

And finally we use the same method to create our gameobjects in the scene, OnWayspotAnchorsAdded.

To call this function we will need to have our service running and localized, for this demo we can use our implementation inside the Update method that evaluates when the service is localized.

Let’s add a call before we set our InitialLocalizationFired boolean, that’s a nifty little trick ;)

Since we are using the same method (OnWayspotAnchorsAdded) when we create an Anchor or load an Anchor the SaveLocalRefences call will occur in both situations.

To prevent that from happening let’s use our InitialLocalizationFired variable, since we define its value after we load the anchors we know that will be true only when you go through the adding anchor process.

So let’s use it as a flag to prevent saving on load.

And with that little trick our demo is done!

Conclusion [31:54]

Now you can successfully create your own version of a VPS-enabled experience that allows users to locate objects with precision in the real world.

You can also test out what you built on Unity’s mockup scene and on your devices. If you need guidance, check out our documentation on the Developers’ Portal.

What now?

Check out our Tutorials page for several tutorials on specific Lightship topics
Join the Lightship Community Discord to discuss Lightship projects with other developers
Follow Lightship on Twitter and Instagram to see projects built with Lightship and stay in the know with updates to the platform

Happy Building!

Learn more