How to Debug Shared AR

Because Shared AR combines the complexity of AR with the networking issues of multiplayer, it can be difficult to debug. To debug development issues, you will need multiple devices or multiple instances of the Unity editor running on one machine. Multiplayer also causes more race conditions and timing issues than single-player applications. This How-To covers some tips for debugging Shared AR experiences and serves as a troubleshooting guide for common problems.

Prerequisites

This How-To requires the following:

1. Completion of either How to Use VPS Colocalization with Netcode or How to Use Image Tracking Colocalization with Netcode.
2. Playback ready to use.
3. ParrelSync installed on your machine.

Debugging in Unity Editor using Playback and ParrelSync

ParrelSync is a Unity editor extension that lets you test multiplayer gameplay without building the project by opening a second Unity editor window and mirroring the changes from the original project.

To set up ParrelSync and start debugging:

1. From the ParrelSync menu, open Clones Manager. Click Create new clone in Clones Manager to create a clone project. This may take a while to complete.
2. In Clones Manager, click Open in New Editor to open the cloned project. When the original project is modified, the cloned project will also update.
3. Run the project in Play mode on every editor. Every instance should join the session and colocalize. Make sure all instances connect to the same Room.

Cross-Platform Debugging

Even if your application's AR tracking does not work well in the Unity Editor, debugging across platforms by using a device and the Unity Editor together is helpful for inspecting game states and the scene hierarchy in Unity while running AR multiplayer on a physical device. ARDK also supports a "Mock Colocalization" mode that lets you test and debug between the Editor and a device. (This mode only applies to networking functionality.)

To start a cross-platform test session:

1. Select XR Origin in the Hierarchy. In the Inspector window, find the Shared Space Manager component, then select Colocalization Type and choose MockColocalization.

   

2. In your script, test for the MockColocalization setting and set up the room with your payload value. For example:

if (_sharedSpaceManager.GetColocalizationType() ==
    SharedSpaceManager.ColocalizationType.MockColocalization)
{
    // Set room to connect
    var mockTrackingArgs = ISharedSpaceTrackingOptions.CreateMockTrackingOptions();
    var roomArgs = ISharedSpaceRoomOptions.CreateLightshipRoomOptions(
            "TestRoom",
            32,
            "Test room using mock colocalization"
        );
    _sharedSpaceManager.StartSharedSpace(mockTrackingArgs, roomArgs);
}

1. When you start the scene, the app should go straight to your UI for selecting Host or Client, both on the device and in the editor.

Debugging on Devices

See Sample Projects for instructions on how to download and install the Shared AR sample.

Debugging Colocalization by Showing a Shared AR Origin

Placing a cube at the origin in the scene that contains the Shared AR XROrigin is a useful technique for testing colocalization. By watching the cube's behavior, you can check if VPS and image tracking are working, find the real-world location of the Shared AR origin point, and test positional and rotational accuracy of VPS or image tracking between devices.

If the persisent anchor's location or rotation are different between devices, the devices are out of sync. If the anchor is in the correct place and orientation but the networked objects are not, the devices are in sync but the network objects were not placed in sync.

Niantic offers a prefab cube with directional arrows for this purpose. Look for OriginVisualizer in the Shared AR sample under Assets/Samples/SharedAR/Prefabs.

To show the prefab at the anchor location, add the following code snippet to the SharedSpaceManager tracking event handler:

[SerializeField]
private SharedSpaceManager _sharedSpaceManager;

[SerializeField]
private GameObject _sharedRootMarkerPrefab;

private void OnColocalizationTrackingStateChanged(SharedSpaceManager.SharedSpaceManagerStateChangeEventArgs args)
{
    if (args.Tracking)
    {
        Instantiate(_sharedRootMarkerPrefab,
            _sharedSpaceManager.SharedArOriginObject.transform, false);
    }
}

Networking Statistics

Network statistics are very useful when analyzing multiplayer issues such as latency and connection issues. The SharedAR package has a prefab called LightshipNetcodeTransportStatsUI that provides a UI panel with networking statistics, such as round-trip time to host, the amount of data sent, and the number of messages. This prefab is located at Packages/Niantic Lightship Shared AR Client Plugin/Assets/Prefabs.

To enable the network statistics UI:

1. Add the LightshipNetcodeTransportStatsUI prefab to your project's Assets folder.

2. If your scene does not contain a Canvas object, create one.

   1. Right-click in the Hierarchy under your AR scene, then open the Create menu, followed by UI, then select Canvas.

3. Drag the prefab from the Assets folder to your Canvas object.

4. In the Hierarchy, select the prefab, then find the Lightship Netcode Transport Stats Display in the Inspector. Check the following settings:

   1. Make sure Lightship Netcode Transport is set to NetworkManager (Lightship Netcode Transport).
   2. In the Text property, specify which text object you want the output to go to.
   3. In the BG Image property, set the background image for the Canvas. Use the Image component to modify the current image.
   4. To change the sample rate, update Sample Rate in Seconds. By default, the UI samples once per second. Note: higher sample rates can degrade network performance.
   5. Check Verbose Text if you would like more detailed information in the networking panel. The below is an example of regular vs verbose text:
   Regular Text

   

   Verbose Text

   

Troubleshooting

Connection Fails

• Make sure your API key is set properly.
• Make sure player count is less than the room capacity. Room capacity is set when calling ISharedSpaceRoomOptions.CreateVpsRoomOption() or ISharedSpaceRoomOptions.CreateLightshipRoomOptions().
• Make sure one player started as the Host and the other players joined as Clients.
• There is currently a known issue in which a Room becomes unjoinable if a user leaves the Room without explicitly calling the Leave() function multiple times. To work around this, delete the Room by calling RoomManagementService.DeleteRoom() and create a new one. When using Netcode, players automatically leave the Room when SharedSpaceManager and LightshipNetcodeTransport are destroyed, such as when leaving the scene.

Unexpected Disconnects

• Check device network connectivity.
• Currently, players will automatically be disconnected after 30 minutes regardless of activity.

Undelivered Messages

• The maximum payload size for INetworking.SendData() is 1KB. Messages that exceed 1KB will not be sent.
• The Lightship network can send 2000 messages per second per device. Client connections that exceed this limit continually will receive messages more slowly and will eventually be dropped by the server. Using Netcode makes reaching this limit more likely for host devices because the host sends a message to each client individually to update game state. We recommend limiting network update frequency or the number of players to reduce the chance of this happening.

Diagnosing Mobile Network Bandwidth Issues

• Mobile network bandwidth availability can change due to many factors, such as weather and geography. In situations where bandwidth is limited, players will observe latency.
• Upstream bandwidth is usually much less available than downstream, so be aware that messages to the server may be slow.
• Testing in the Unity Editor with a wired internet connection may not reveal potential bandwidth-related issues. On Mac and iOS, the Network Link Conditioner (opens in external window) can be used to simulate network conditions for testing.
• VPS tracking uses approximately 1.6Mbps of upstream bandwidth before reaching the tracking state. To avoid critical messaging and synchronization issues, wait until the application reaches the tracking state to begin synchronization.