User Manual

ARDK Setup

• Getting Started with ARDK - Get started with developing AR experiences with ARDK.

• ARDK Downloads - Downloads page for ARDK.

• Adding ARDK to Unity - How to import ARDK into Unity.

• Authentication - How to retrieve an authentication token configured with your API key.

• Building for iOS - How to build and deploy your ARDK app to iOS devices.

• Building for Android - How to build and deploy your ARDK app to Android devices.

• Using ARDK’s Feature Managers - The Manager components provide simple Unity-friendly ways to access ARDK features.

• Subscribing to ARDK Events - Get notified when AR and networking events relevant to your experience occur.

• Development and Runtime Requirements - ARDK requirements for Mac, iOS, and Android.

• Validating Device Requirements - How to check a device for AR support and update software.

• Permissions - How to handle device permissions required for some ARDK features.

• Using ARDK User IDs - How to maintain data privacy best practices by setting User ID information used in ARDK requests.

• Upgrading ARDK - How to upgrade your Unity ARDK project to a new version of ARDK.

• Using ARDK Logging - How to use ARDK logging in your Unity project.

AR Fundamentals

• Creating an AR Session - How to create an AR experience with the ARSession class.

• Tracking AR Anchors - Designate and track points of interest in the physical environment.

• Detecting Planes - Detect and interact with real-world planes to create a more immersive AR experience.

• Detecting Images - Search for images in the real world and use them for a variety of AR effects.

• Hit-Testing - Using a screen location, find points on real-world surfaces and objects.

• Tutorial: Basic Placement - Learn how to use basic AR features to place virtual objects on floors and walls.

Contextual Awareness

Depth

• Generating Depth Data - Take advantage of depth maps to create immersive AR experiences.

• Depth-Based Occlusions - Place or move content behind real world objects without breaking immersion.

• Point Clouds - Turn a depth buffer into a world-space point cloud.

• Tutorial: Basic Occlusions - Learn how to use depth to do basic object occlusions.

• Advanced Tutorial: Depth Textures - Learn how to extract the depth buffer into a texture and then overlay the texture on the screen using a full screen shader.

Semantic Segmentation

• Semantic Segmentation - Identify real world features in your scene.

• Tutorial: Basic Semantics - Learn how to do basic inspections of the semantic buffer to determine what types of objects the user can see.

• Tutorial: Semantic Texture Using CopyToAligned - Learn how to do get an aligned version of the semantics texture using the CopyToAligned API.

• Advanced Tutorial: Semantic Segmentation Textures - Learn how to extract masks from the segmentation system and use these to overlay simple effects on the detected objects in the scene, like changing the color or pattern of the sky.

Meshing

• Meshing Overview - Understand what meshing is, what it’s best suited for, and how to best leverage its capabilities.

• Getting Started - How to setup meshing or add meshing to an existing scene.

• Advanced Tutorial: Meshes in the Unity Editor - How to save a mesh as a file and reuse it on a device.

• Advanced Tutorial: Garden Chunks - Build a simple custom AR meshing experience that turns the real world into a garden.

• Advanced Tutorial: Meshing and Collisions - Learn how to use meshing and collide objects with the generated mesh

• Addendum: Low Level Meshing - See an example memory layout of a low level mesh.

Gameboard

• Gameboard - Use Gameboard to query the environment for object placement, procedural gameplay and character navigation.

• Advanced Tutorial: Gameboard - Walk through the steps for adding Gameboard to your AR experience.

Model Cards

• Model Card: Person Segmentation - Understand the machine learning model used by the person segmentation feature of semantic segmentation.

Shared AR

• Shared AR Overview - Use Lightship ARDK to create AR experiences that multiple users can participate in

• Getting Started with Shared AR - Start creating shared AR experiences

• Setting Up a Networked AR Session - Enable nearby players to join the same AR experience.

• Localizing With a Shared Environment - Synchronize clients to a common object in their environment to ensure all clients can interact in a shared AR environment.

• Localizing With Marker Sync - Synchronize clients using CV objects (such as QR codes) for fast join and sync in a shared AR environment.

• Sharing Virtual Object Information - Share data between clients using the ARDK networking APIs

• Understanding Networking and LLAPI - Using ARDK’s low-level peer-to-peer networking API for a fluid multiplayer experience.

  • Additional LLAPI Features - Learn about additional networking features available in the ARDK.

  • Networking Limits and Best Practices - Some networking nuances to keep in mind when creating networked logic.

  • Networking Error Codes - A list of the possible error codes surfaced through the IMultipeerNetworking.ConnectionFailed event.

  • Serializing Data - Learn how to use helper functions to serialize data for IMultipeerNetworking.

  • Tutorial: Pong - Creating a Multiplayer AR Game - An example Unity project in which ARDK’s LLAPI is used to create an AR Multiplayer version of Pong.

• Using the High-level networking API - Quickly create distributed, stateful gameplay with ARDK’s built-in high level networking API.

  • HLAPI Network Spawning - How network spawning works with the High-level network API.

  • HLAPI FAQ - Frequently Asked Questions about the High-level networking API.

  • Tutorial: Pong - HLAPI Version - How to create an AR ping-pong game using HLAPI.

• Using both HLAPI and LLAPI - Things to consider if you have to use both HLAPI and LLAPI in your AR experience.

• Placing Virtual Content - Understand how to place and orient virtual content in a Shared AR experience.

Lightship VPS

• Lightship VPS - Create immersive experiences in which your users interact with persistent, shareable AR content at real-world locations.

• Getting Started - Get a quick overview of the steps you’ll need to take to start developing with Lightship VPS.

• Geospatial Browser (Beta) - Search and find VPS-activated Wayspots across the globe.

• Managing Private Scans - Use private VPS locations in your surrounding area to test with using the Niantic Wayfarer App.

• Creating New VPS-Activated Wayspots - Submit Wayspot locations to Niantic.

• Authoring Wayspot Anchors in Unity - Create Wayspot Anchors and save them to your Unity file or project, so that you can easily create experiences for VPS end-to-end, without needing to leave your home or office.

• Building and Testing Apps with VPS - Tips on how to build and test your VPS apps.

• Generating Scans for VPS - Use the Niantic Wayfarer app to view Niantic Wayspots, create Private Scans for Private VPS Locations for testing purposes, and test localization with VPS-activated Wayspots.

• Using the VPS Coverage API - Use the VPS Coverage API to discover VPS coverage areas and VPS localization targets.

• Localizing with VPS - Synchronize users to a VPS-activated Wayspot.

• Using VPS Wayspot Anchors - Use the VPS Wayspot Anchors API to precisely place virtual objects that maintain a consistent, stable pose in an AR environment. Wayspot Anchors can also be restored in future VPS sessions and shared between users of your app.

  • Using WayspotAnchorService - Use WayspotAnchorService to create and restore Wayspot Anchors.

  • Using WayspotAnchorController - Use WayspotAnchorController to create and restore Wayspot Anchors.

Rendering

• Rendering in ARDK - Customize how the camera feed is rendered in your app.

• Using the Universal Render Pipeline - How to use ARRenderingManager with the Universal Render Pipeline.

Tools

• Virtual Studio - Improve the experience of developing with ARDK using Virtual Studio.

  • Playing in Mock Mode - Use Mock mode to test without a mobile device.

  • Playing in Remote Mode - Use Remote mode to test on a device without redeploying to it.

  • VPS Remote Authoring - Use VPS Remote Authoring to author VPS content within Unity.

See Also

• Unity API reference documentation