Lightship VPS
With Lightship's Visual Positioning System (VPS) you now have the power to determine a user’s position and orientation with centimeter-level accuracy - in seconds. You can use Lightship VPS to create location-based AR experiences that connect the real world with the digital one. Content can be anchored to these locations so they can persist over AR sessions. VPS makes augmented reality feel more relevant to the communities that live within these locations and gives users new reasons to explore the world around them.
This is a Public Beta feature and functionality may change in a future release. For more information on ARDK feature launch types like Public Beta, see Lightship Launch Types.
What is VPS?
Niantic's Visual Positioning System (VPS) is a cloud-based system that allows devices to know where they are relative to the Lightship map. By determining the six-degrees-of-freedom pose of a device relative to the map, you can place persistent virtual objects that stay aligned with the real world.
The VPS Map
The Lightship VPS map is built using scans submitted by users and is organized by the coverage of the locations in the scans. When enough scans have been submitted for a location, the location becomes "VPS-Activated". See How to Create a Public Location for more information on how to create VPS-Activated locations.
Once enough scans of an area are collected, they are processed to create a map of that area. Connected nodes on a map form a space, and those spaces make up each VPS-Activated Location. The highest-quality space of a VPS-Activated location will be saved as the default space for that location with each default space having a default anchor associated with it. The VPS Coverage API provides the default anchor as part of the LocalizationTarget. Locations downloaded from the Geospatial Browser also provide a default anchor in the mesh zip file that can be loaded into Unity as an ARLocation.
Anchors
Each anchor represents a stable point and orientation in the real world. Anchors can be stored as a payload and resolved across sessions to create persistent virtual objects. The highest abstraction layer of an anchor is the ARLocation. See How to Place Content in Real-World Locations Using Location AR to learn how to use ARLocations in Unity.
The Localization Process
Localization is the process of resolving an anchor's position and orientation in a device's local tracking coordinate system. To localize, the user needs to point their device at the real-world location associated with that anchor. Lightship then sends the camera frames to the cloud and looks for a match. (If users are having trouble, the VPS Coverage API provides a field to add a hint image for them to look for.)
Because device local tracking is inexact, anchor positions can drift after localizing. To combat this, Lightship offers Location Drift Mitigation features that use a variety of techniques to recompute the correct anchor position.
There are a number of other factors that can also impact localization quality. Moving too fast, darkness or fog, dirty camera lenses, and poor internet connection can all impact the speed and quality of localization. If local tracking fails or jumps suddenly to another place, the anchor will become "Untracked" and the user will need to localize again.
Troubleshooting
If localization is failing, try these steps:
- Check the internet connection and location permissions. Localization requires location permissions to work.
- Verify that the
ARLocationcorresponds to the correct real-world location and that the device is pointed at the hint image. - If you are trying to localize at night, try again during the day when there is more light.
- Try another device to rule out local settings issues.
If none of these suggestions work, you can create a Playback dataset and set it up in Unity to test your content.
Reach out if you need further help with this or any other Lightship content!
Creating Location AR Experiences with VPS
You can use VPS to do the following:
- Using one or a handful of Public Locations from the Geospatial Browser, author content against a 3D mesh within Unity.
- At runtime, allow end-users to place content relative to any Public Location.
- Create and use a Test Scan for prototyping.
With Shared AR’s VPS Colocalization you can also enable multiple users to easily join the same session and play within a Public Location.
One example use case for VPS is a geo-caching AR game. Players can localize their devices at nearby Public Locations. You can create and "hide" virtual objects at the location for other players to find.
Location AR currently provides these features:
- Public Locations are unique or notable, publicly accessible, real-world locations that are VPS-Activated and apps can engage with.
- The VPS Coverage API which lets you discover Public Locations that are VPS-Activated.
- VPS Coverage Areas are geographic regions where users can localize with VPS and show the area around a Public Location where users can localize.
- VPS Localization Targets are real-world images of Public Locations that users can track with their device to localize.
- The Geospatial Browser (Beta) allows you to search, nominate, and find Public Locations across the globe. You can select the location you want to use for your project and download the associated 3D mesh assets.
- Test Scans are VPS locations in your surrounding area to test with using the Niantic Wayfarer App.
- AR Location Manager allows you to easily persist content at Public Locations. You can download a mesh from the Geospatial Browser, import it into Unity, and add an AR Location within the AR Location Manager. Then you can drag and drop content onto the 3D mesh and find it in the location later.
Niantic also provides the following to help you develop your VPS-enabled AR experience:
- The Niantic Wayfarer App allows you to capture and submit scans at Public Locations to improve the coverage of Lightship VPS. You can also use the app to create Private VPS Locations for your own rapid testing and development. Finally, the Wayfarer App can be used to test localization at Public Locations and Private VPS Locations.
- A VPS developer portal on lightship.dev that lets you visualize Public Locations and manage Private VPS Locations.
Lightship VPS APIs make requests to the VPS backend, which needs to receive user information such as user location and an ID unique to the user using your app to make the requests. To facilitate this, ARDK provides a User ID data element that’s automatically used in Lightship VPS requests. This User ID is considered private user data, and your app will need to provide privacy policies and guidance for handling private user data, as described in ARDK and Data Privacy.