Skip to main content

Meshing

The meshing feature uses depth and tracking data to generate a mesh representing the estimated geometry of the scanned real world.

By placing the standard ARMeshManager (opens in new window) in a scene, developers can access a live mesh that allows virtual objects to interact with the real world environment. For example, a virtual ball thrown into a meshed scene will realistically bounce off of the floor and walls.

When the Niantic Lightship SDK is enabled in Unity, meshing is still provided through Unity's AR Foundation Meshing Subsystem (opens in new window) and enabled with the standard ARMeshManager (opens in new window). Lightship overrides the default implementation and provides Niantic's proprietary meshing technology through the standard interface.

note

Meshing will work on both lidar and non-lidar devices. This feature will also work on both Android and iOS platforms. For support on lidar devices, please ensure an [AROcclusionManager] (https://docs.unity3d.com/Packages/com.unity.xr.arfoundation@5.0/api/UnityEngine.XR.ARFoundation.AROcclusionManager.html) (opens in new window) is present in your scene.

Falling Objects on the Gandhi Statue

Mesh Blocks​

The 3D world is divided into "blocks" of a fixed size, with the name of each mesh block corresponding to its position in the 3D world. When Meshing is running, new mesh blocks will be created in the scene Hierarchy underneath XROrigin > Trackables. Each block has its own renderer and collider defined by the Mesh Prefab specified in the ARMeshManager. Mesh blocks are continually updated as new data is added to the 3D representation.

Meshing Extensions​

Lightship's meshing system exposes extra options to allow you to tweak meshing rules for distance, quality, and clean-up.

Once you add the Lightship Meshing Extension script to the Meshing object, you will have access to these settings:

Lightship Meshing Extension settings
  • Target Frame Rate: The number of times per second to run the mesh update routine.
  • AR Fusion Parameters:
    • Maximum Integration Distance: The far distance threshold (in meters) for integrating depth samples into the 3D scene. New mesh blocks will not be generated further than this distance from the camera.
    • Voxel Size: The size (in meters) of individual voxel elements in the scene. Higher values will save memory but also reduce the precision of the surface.
    • Enable Distance Based Volumetric Cleanup: Enable this to save memory and smooth latency by cleaning up already processed elements in the volumetric representation (once they move outside the region where new meshes are generated).
  • AR Meshing Parameters:
    • Mesh Block Size: The size of the mesh blocks used for generating the mesh filter and mesh collider.
    • Mesh Culling Distance: The distance where mesh blocks will be removed from the scene. Set to 0 to disable culling.
    • Enable Mesh Decimation: Enable to save memory be removing excess triangles from the mesh.
  • Mesh Filtering:
    • Is Mesh Filtering Enabled: Check this box to enable mesh filtering.

Mesh Filtering​

Mesh Filtering uses semantic segmentation to identify sections of a mesh as common parts of the world, such as ground or sky, then uses that information to determine what should be part of the final mesh with a user-defined allowlist and blocklist. For example, a blocklist containing sky would remove the sky from the final mesh, while an allowlist containing ground would exclude everything but the ground. See How to Exclude Semantic Channels with Mesh Filtering for more information.

Example usage of a semantic filtering allowlist
Example of Mesh Filtering being turned on and off

Long Distance Meshing​

For example, you can configure your application to mesh much larger distances by increasing the Voxel Size, turning on Enable Distance Based Volumentric Cleanup, and increasing Maximum Integration Distance and Mesh Culling Distance between 20 and 40 meters:

caution

These settings are only recommended for high-end devices. For information on supported devices, see the Google ARCore device list and Apple ARKit device list.

  • Target Frame Rate: 20
  • AR Fusion Parameters:
    • Maximum Integration Distance: 40
    • Voxel Size: 0.05
    • Enable Distance Based Volumentric Cleanup: True
  • AR Meshing Parameters:
    • Mesh Block Size: 1.4
    • Mesh Culling Distance: 40 (should be >= Maximum integration distance)
    • Enable Mesh Decimation: True

Note, you sholud also increse Concurrent Queues Size on the ARMeshManager when increaseing meshing distance to ensure it can keep up with the number of tiles that are surfaced. However this will increase cpu/gpu usage so you will need to tweak it for your scenario. For LDM >20 on high end devices is recommended to get best quality results.

Long Distance Meshing exampleLong Distance Meshing example

Lidar Devices​

On lidar devices, it is possible to use Lightship Meshing with either lidar depth or Lightship depth. To use lidar depth, ensure that Prefer LiDAR if Available is enabled in the Niantic Lightship Settings menu (Lightship top menu > Settings) and that an [AROcclusionManager] (https://docs.unity3d.com/Packages/com.unity.xr.arfoundation@5.0/api/UnityEngine.XR.ARFoundation.AROcclusionManager.html) (opens in new window) is present in the scene. (Note that meshing will still be supported even if No Occlusion is selected as the Occlusion Preference Mode.)

Lidar depth will only support a maximum integration distance of around five meters. If you wish to generate mesh blocks further away from the user, disable Prefer LiDAR if Available to use Lightship Depth instead.