Skip to main content

Meshing

The meshing feature uses depth and tracking data to generate a mesh representing the estimated geometry of the scanned real world.

By placing the standard ARMeshManager (opens in new window) in a scene, developers can access a live mesh that allows virtual objects to interact with the real world environment. For example, a virtual ball thrown into a meshed scene will realistically bounce off of the floor and walls.

When the Niantic Lightship SDK is enabled in Unity, meshing is still provided through Unity's AR Foundation Meshing Subsystem (opens in new window) and enabled with the standard ARMeshManager (opens in new window). Lightship overrides the default implementation and provides Niantic's proprietary meshing technology through the standard interface.

note

Lightship Meshing works on lidar and non-lidar devices running either Android or iOS. For support on lidar devices, ensure the Prefer LiDAR if Available setting is checked in Lightship Settings and an AROcclusionManager (opens in new window) is present in your scene.

Falling Objects on the Gandhi Statue

Mesh Blocks​

The 3D world is divided into "blocks" of a fixed size, with the name of each mesh block corresponding to its position in the 3D world. When Meshing is running, new mesh blocks will be created in the scene Hierarchy underneath XROrigin > Trackables. Each block has its own renderer and collider defined by the Mesh Prefab specified in the ARMeshManager. Mesh blocks are continually updated as new data is added to the 3D representation.

Meshing Extensions​

Lightship's meshing system exposes extra options to allow you to tweak meshing rules for distance, quality, and clean-up.

Once you add the Lightship Meshing Extension script to the Meshing object, you will have access to these settings:

Lightship Meshing Extension settings
  • Target Frame Rate: The number of times per second to run the mesh update routine.
  • AR Fusion Parameters:
    • Maximum Integration Distance: The far distance threshold (in meters) for integrating depth samples into the 3D scene. New mesh blocks will not be generated further than this distance from the camera.
    • Voxel Size: The size (in meters) of individual voxel elements in the scene. Higher values will save memory but also reduce the precision of the surface.
    • Enable Distance Based Volumetric Cleanup: Enable this to save memory and smooth latency by cleaning up already processed elements in the volumetric representation (once they move outside the region where new meshes are generated).
  • AR Meshing Parameters:
    • Mesh Block Size: The size of the mesh blocks used for generating the mesh filter and mesh collider.
    • Mesh Culling Distance: The distance where mesh blocks will be removed from the scene. Set to 0 to disable culling.
    • Enable Mesh Decimation: Enable to save memory be removing excess triangles from the mesh.
  • Mesh Filtering:
    • Is Mesh Filtering Enabled: Check this box to enable mesh filtering.
  • Experimental Meshing Options:
    • Enable Levels of Detail: Check this box to enable the experimental level of detail meshing feature which saves memory and reduces latency. For more information, see the Meshing Level of Detail feature page.

Mesh Filtering​

Mesh Filtering uses semantic segmentation to identify sections of a mesh as common parts of the world, such as ground or sky, then uses that information to determine what should be part of the final mesh with a user-defined allowlist and blocklist. For example, a blocklist containing sky would remove the sky from the final mesh, while an allowlist containing ground would exclude everything but the ground. See How to Exclude Semantic Channels with Mesh Filtering for more information.

Example usage of a semantic filtering allowlist
Example of Mesh Filtering being turned on and off

Long Distance Meshing​

caution

These settings are only recommended for high-end devices. For information on supported devices, see the Google ARCore device list and Apple ARKit device list.

You can configure your application to mesh over much larger distances by increasing the Voxel Size, turning on Enable Distance Based Volumetric Cleanup, and increasing the Maximum Integration Distance and Mesh Culling Distance to between 20 and 40 meters. For example, try out the following settings:

  • Target Frame Rate: 20
  • AR Fusion Parameters:
    • Maximum Integration Distance: 40
    • Voxel Size: 0.05
    • Enable Distance Based Volumetric Cleanup: True
  • AR Meshing Parameters:
    • Mesh Block Size: 1.4
    • Mesh Culling Distance: 40 (should be >= Maximum integration distance)
    • Enable Mesh Decimation: True

If you use these settings, increase the Concurrent Queue Size setting in the ARMeshManager Component as well to make sure that the manager can keep up with the amount of tiles being surfaced. This will increase CPU and GPU usage, so we recommend testing thoroughly on a variety of devices.

Long Distance Meshing exampleLong Distance Meshing example

Lidar Devices​

On lidar devices, it is possible to use Lightship Meshing with either lidar depth or Lightship depth. To use lidar depth, ensure that Prefer LiDAR if Available is enabled in the Niantic Lightship Settings menu (Lightship top menu > Settings) and that an AROcclusionManager (opens in new window) is present in the scene. (Note that meshing will still be supported even if No Occlusion is selected as the Occlusion Preference Mode.)

Lidar depth will only support a maximum integration distance of around five meters. If you wish to generate mesh blocks further away from the user, disable Prefer LiDAR if Available to use Lightship Depth instead.