How to Setup Real-World Occlusion

Lightship Occlusion creates depth in AR applications, rendering game objects in front of or behind objects in the real world. In ARDK 3.0, Lightship integrates seamlessly with the AR Foundation Occlusion Manager, enabling occlusion options not available in ARKit and ARCore.

Prerequisites

You will need a Unity project with ARDK installed and a set-up basic AR scene. For more information, see Installing ARDK 3 and Setting up an AR Scene.

Setting Up Occlusion

To setup Lightship occlusion:

1. Add an AROcclusionManager to your Main Camera GameObject:
   1. In the Hierarchy, expand the XROrigin and Camera Offset, then select the Main Camera object. Then, in the Inspector, click Add Component and add an AROcclusionManager.
2. Add a cube as a child of your Camera and set its position and scale:
   1. In the Hierarchy, right-click the Main Camera object, then mouse over 3D Object and select Cube.
   2. In the Inspector, under the Transform heading, set the cube's position to (0, 0, 2), its scale to (0.2, 0.2, 0.2), and its rotation to (0, 45, 45).
3. When you build to device or use playback, your cube will be occluded by physical objects that are less than 2 meters away from your phone.

Improving Occlusion Quality with the Lightship Occlusion Extension

By adding the Lightship Occlusion Extension, you can improve the visual quality of occlusions by adding functionality to the standard AROcclusionManager.

To add the extension and try it out:

1. Add a LightshipOcclusionExtension to the Main Camera GameObject.
   1. In the Hierarchy, expand the XROrigin and select the Main Camera. Then, in the Inspector, click Add Component and add a Lightship Occlusion Extension.
2. In the extension options menu, set the Optimal Occlusion Distance Mode to Specified Game Object.
3. Set the Cube you created earlier as the Principal Occludee.
4. When you build to device or run in playback, the edges of objects in the image should now line up more precisely with the occlusion boundaries of the cube.

Setting up Occlusion Suppression

Occlusion Suppression prevents pixels containing specified semantic information from occluding AR assets. Depth-based occlusion can be noisy and lead to inconsistencies with particular semantic channels. Enabling Occlusion Suppression can improve the visual quality of occlusions, particularly when AR characters appear to clip into the floor or disappear into the sky.

1. Follow the steps in Improving Occlusion Quality with the Lightship Occlusion Extension.
2. Add an ARSemanticSegmentationManager to the Main Camera GameObject.
   1. In the Hierarchy, expand the 'XROrigin' and select the Main Camera. Then, in the Inspector, click Add Component and add an AR Semantic Segmentation Manager to it.
3. In the Inspector, open the Lightship Occlusion Extension options menu, then check the box labeled Enable Occlusion Suppression. This will make new options appear.
4. Drag the Main Camera GameObject from the Hierarchy to the Semantic Segmentation Manager field in the Inspector.
5. In the Suppression Channels list, add sky for Element 0 and ground for Element 1.
6. Done! When you test your application, pixels corresponding to the ground or sky should not occlude your cube. Try using other semantic channels from the Semantics and Experimental Semantic Channels pages to see what happens!

Setting up Occlusion Stabilization

Occlusion Stabilization combines information from the instantaneous depth buffer and a depth field rendered from the world mesh to stabilize occlusions between frames. This leads to higher quality, more consistent occlusions in static parts of the scene.

1. Follow the steps in Improving Occlusion Quality with the Lightship Occlusion Extension.
2. Set up Meshing in your scene:
   1. Add an empty GameObject to the XROrigin. Name it Meshing.
   2. Select Meshing and add an ARMeshManager Component to it.
   3. Set the MeshPrefab in the ARMeshManager to FusedMesh (located in NianticLightshipARPlugin/Assets/Prefabs).
   4. [Optional] To configure advanced settings for meshing, add a LightshipMeshingExtension Component to the Meshing GameObject.
   5. [URP Only] If using the universal render pipeline, change the Shader for the FusedMesh Material to Lightship/FusedDepthChunkURP.
3. Enable Occlusion Stabilization:
   1. In the Inspector, open the Lighthsip Occlusion Extension options menu, then check the box labeled Enable Occlusion Stabilization. This will make new options appear.
   2. Drag the Meshing GameObject from the Hierarchy to the Meshing Manager field in the Inspector.

More Information

For more information, see Occlusion Features.