class Provider

(Niantic.Lightship.AR.XRSubsystems.XRSemanticsSubsystem.Provider)

Overview

The provider which will service the XRSemanticsSubsystem.

    class Provider: SubsystemProvider< XRSemanticsSubsystem > {
    public:
        // fields
    
         ? uint LatestFrameId => throw new NotSupportedException("Getting the latest frame id is not supported by this implementation");
         bool IsMetadataAvailable => throw new NotSupportedException("Getting if metadata is available is not supported by this implementation");

        // properties
    
        uint TargetFrameRate;

        // methods
    
        virtual bool TryPrepareSubsystem();
    
        virtual bool TryGetSemanticChannel(
            string channelName,
            out XRTextureDescriptor semanticChannelDescriptor,
            out Matrix4x4 samplerMatrix,
            XRCameraParams? cameraParams = null
        );
    
        virtual bool TryAcquireSemanticChannelCpuImage(
            string channelName,
            out XRCpuImage cpuImage,
            out Matrix4x4 samplerMatrix,
            XRCameraParams? cameraParams = null
        );
    
        virtual bool TryGetPackedSemanticChannels(
            out XRTextureDescriptor packedSemanticsDescriptor,
            out Matrix4x4 samplerMatrix,
            XRCameraParams? cameraParams = null
        );
    
        virtual bool TryAcquirePackedSemanticChannelsCpuImage(
            out XRCpuImage cpuImage,
            out Matrix4x4 samplerMatrix,
            XRCameraParams? cameraParams = null
        );
    
        virtual bool TryGetChannelNames(out IReadOnlyList<string> names);
        virtual bool TrySetChannelConfidenceThresholds(Dictionary<string, float> channelConfidenceThresholds);
        virtual bool TryResetChannelConfidenceThresholds();
    };

Detailed Documentation

The provider which will service the XRSemanticsSubsystem.

Properties

TargetFrameRate

uint TargetFrameRate

Property to be implemented by the provider to get or set the frame rate for the platform's semantic segmentation feature.

The requested frame rate in frames per second.

    Parameters:

    System.NotSupportedException - Thrown when requesting a frame rate that is not supported by the implementation.

Methods

TryPrepareSubsystem

virtual bool TryPrepareSubsystem()

If the semantic segmentation model is ready, prepare the subsystem's data structures.

    Returns:

    true if the semantic segmentation model is ready and the subsystem has prepared its data structures. Otherwise, false.

TryGetSemanticChannel

virtual bool TryGetSemanticChannel(
        string channelName,
        out XRTextureDescriptor semanticChannelDescriptor,
        out Matrix4x4 samplerMatrix,
        XRCameraParams? cameraParams = null
    )

Method to be implemented by the provider to get the semantic channel texture descriptor and a matrix that converts from viewport to the texture's coordinate space.

    Parameters:

    channelName - The string description of the semantics channel that is needed.

    semanticChannelDescriptor - The semantic channel texture descriptor to be populated, if available.

    samplerMatrix - Converts normalized coordinates from viewport to texture.

    cameraParams - Describes the viewport.

    System.NotSupportedException - Thrown if the implementation does not support semantic channel texture.

    Returns:

    true if the semantic channel texture descriptor is available and is returned. Otherwise, false.

TryAcquireSemanticChannelCpuImage

virtual bool TryAcquireSemanticChannelCpuImage(
        string channelName,
        out XRCpuImage cpuImage,
        out Matrix4x4 samplerMatrix,
        XRCameraParams? cameraParams = null
    )

Tries to acquire the latest semantic channel CPU image.

    Parameters:

    channelName - The string description of the semantics channel that is needed.

    cpuImage - If this method returns true, an acquired XRCpuImage. The XRCpuImage must be disposed by the caller.

    samplerMatrix - Matrix to sample in pixel coordinates with, composed of display and warp matrix

    cameraParams - Params of the viewport to sample with

    System.NotSupportedException - Thrown if the implementation does not support semantic channels CPU images.

    Returns:

    Returns true if an XRCpuImage was successfully acquired.

TryGetPackedSemanticChannels

virtual bool TryGetPackedSemanticChannels(
        out XRTextureDescriptor packedSemanticsDescriptor,
        out Matrix4x4 samplerMatrix,
        XRCameraParams? cameraParams = null
    )

Gets a packed semantics texture descriptor.

    Parameters:

    packedSemanticsDescriptor - The packed semantics texture descriptor to be populated, if available from the provider.

    samplerMatrix - A matrix that converts from viewport to texture coordinates.

    cameraParams - Describes the viewport.

    System.NotSupportedException - Thrown if the implementation does not support packed semantics texture.

    Returns:

    true if the packed semantics texture descriptor is available and is returned. Otherwise, false.

TryAcquirePackedSemanticChannelsCpuImage

virtual bool TryAcquirePackedSemanticChannelsCpuImage(
        out XRCpuImage cpuImage,
        out Matrix4x4 samplerMatrix,
        XRCameraParams? cameraParams = null
    )

Tries to acquire the latest packed semantic channels XRCpuImage.

    Parameters:

    cpuImage - If this method returns true, an acquired XRCpuImage. The XRCpuImage must be disposed by the caller.

    samplerMatrix - A matrix that converts from viewport to texture coordinates.

    cameraParams - Describes the viewport.

    Returns:

TryGetChannelNames

virtual bool TryGetChannelNames(out IReadOnlyList<string> names)

Method to be implemented by the provider to get a list of the semantic channel names for the current semantic model.

    Parameters:

    names - A list of semantic category labels. It will be empty if the method returns false.

    System.NotSupportedException - Thrown when reading the channel names is not supported by the implementation.

    Returns:

    True if channel names are available. False if not.

TrySetChannelConfidenceThresholds

virtual bool TrySetChannelConfidenceThresholds(Dictionary<string, float> channelConfidenceThresholds)

Sets the confidence threshold for including the specified semantic channel in the packed semantic channel buffer.

Each semantic channel will use its default threshold value chosen by the model until a new value is set by this function during the AR session.

    Parameters:

    channelConfidenceThresholds - A dictionary consisting of keys specifying the name of the semantics channel that is needed and values between 0 and 1, inclusive, that set the threshold above which the platform will include the specified channel in the packed semantics buffer. The key must be a semantic channel name present in the list returned by TryGetChannelNames.

    System.NotSupportedException - Thrown when setting confidence thresholds is not supported by the implementation.

    Returns:

    True if the threshold was set. Otherwise, false.

TryResetChannelConfidenceThresholds

virtual bool TryResetChannelConfidenceThresholds()

Resets the confidence thresholds for all semantic channels to the default values from the current model.

This reverts any changes made with TrySetChannelConfidenceThresholds.

    Parameters:

    System.NotSupportedException - Thrown when resetting confidence thresholds is not supported by the implementation.

    Returns:

    True if the thresholds were reset. Otherwise, false.