C++ Inference

CV C++ scripts use Helios::Inference::Session from the SDK included with the current Helios release. Helios uses the Compute GPU selected in Preferences.

Basic Flow

#include <helios/HeliosCVSDK.h>

class CVWorker {
    Helios::Inference::Session engine_;

public:
    CVWorker(std::uint32_t, std::uint32_t)
        : engine_(Helios::Inference::Session::create("MODEL_UUID")) {
        if (!engine_.valid()) {
            throw std::runtime_error("Could not create inference session");
        }

        engine_.setConfidenceThreshold(0.45f);
        engine_.setNmsThreshold(0.50f);

        if (!engine_.start()) {
            throw std::runtime_error("Could not start inference");
        }
    }

    void process(Helios::Frame& frame) {
        if (const HeliosInferenceResultsBlock* result =
                engine_.waitForResultsRaw()) {
            for (std::uint32_t i = 0; i < result->num_detections; ++i) {
                const HeliosInferenceDetection& detection =
                    result->detections[i];

                const float confidence = detection.confidence;
                const float centerX = detection.center_x;
                const float centerY = detection.center_y;
                (void)confidence;
                (void)centerX;
                (void)centerY;
            }
        }

        (void)frame;
    }
};

HELIOS_CV_SCRIPT(CVWorker)

Create and start the session once in the worker constructor. waitForResultsRaw() waits indefinitely by default so the result stays paired with the current frame. Pass a finite timeout only when the script intentionally accepts dropping a result after that deadline.

A session started during process() first participates on the next frame; an earlier wait returns no result. Check the returned pointer before reading it. No result is different from a completed result containing zero detections.

Session::create(uuid) creates a detection, pose, or segmentation session. Session::create() creates a session that can be configured for OCR without an object model.

Results

HeliosInferenceResultsBlock includes:

• frame_sequence, timestamp_ns, and inference_time_ms;
• model_type, frame_width, and frame_height;
• num_detections;
• up to 64 entries in detections across all classes.

Only read the first num_detections entries. Detection, pose, and segmentation share this global 64-result capacity. Each HeliosInferenceDetection provides:

• class id and confidence;
• box coordinates, size, center, and area;
• configured anchor plus polar distance and angle;
• predicted anchor, prediction offsets/input/motion, continuous detection age, prediction quality, and prediction flags;
• apparent_distance, a dimensionless size-derived relative distance; and
• 17 pose keypoints when the model supplies them.

Larger apparent_distance values represent smaller, apparently farther detections; they are not physical distance units. Configure the expected target height-to-width ratio with setApparentDistanceAspectRatio(...). The default ratio is 2.5.

The returned block is a borrowed view. Consume it during the current callback and do not retain its pointer.

Segmentation Results

After waitForResultsRaw() succeeds for a segmentation model, read the companion result:

const HeliosInferenceSegmentationResultsBlock* masks =
    engine_.segmentationResultsRaw();

if (masks) {
    for (std::uint32_t i = 0; i < masks->mask_count; ++i) {
        const std::uint8_t* plane =
            masks->masks + i * masks->mask_plane_stride;
        (void)plane;
    }
}

Mask i corresponds to detection i. Rows use mask_row_bytes; planes use mask_plane_stride. Pixels are packed least-significant-bit first, and the origin/scale fields map mask coordinates to the source frame. This block is also borrowed and must not be retained across frames.

Segmentation Settings

Use setSegmentationAnchorPoint(INFCORE_SEGMENTATION_ANCHOR_MASK, classIds, count) for mask-based anchors, or INFCORE_SEGMENTATION_ANCHOR_BBOX for the default box-based anchors. The existing anchor fractions and pixel offsets apply to the selected mode. In Mask mode, Y selects the nearest occupied row across the mask height and X selects a fraction across that row’s foreground span. X can lie over a hole; an empty mask excludes the detection. BBox mode permits empty masks.

setSegmentationMaskThreshold(threshold, classIds, count) accepts a value strictly between 0 and 1 (default 0.5). It changes published masks and mask anchors. setSegmentationMaskOpacity(opacity, classIds, count) accepts 0..255 (default 128) and affects drawing only. The existing global overloads remain available. Per-class values override global settings until changed or cleared with clearClassOverrides(...).

Common Settings

Colors passed to inference drawing calls use BGR order. Pose sessions also provide keypoint and skeleton drawing settings. Drawing is independent of returned results. The independent apparent-distance toggle is disabled by default. Confidence drawing also includes apparent distance and the selected target’s available motion speed.

The session supports class priorities, anchors, ignore regions, search/zoom controls, pausing, and model changes. setSearchStick(...) accepts stick 1 or 2, a radius from 0 through 100, and an optional less-than comparison. Search-button and stick triggers are alternatives; either activates inference. With neither configured, inference runs on every frame. Only the highest-priority tier represented by retained candidates is published; lower tiers do not backfill unused slots.

One session holds one object model. Use multiple sessions for multiple models and account for the added CPU/GPU load. Helios supports up to 30 simultaneously started inference sessions in total, and each session supports up to 16 OCR regions. A pause or resume requested during a frame takes effect with the next frame, and a wait during that transition returns immediately.

SDK and ABI Compatibility

Use the complete SDK under versions/<version>/sdk/ from the installed Helios release. Keep all headers from the same release and include HeliosCVSDK.h for normal CV C++ scripts. Python scripts do not compile against the native ABI.

Build against the public SDK packaged with Helios. Existing public calls remain compatible as new functions are added. A script that uses a newer function requires a Helios release that provides it; update the runtime and SDK together when adopting new features.

HELIOS_CV_SCRIPT(CVWorker) generates the three lifecycle exports helios_cv_script_create, helios_cv_script_process, and helios_cv_script_destroy. Low-level consumers should use InferenceCore.h, InferenceCoreApi.h, and the matching result headers under sdk/include/helios instead of copying declarations. The complete callable reference is included under sdk/docs.

Vision and Meter continue to gain features while preserving published call signatures and data layouts. Use the current SDK for newly added calls.

Anchor Prediction

For workflows that compensate an anchor using measured camera motion, configure an InfcoreAnchorPredictionConfig and pass it to:

InfcoreAnchorPredictionConfig config{};
config.struct_size = sizeof(config);
config.x_enabled = 1;
config.y_enabled = 1;
config.response_delay_ms = 8.0f;
config.lead_x = 1.0f;
config.lead_y = 1.0f;

engine_.setAnchorPrediction(config);

Provide motion samples with:

engine_.recordAnchorPredictionMotion(timestampNs, velocityX, velocityY);

Use the same monotonic nanosecond time domain as the frame timestamp. X and Y velocities are measured in screen-heights per second. Enable only the axes the script can measure reliably.

Use setApparentDistanceZoom(zoomFactor) to compensate relative distance for external camera/application zoom: 1.0 is normal view and 2.0 is 2× magnification. The factor must be finite and positive. Helios ROI cropping needs no additional zoom compensation.

setAnchorPredictionDistanceLead(millisecondsPerDistanceUnit) adds optional relative-distance prediction time, accepting finite 0..1000 with a default of 0. Calibrate it for the application; it does not express physical distance or gravity. getDetectionMotion(detection, motion) fills an InfcoreTargetMotion with velocity_x, velocity_y, speed, and prediction_flags. Velocity is relative to the background in frame-heights per second, positive right/down. Check the VALID flag before using it and distinguish a current camera measurement from a continuing estimate. Read motion from the detection accessor rather than deriving it from prediction offsets.

Camera Motion

setCameraMotionEnabled(true) enables optional background observations. Read them with getCameraMotion(InfcoreCameraMotion&); disable the feature when unused. The returned timestamps identify the observed frame interval. Displacement is background movement divided by frame height, positive right/down; variance uses squared normalized units.

Check valid_axes separately for X (bit 1) and Y (bit 2), timestamp freshness, and duplicate intervals. Disabled, stopped, paused, or unavailable observations have zero validity. Enabling or resuming requires a fresh frame pair. Camera movement has the opposite sign to background displacement.

For text recognition, use the OCR methods on the same session.

Model Precision

DirectML accepts object models with FP16 or FP32 inputs and outputs and uses their declared types automatically. No user precision toggle is needed. The model description reports its input/output precision. TensorRT precision is selected when building an engine; its model inputs and outputs must remain FP32. See Model Compatibility.

Practical Guidance

• Use the HeliosCVSDK.h and ABI headers included with the running Helios version.
• Create and start a session outside process().
• Consume raw result views promptly and only through their documented fields.
• If a wait returns no result unexpectedly, check engine_.runtimeError() and the Output Panel; a hard runtime failure is not an ordinary zero-detection frame.
• INFCORE_ERROR_INSTALLATION_INVALID (18) means the installation could not be verified. Install the latest complete official release and check whether security software quarantined a packaged file.
• Begin with the model's recommended ROI and class-priority settings.
• Report persistent setup failures once rather than once per frame.
• Check the Output Panel when model startup fails.