ANNOTATION

Multimodal robot data, aligned with human understanding.

PRIVATE PREVIEW · AUTOMATED PROPOSALS · EXPERT REVIEW

Annotation is the timeline-centred workbench inside Zen Matrix. It keeps video, robot state, human command and task meaning together so every label can be reviewed in its physical context.

ONE TIMESTAMP, THE COMPLETE ROBOT CONTEXT

At any point in an Episode, reviewers can inspect the information that produced an action.

t = 12.480 s

  • Front RGB and wrist-camera frames
  • Joint position, velocity and torque
  • End-effector position and quaternion
  • Gripper opening and discrete mode
  • Force / torque and contact signal
  • IMU and teleoperation command
  • Alignment confidence and nearest-frame time error

ANNOTATION LAYERS

Task and skill

Task definition · skill primitive · subtask phase · operator intent · natural-language instruction

Scene and object

Object identity · pose · state change · spatial relation · occlusion · camera visibility

Interaction and event

Grasp · release · insertion · contact · collision · mode switch · recovery · human intervention

Outcome and quality

Success · failure · partial completion · anomaly · unsafe action · low-confidence segment · rejection reason

AUTOMATION WITH HUMAN CONTROL

01 · Automated proposals

Models and agent APIs can propose task boundaries, object references, events, captions, anomalies and quality flags. Every proposal records its model, prompt, timestamp and confidence.

02 · Multimodal review

Reviewers compare the proposed label against synchronized camera, action, state and force signals instead of judging a single image in isolation.

03 · Approval workflow

Filter low-confidence segments, assign reviewers, add comments, approve revisions and retain a traceable audit history.

04 · Dataset release

Freeze an annotation version together with the aligned dataset, calibration, quality report and export schema.

WHAT THE WORKSPACE MAKES VISIBLE

  • Episode-level and frame-level annotations
  • Task ontology and reusable skill taxonomy
  • Boundary editing directly on the synchronized timeline
  • Success / failure trajectory handling
  • Automated anomaly detection and human corrections
  • Reviewer status, confidence and disagreement
  • Dataset version, lineage and approval state

QUALITY METRICS: label coverage · boundary error · reviewer agreement · auto-label acceptance · unresolved anomalies · missing modality rate

EXPORT: REF · LeRobot · RLDS · HDF5 · Zarr · project-specific training schema

FROM RAW EPISODES TO TRUSTED TRAINING DATA

INGEST · SYNCHRONIZE · PROPOSE · REVIEW · APPROVE · EXPORT

See the complete Zen Matrix governance workflow →# ANNOTATION

Multimodal robot data, aligned with human understanding.

Private Preview

Annotate robot episodes across video, actions, states, events and task semantics on one synchronized timeline. Annotation is a core workspace inside Zen Matrix, designed for both automated agents and expert human review.

What the workspace connects

Video and sensor context

RGB, depth, force-torque, IMU, joint state, end-effector pose, gripper state and teleoperation commands remain synchronized throughout review.

Task and event structure

Create task boundaries, subtask segments, success and failure labels, contact events, object states, anomalies and natural-language descriptions.

Automated annotation

Use model and agent APIs to propose task segments, events, object references and quality flags before human review.

Human-in-the-loop review

Review low-confidence segments, correct timestamps, compare modalities and maintain traceable approval history.

Built for governed datasets

  • Episode-level and frame-level annotation
  • Unified timeline across asynchronous topics
  • Confidence scores and reviewer status
  • Dataset lineage and version history
  • Quality checks for missing data, drift and timestamp error
  • Export to REF, LeRobot, RLDS, HDF5 and Zarr

Workflow

INGEST · SYNCHRONIZE · AUTO-LABEL · REVIEW · APPROVE · EXPORT

The public website presents the workflow. Access to the live workspace is provided through private customer previews and deployment projects.