CERES user guide

Data description and output format

Technical documentationUpdated 2026-08-08

CERES exports one immutable LeRobotDataset v3 shard per episode. A browser export stages the result below ceres-lerobot-v3/sessions/<session-id> in browser origin-private storage. Folder export and Hugging Face sync use the same artefacts and paths.

Episode layout

Episode indices are zero-padded and form a self-contained shard prefix:

shards/episode-000017/
  data/chunk-000/file-017.parquet
  meta/episodes/chunk-000/file-017.parquet
  meta/info.json
  meta/stats.json
  meta/tasks.parquet
  videos/observation.images.passthrough/chunk-000/file-017.mp4
  ceres/episode-metadata.json
  ceres/task-specifications/<sha256>.json
  ceres/browser-export-receipt.json

The video file is present when an outward-camera video exists. The task-specification sidecar is present for recordings that carry a canonical task specification. All paths inside the shard are relative dataset artefact paths, not browser or headset paths.

LeRobot data parquet

data/chunk-.../file-....parquet stores one row for every retained recorder slot. It has the following columns:

observation.state: a 410-value float32 vector.
action: a two-value float32 vector containing left_hand.pinch_distance and right_hand.pinch_distance.
timestamp: compact output time in seconds, calculated from the exported frame index and dataset FPS.
frame_index: zero-based frame index inside the exported episode.
episode_index: zero-based episode index in the dataset.
index: zero-based global dataset frame index.
task_index: stable index into meta/tasks.parquet.
ceres.source_frame_index: the original recorder-frame index before retry ranges were removed.
ceres.source_timestamp: the original monotonic source time in seconds.
ceres.source_gap: true when the row is an explicit missing-observation slot.

meta/info.json declares these feature names, vector shapes, data types, FPS, path templates and the v3.0 LeRobot compatibility version. meta/stats.json stores population minimum, maximum, mean, standard deviation and count for every exported feature.

Observation state

observation.state is named in meta/info.json and always has 410 values:

head.tracked                                      1
head.position.{x,y,z}, head.rotation.{x,y,z,w}    7
left_hand.tracked                                 1
left hand: 25 joints x position, rotation, radius 200
right_hand.tracked                                1
right hand: 25 joints x position, rotation, radius 200

Each hand uses the standard WebXR joints from wrist through pinky-finger-tip. Each joint contributes position x, y, z, rotation quaternion x, y, z, w and radius. When a hand or joint is not tracked, its numeric state values are exported as zero and the associated tracking flag identifies that state. A gap row has all tracking flags cleared and both pinch-distance action values set to zero.

Tasks, segments and retries

meta/tasks.parquet maps each stable task_index to the task text. meta/episodes/...parquet describes the episode length, global dataset index range, task texts, data locator, video locator when present and per-feature statistics.

ceres/episode-metadata.json uses the ceres-episode-export-metadata schema, version 3. It records:

the CERES episode identifier and exported episode index;
the selected camera, dimensions, frame rate, capture device characteristics and recorder rate;
optional study-assigned headset, demonstrator and project identifiers, consent date and consent-document identifier, plus whether raw microphone audio was retained;
camera calibration when available, including its pinhole-radtan5 intrinsics, distortion coefficients, RMS error, sample count and capture time;
retained task segments with cycle, repetition, take, annotations, outcome, durable frame and gap counts, plus source start and end timestamps;
the version, SHA-256 and path of the canonical task specification when one was attached.

Segment outcome retry is preserved in the durable source journal but does not appear in the exported row ranges or output MP4. The next successful or completed attempt supplies the retained segment for that repetition.

Capture metadata uses explicit availability values. known carries an observed value, unknown means the platform did not expose it and unavailable means the relevant capability was not active. The sidecar does not include raw camera device identifiers or browser user-agent strings.

Study metadata is optional. A blank demonstrator ID is replaced at capture arming by a readable designator in the form family,,animal,material. Its family word-pair is derived from the final two bytes of the study headset ID when one is provided. The animal and material are generated randomly. This lets a study distinguish recordings without using a participant name.

Video and audio

videos/observation.images.passthrough/...mp4 is the outward camera's aligned H.264 MP4. When the source is already aligned, CERES remuxes it without recompression. Range-aware processing is used only when retry footage must be removed or input media is incompatible.

The export video is silent. If enabled, raw microphone audio remains in the durable CERES capture source but is not mixed into the LeRobot MP4. This separation lets an ethics or data-governance workflow control audio independently.

Integrity receipt

ceres/browser-export-receipt.json enumerates every shard artefact with its relative path, SHA-256 digest, byte length and media type. Use it to verify a copied folder or upload manifest. It is an export-integrity receipt, not a replacement for the source journal.

Reading the dataset

Use a Parquet reader to load sensor rows and metadata. Use meta/info.json rather than hard-coding feature dimensions or video paths. Join task_index through meta/tasks.parquet, keep source-trace columns during filtering and use ceres/episode-metadata.json for task and calibration provenance.

The dataset viewer is the quickest way to inspect video, hand pose, gaps and segment boundaries before writing a custom analysis pipeline. Read the technical overview for the recorder and durability contracts behind these artefacts.