Technical documentation

This reference is for researchers, dataset users and developers who need to interpret a CERES capture beyond the viewer. It describes the products written by the current browser recorder and the LeRobot v3 export path. It is not a deployment runbook.

Capture model

CERES records outward-facing video, optional raw microphone audio, headset pose and both 25-joint WebXR hands. A cycle produces one recorder episode. Recordable task occurrences become ordered segments inside that episode, with their cycle, repetition, take, outcome, annotations, slot accounting and source-time bounds retained as provenance.

The recorder clock is authoritative. It defaults to 30 Hz and owns scheduled slots independently of rendering, WebRTC, speech recognition and user-interface updates. A source observation may fill one slot only. When an observation is missing, CERES writes an explicit gap slot at the scheduled timestamp rather than duplicating another sample.

Durability and boundaries

The capture worker writes an ordered journal in browser origin-private storage before the recording indicator is presented. In paired capture, the capture director maintains the matching committed journal. Solo maintains its own local authority journal. A recorder, checksum, journal or storage-headroom failure stops recording rather than presenting an unreliable result.

The live monitor is deliberately lossy and may reduce rendering work under load. It is not the source of record. The exporter reads the committed journal and reconstructs the dataset from durable sensor frames, explicit gaps, media chunks and segment accounting.

Episode retention

Pass and fail are annotations. They describe a completed segment but do not rewrite its sensor rows. Retry is different: its attempted segment remains present in the durable CERES journal for auditability, then is removed from the exported dataset and final video. Reset windows and configured pauses are paused out of the source timeline.

An export includes every retained recorder slot, including explicit gaps. An episode with no retained sensor data is not exportable. Interrupted or manually stopped episodes remain exportable when they contain a durable retained sensor frame.

Compatibility

The browser exporter writes immutable episode-sized shards compatible with LeRobotDataset v3.0. It uses a dedicated Wasm worker, with scalar and SIMD variants, so exporting does not share the Quest capture path or the monitor rendering worker. The exporter produces deterministic Parquet structure and metadata, while video preparation attaches an aligned H.264 MP4 when video is available.

Read Data description and output format for the complete folder layout, Parquet columns and CERES provenance sidecars.

What technical readers should preserve

• Keep ceres.source_frame_index, ceres.source_timestamp and ceres.source_gap when deriving a new dataset. They preserve the source timing and loss history behind compact exported rows.
• Keep ceres/episode-metadata.json with its sibling Parquet and MP4 artefacts. It links the episode to camera, task and segment provenance.
• Use the task specification sidecar when present. Its SHA-256-named path is part of the episode metadata and identifies the exact task plan used for the run.
• Treat unknown and unavailable capture-metadata values as meaningful states. Do not convert them to measured zeros or inferred device properties.