Physical AI teams win by iterating quickly on data composition and modeling while scaling data and compute.

Open datasets are an important part of that process, but they often arrive in different formats and schemas. Using them effectively can require substantial dataset-specific work before they are ready to feed into training pipelines.

This repository provides examples for converting open-source datasets into the Rerun file format (.rrd), making them easier to inspect, combine, and use in training workflows. Converted datasets are published on Hugging Face, with more to come.

Already using .rrd? You're in the right place.

New to Rerun? Start with What is Rerun?, How does Rerun work?, and A new data layer for robot learning.

Dependencies are managed with Pixi. Install it:

curl -fsSL https://pixi.sh/install.sh | sh

Clone the repository:

git clone https://github.com/rerun-io/rrd-datasets.git
cd rrd-datasets

For a quick demo, run an ABC-130k dataset example end to end — download sample episodes, convert them, and open the results in the Rerun Viewer:

pixi run -e abc demo

Note that the dataset is gated on Hugging Face — accept its terms and authenticate first, see examples/abc-130k.

This downloads about 900 MB into data/.

Run pixi task list to see all available tasks.

This repository was built using Rerun's agent skills. The set used is listed in skills-lock.json; install them with:

npx skills experimental_install

Currently supported datasets:

The repository is organized around self-contained dataset examples, with shared utilities kept alongside them.

examples/       one directory per dataset — the conversion code
packages/       shared utilities used by the examples
blueprints/     each dataset's generated default viewer layout (gitignored)
data/  rrds/    created locally by the tasks: downloads and converted recordings (gitignored)

Each example is a small Python package:

examples/<dataset>/
    README.md       source, license, mapping to Rerun, how to run
    <dataset>/      the conversion code: download.py, convert.py, blueprint.py, catalog.py, …
    tests/

It registers its pixi tasks in pixi.toml and generates its default blueprint under blueprints/<dataset>/.

Each example follows the same pipeline:

📥 Download a sample of the source dataset
      ↓
🔄 Convert it to .rrd and generate a blueprint
      ↓
👀 Inspect the recordings in the Rerun Viewer
      ↓
🗄️ Serve the recordings and register them to a catalog
      ↓
🔎 Query the catalog for your curation/training

Each stage is a pixi task of the same name, run in the dataset's environment. For example:

pixi run -e abc download
pixi run -e abc convert
pixi run -e <env> <stage>      # blueprint, register, …

The examples intentionally favor readable conversion code over abstraction.

Code in this repository is dual-licensed under MIT or Apache-2.0, at your option.

Individual datasets retain their original licenses. See each dataset's README for source and licensing information.