01 / 05
iPhone and ARKit, today. Coming · M1
Most hours start on an iPhone: RGB, LiDAR depth and pose from ARKit. Smaller cohorts wear glasses, headsets and gloves, which see hands natively.
Baton by HumanRelay
Baton turns real skilled work, captured with consent at partner sites, into verified episodes your models can train on. Then it turns every robot failure into the next capture brief.
Capture · ingest
Whatever a site can wear, Baton writes one episode: time-synced, calibrated, and honest about where every stream came from.
01 / 05
Most hours start on an iPhone: RGB, LiDAR depth and pose from ARKit. Smaller cohorts wear glasses, headsets and gloves, which see hands natively.
02 / 05
Whatever the device, every stream lands in the same episode, with its rate, units and frame written down.
03 / 05
Each device keeps its own clock. Baton stores its offset to a session master and snaps every lane into line at a sync event. Every lens has an entry in a versioned calibration registry.
04 / 05
Device-native, model-estimated or human-verified. Labs filter by source before they train.
05 / 05
Where a headset's hand joints and the model's estimate disagree, the frame goes to a specialist. That is how iPhone hours earn a calibrated confidence.
{
"episode_id": "ep_0412_7f3a",
"task": "tie rebar mesh · slab B2",
"consent_id": "cr_88d1",
"device_profile": ["iphone_arkit@v7", "xr_headset@v3"],
"clock": { "master": "session", "max_offset_ms": 0.4 },
"calibration": "calreg/v14",
"streams": [
{ "type": "rgb", "hz": 30, "source": "device_native" },
{ "type": "hand_joints", "joints": 26, "source": "device_native" },
{ "type": "hand_joints", "joints": 26, "source": "model_estimated:hands@2.1" },
{ "type": "steps", "source": "human_verified" }
],
"outcome": "success"
}
The product
Follow one ICU line change from the bedside to your bucket. Captured to a brief, drafted by models, verified by a nurse, mapped, measured and delivered.
01 · Capture
A nurse records a line change on her shift, to a brief that names the site, the hand and the light. The app checks hands, blur and narration before she leaves the bed.
02 · Draft → Verify
Models draft the task, steps and actions. An ICU nurse with eight years on the ward accepts or corrects each one, then writes down why. Every correction trains the next draft.
03 · Coverage Graph
Each verified hour lands on the Coverage Graph. Thin cells pulse as gaps, and one click turns them into briefs for the sites that can fill them.
04 · QA report
Every delivery ships with its QA report: agreement by label type, hidden-gold accuracy, audited residual error, sync error and reject rate, tracked by drafter version.
05 · Deliver
LeRobot v3 by default, RLDS or MCAP if you prefer. The manifest, consent records and QA report travel with every episode.
Example data. Product screens show illustrative values.
“Clamping first, then I flush the new line.”
Central line change
ICU · Mumbai · line change, left-handed, low light · 12 episodes needed
Step · Hand hygiene
Selected segment
Step 3 · Flush new line
Why · written by the specialist
Coverage Graph · Lab A
Tasks × environments
QA report · Delivery D-0219
ICU procedures slice
1,284 episodes · 412.6 h · 09 Oct 2026
Agreement by label type · floor 0.85
Blind re-label of a 5% audit sample · 64 episodes · two specialists per item
Hidden-gold accuracy
97.6%v10v14
Residual error · audited
1.8%v10v14
Sync error · median
6 msv10p95 11 ms
Reject rate
3.2%v10v14
Trends by drafter version, v10 → v14Signed · QA lead · sha256 7f3a…c91e
Export to your bucket
Format
Destination
Manifest preview
dataset hr-icu-procedures format lerobot/v3 episodes 1,284 · 412.6 h · 2.31 TB streams rgb_ego depth imu hands_26j labels task step action why outcome provenance native · estimated · verified consent 1,284 / 1,284 · revocable qa_report qa/D-0219.pdf
The laps
Three laps run on Baton today. Each one hands its output to the next. Each one is timed.
Lap 1 of 3 · hands episodes to Lap 2
Specialists record real shifts at partner sites. Every hour arrives consented.
Lap time
Cost per usable, consented hour
Lap 1Example data · indexedLap n
Lap 2 of 3 · hands labels to Lap 3
Our AI drafts. Our people decide. Every correction makes the next hour faster.
Lap time
Human seconds per verified label, by drafter version
Drafter v1Example data · indexedv n
Lap 3 of 3 · hands briefs to Lap 1
Your robot's failures come back as the demonstrations that fix them.
Lap time
Days from failure to a delivered slice
Lap 1Example data · indexedLap n
Next laps · drawn, not yet run
All three laps feed one Coverage Graph. Each lap starts higher than the last.
05 Recapture · Lap 3 Coming · M3
01 · Fail
Your eval finds the gap. The hand loses the IV bag on the regrasp, 13 times in 20. You tag the failure in Baton. Tags travel, policy logs stay in your lab.
02 · Map
The tag finds its cells. Baton places it on the Coverage Graph, task by environment by condition. Transparent containers, in an ICU, in low light: three episodes. Now you can see the hole.
03 · Brief
A brief goes to the wards that do this every night. Specialists at partner sites record it on shift. Every episode is verified and carries its consent and credentials.
04 · Re-run
The slice lands. You re-run. A targeted slice arrives in your bucket with its QA report. You measure it against a control on your own eval. The numbers are yours.
Example data, shown for illustration. Results are measured against a control on your own eval. Shadow Dexterous Hand: Shadow Robot Company, via MuJoCo Menagerie (Apache 2.0).
06 Consent & credentials Coming · M1–M2
Consent, the site agreement, licence terms, redaction status and the specialist's credentials travel with every episode, from the first frame to your bucket. Nothing enters storage without them.
01 / 04 · Consent
The worker consents in the app. The site signs its agreement. In a hospital, patient consent runs through the hospital's own protocol, and bystanders are redacted before upload.
02 / 04 · Credentials
Credentials are verified before a specialist records a single episode. When the passport says ICU nurse, eight years, someone on our team has checked it.
03 / 04 · Licence & redaction
Licence terms are set per episode, including whose hands it must never reach. Faces, screens and badges are blurred on the phone, before anything leaves the ward.
04 / 04 · Quality & provenance
Agreement and hidden-gold accuracy are stamped on the episode. Every stream says whether it came from the device, a model, or a person who checked it.
Revocation
Anyone captured can change their mind, at any time, for any reason. One withdrawal moves through the ledger like this.
The nurse opts out in the app or tells her site lead. No reason needed.
EP-2609-ICU-04127 is locked out of every export queue. It ships nowhere again.
Lab A's refresh carries a deletion notice for the episode, as its licence requires.
The hospital sees the withdrawal, the lock and the lab's confirmation, each timestamped.
Example data. Timestamps illustrate the sequence.
For partner sites
01
Mark no-go zones on the floor plan. Set the hours we may record. Cap recorders per ward, bay or line. The phone enforces all three, so nobody has to remember.
02
Faces, screens and badges are blurred on the device, before anything leaves the building. Anyone can opt out at any time, and the opt-out reaches every copy we control.
03
Name the companies in your market. Baton excludes them at export, on every dataset that holds your hours, and logs the check each time.
| Site | Hours | Recorders | No-go zones | Redaction checks | Status |
|---|---|---|---|---|---|
| ICU · Mumbai | 412 | 2 / 2 | 3 | 3,206 | Recording |
| Operating theatres · Mumbai | 368 | 2 / 3 | 5 | 2,941 | Recording |
| Ward 4B · Hyderabad | 297 | 1 / 2 | 2 | 2,330 | Recording |
+ 11 more sites
Example data. Partner trust view for a hospital group; site names are generic.
For labs
Rights your lawyers can read. Per episode.
Try the demo
Click through Baton as it runs now. Every screen shows demo data only, behind the same password as this site. Features tagged Coming on this page are planned and not in the demo yet.
Demo data only. Role-based access and saved edits are still to come.
For labs
Every delivery lands as plain files with a manifest. Load it with the tools you already use, and read the hands, the task and the why on every frame.
from lerobot.datasets.lerobot_dataset import LeRobotDataset
ds = LeRobotDataset("humanrelay/icu-line-change-v3")
for ep in ds.meta.episodes:
start, end = ep["dataset_from_index"], ep["dataset_to_index"]
for i in range(start, end):
frame = ds[i]
hands = frame["observation.hands"] # 2 × 26 joints × xyz
task = frame["task"] # "Change the line dressing"
why = frame["why"] # "Second check before the flush"
import tensorflow_datasets as tfds
builder = tfds.builder_from_directory("gs://your-lab/icu-line-change/rlds/1.0.0")
ds = builder.as_dataset(split="train")
for episode in ds:
for step in episode["steps"]:
hands = step["observation"]["hands"] # 2 × 26 joints × xyz
task = step["language_instruction"] # "Change the line dressing"
why = step["why"] # "Second check before the flush"
from pathlib import Path
from mcap.reader import make_reader
from mcap_protobuf.decoder import DecoderFactory
for path in sorted(Path("icu-line-change/episodes").glob("*.mcap")):
with open(path, "rb") as f:
reader = make_reader(f, decoder_factories=[DecoderFactory()])
topics = ["/observation/hands", "/task", "/why"]
for _, channel, msg, data in reader.iter_decoded_messages(topics=topics):
print(channel.topic, msg.log_time, data) # hands, task, why
Example data. Dataset name, paths and counts are illustrative. LeRobot v3 by default; RLDS and MCAP on request.
VLA policies
Hands, actions and language, aligned on every frame.
World models
Hours of first-person video, at the volume pretraining needs.
Fast decision models
Decision points, outcomes and calibrated confidence.
Request a sample pack
One verified hour, free. It ships with its QA report and consent manifest.
Run a pilot slice
Name your task. We capture it at partner sites and deliver it in your schema.
Recapture on your failures
Send us your eval failures. We return the demonstrations that fix them.