Baton by HumanRelay

The relay, run as software.

Baton turns real skilled work, captured with consent at partner sites, into verified episodes your models can train on. Then it turns every robot failure into the next capture brief.

EP-24-0412 · ICU · central line check 00:42 · 6 layers · time-synced
RGBiPhone + glasses
DepthARKit LiDAR
Hands26 joints
Stepstask → step Scrub in Glove Check the line Flush Hand over
Whyspecialist why · sterile field why · checks it twice next · hand over
QApublished agreement · 0.92 hidden gold · pass consent · on file
Example data One episode, every device
  • 100,000 hrs captured monthly
  • 5,000+ contracted businesses
  • 1,000+ salaried specialists
  • QA published on every delivery

Capture · ingest

Every device. One episode.

Whatever a site can wear, Baton writes one episode: time-synced, calibrated, and honest about where every stream came from.

EP-0412 · rebar tie · slab B2 clock offset −12.4 ms → −12.4 ms◆ synced
RGB 30fpsdevice-nativenative
LiDAR depthdevice-nativenative
6-DoF head posedevice-nativenative
26-joint handsdevice-nativenativemodel-estimatedestimated
IMU 200Hzdevice-nativenative
Eye gazedevice-nativenative
Audiodevice-nativenative
Steps · whyhuman-verifiedverified
calreg/v14 · 3 rigs checked 07:02 Example data

01 / 05

iPhone and ARKit, today. Coming · M1

Most hours start on an iPhone: RGB, LiDAR depth and pose from ARKit. Smaller cohorts wear glasses, headsets and gloves, which see hands natively.

02 / 05

One schema.

Whatever the device, every stream lands in the same episode, with its rate, units and frame written down.

03 / 05

One clock.

Each device keeps its own clock. Baton stores its offset to a session master and snaps every lane into line at a sync event. Every lens has an entry in a versioned calibration registry.

04 / 05

Every lane names its source.

Device-native, model-estimated or human-verified. Labs filter by source before they train.

05 / 05

Native hands check the estimators. Coming · M3

Where a headset's hand joints and the model's estimate disagree, the frame goes to a specialist. That is how iPhone hours earn a calibrated confidence.

  • iPhone + ARKit todayThe capture app runs on a phone at work. Glasses and headsets join for ground-truth hands.
  • One schemaEvery device writes one episode format, exported as LeRobot v3, RLDS or MCAP.
  • Time sync and calibrationPer-device clock offsets to a session master, and a versioned registry for every lens and rig.
  • Native hands calibrate estimatorsWhere device joints and model estimates disagree, a specialist decides.
episode.manifest.jsonExample data
{
  "episode_id": "ep_0412_7f3a",
  "task": "tie rebar mesh · slab B2",
  "consent_id": "cr_88d1",
  "device_profile": ["iphone_arkit@v7", "xr_headset@v3"],
  "clock": { "master": "session", "max_offset_ms": 0.4 },
  "calibration": "calreg/v14",
  "streams": [
    { "type": "rgb", "hz": 30, "source": "device_native" },
    { "type": "hand_joints", "joints": 26, "source": "device_native" },
    { "type": "hand_joints", "joints": 26, "source": "model_estimated:hands@2.1" },
    { "type": "steps", "source": "human_verified" }
  ],
  "outcome": "success"
}

The product

One hour of work, through Baton.

Follow one ICU line change from the bedside to your bucket. Captured to a brief, drafted by models, verified by a nurse, mapped, measured and delivered.

01 · Capture

Captured to a brief. Coming · M1

A nurse records a line change on her shift, to a brief that names the site, the hand and the light. The app checks hands, blur and narration before she leaves the bed.

02 · Draft → Verify

AI drafts. A nurse decides.

Models draft the task, steps and actions. An ICU nurse with eight years on the ward accepts or corrects each one, then writes down why. Every correction trains the next draft.

03 · Coverage Graph

See what robots haven't seen. Coming · M3

Each verified hour lands on the Coverage Graph. Thin cells pulse as gaps, and one click turns them into briefs for the sites that can fill them.

04 · QA report

Quality, published. Coming · M2

Every delivery ships with its QA report: agreement by label type, hidden-gold accuracy, audited residual error, sync error and reject rate, tracked by drafter version.

05 · Deliver

Straight to your bucket. Coming · M1

LeRobot v3 by default, RLDS or MCAP if you prefer. The manifest, consent records and QA report travel with every episode.

Example data. Product screens show illustrative values.

Baton.
Briefs / B-2207 / EP-0412 · Capture
Live
RK
REC 02:14EP-0412

“Clamping first, then I flush the new line.”

  • hands in view 98%
  • blur ok
  • narration on
ReshootAccept
Brief · B-2207Open

Central line change

ICU · Mumbai · line change, left-handed, low light · 12 episodes needed

Recorder
ICU nurse · 8 yrs
Site rules
Faces and monitors blurred on device · beds 4–9 · 22:00–06:00
Prompts
Keep both hands in frame · say why before each step
Opened by
Coverage gap · Insert & connect × ICU · low light · 0.4 h
Episodes7 of 12 accepted
EP-0412 accepted · uploading 1.8 GB
EP-0412 · ego · 30 fps00:12 / 03:08

Step · Hand hygiene

Verify · EP-0412ICU nurse · 8 yrs

Selected segment

Step 3 · Flush new line

Draft · drafter v14 · conf 0.7401:11 → 01:49

Why · written by the specialist

Aaccept Ccorrect Wwhy Verified 0 / 24
Frames
Task
Central line change.97
Step
Hand hygiene.95 Clamp & disconnect.88 Flush new line.74 Connect & unclamp.91 Secure & label.83
Action
wash.90 glove.93 clamp.86 twist off.81 push 10 ml.77 check for air.62 attach.89 open clamp.84 tape.88 date label.70
Why
clamp so it can't backflow.69
Outcome
line patent.90 no air · dressing sealed.80
AI draft · confidenceVerified by specialistAccept remaining ↵

Coverage Graph · Lab A

Tasks × environments

Condition · low lightSkill · expertLast 30 days
ICUICUORORWardWRDKitchenKITSiteSITTunnelTNLFactoryFACGarageGARDepotDEPHomeHOM
Insert & connect
Grasp & place
Twist & fasten
Wipe & clean
Tie & wrap
Pour & dispense
Cut & trim
Lift & carry
Hand over
Inspect
+1.0 h · EP-0412
Pour & dispense × Tunnel0.6 h verified · gapNearest sites: 2 tunnel projects
14 briefs drafted · 9 sites · awaiting site approval

QA report · Delivery D-0219

ICU procedures slice

1,284 episodes · 412.6 h · 09 Oct 2026

Publishedships with delivery

Agreement by label type · floor 0.85

  • Outcome0.96κ
  • Step boundaries0.93F1
  • Hand keypoints0.92PCK
  • Masks0.91IoU
  • Actions0.89κ
  • Why notes0.86rubric
Gold items1,920
Audit sample5%
Reviewers38

Blind re-label of a 5% audit sample · 64 episodes · two specialists per item

Hidden-gold accuracy

97.6%

v10v14

Residual error · audited

1.8%

v10v14

Sync error · median

6 ms

v10p95 11 ms

Reject rate

3.2%

v10v14

Trends by drafter version, v10 → v14Signed · QA lead · sha256 7f3a…c91e

v14 · residual 1.8% · −0.3 vs v13
Deliver · D-0219esc

Export to your bucket

Format

LeRobot v3defaultRLDSMCAP

Destination

QA reportConsent manifestCalibration

Manifest preview

dataset     hr-icu-procedures
format      lerobot/v3
episodes    1,284  · 412.6 h · 2.31 TB
streams     rgb_ego depth imu hands_26j
labels      task step action why outcome
provenance  native · estimated · verified
consent     1,284 / 1,284 · revocable
qa_report   qa/D-0219.pdf
Ready · 2.31 TB
Deliver →
Delivered · 1,284 episodes · QA report attached
Ingest healthy · sync drift 6 ms Drafter v14 · Verify queue 212 Example data

The laps

Every lap runs faster than the last.

Three laps run on Baton today. Each one hands its output to the next. Each one is timed.

Lap 1 of 3 · hands episodes to Lap 2

Capture Coming · M1

Specialists record real shifts at partner sites. Every hour arrives consented.

Lap time

Cost per usable, consented hour

Lap 1Example data · indexedLap n

Lap 2 of 3 · hands labels to Lap 3

Labelling

Our AI drafts. Our people decide. Every correction makes the next hour faster.

Lap time

Human seconds per verified label, by drafter version

Drafter v1Example data · indexedv n

Lap 3 of 3 · hands briefs to Lap 1

Recapture Coming · M3

Your robot's failures come back as the demonstrations that fix them.

Lap time

Days from failure to a delivered slice

Lap 1Example data · indexedLap n

Next laps · drawn, not yet run

  • Lap 4 · Deployment Robots tested at the sites we film.
  • Lap 5 · Partners Sites that open their doors get more back.

All three laps feed one Coverage Graph. Each lap starts higher than the last.

05 Recapture · Lap 3 Coming · M3

Your robot's failures, turned into the demonstrations that fix them.

  1. 01 Fail
  2. 02 Map
  3. 03 Brief
  4. 04 Re-run

01 · Fail

Your eval finds the gap. The hand loses the IV bag on the regrasp, 13 times in 20. You tag the failure in Baton. Tags travel, policy logs stay in your lab.

02 · Map

The tag finds its cells. Baton places it on the Coverage Graph, task by environment by condition. Transparent containers, in an ICU, in low light: three episodes. Now you can see the hole.

03 · Brief

A brief goes to the wards that do this every night. Specialists at partner sites record it on shift. Every episode is verified and carries its consent and credentials.

04 · Re-run

The slice lands. You re-run. A targeted slice arrives in your bucket with its QA report. You measure it against a control on your own eval. The numbers are yours.

Example data, shown for illustration. Results are measured against a control on your own eval. Shadow Dexterous Hand: Shadow Robot Company, via MuJoCo Menagerie (Apache 2.0).

06 Consent & credentials Coming · M1–M2

Every episode carries its paperwork.

Consent, the site agreement, licence terms, redaction status and the specialist's credentials travel with every episode, from the first frame to your bucket. Nothing enters storage without them.

  1. 01 / 04 · Consent

    Who said yes, and on what terms.

    The worker consents in the app. The site signs its agreement. In a hospital, patient consent runs through the hospital's own protocol, and bystanders are redacted before upload.

  2. 02 / 04 · Credentials

    Who did the work.

    Credentials are verified before a specialist records a single episode. When the passport says ICU nurse, eight years, someone on our team has checked it.

  3. 03 / 04 · Licence & redaction

    What you may do with it.

    Licence terms are set per episode, including whose hands it must never reach. Faces, screens and badges are blurred on the phone, before anything leaves the ward.

  4. 04 / 04 · Quality & provenance

    How good it is. Where it came from.

    Agreement and hidden-gold accuracy are stamped on the episode. Every stream says whether it came from the device, a model, or a person who checked it.

Baton. Episode passport Example data
Episode EP-2609-ICU-04127 42 min 18 s · 14 Sep 2026 · 06:52 IST
Site
Hospital ICU · Mumbai
Site agreement
SA v3.2 · signed 01 Jul 2026
Consent basis
Worker ✓ Bystanders redacted ✓ Patient · hospital protocol ✓
Specialist
ICU nurse · 8 yrs · verified
Licence
Commercial training ✓ Competitor exclusion On
Redaction
Faces · screens · badges Blurred on device ✓
QA
Agreement 0.94 Gold 97%
Provenance
  • Video · head-mountedDevice-native
  • Depth · LiDARDevice-native
  • Camera pathModel-estimated
  • Hand pose · 21 jointsHuman-verified
  • Steps & the whyHuman-verified

Revocation

Consent can be withdrawn. The paperwork follows.

Anyone captured can change their mind, at any time, for any reason. One withdrawal moves through the ledger like this.

  1. Consent withdrawn

    The nurse opts out in the app or tells her site lead. No reason needed.

  2. Pulled from future deliveries

    EP-2609-ICU-04127 is locked out of every export queue. It ships nowhere again.

  3. Lab notified to delete

    Lab A's refresh carries a deletion notice for the episode, as its licence requires.

  4. Partner audit log updated

    The hospital sees the withdrawal, the lock and the lab's confirmation, each timestamped.

Example data. Timestamps illustrate the sequence.

For partner sites

Your site, your people, your terms.

01

Your rules

Mark no-go zones on the floor plan. Set the hours we may record. Cap recorders per ward, bay or line. The phone enforces all three, so nobody has to remember.

  • No-go · pharmacy store
  • Hours · 07:00–19:00
  • Cap · 2 recorders per ward

02

Your people protected

Faces, screens and badges are blurred on the device, before anything leaves the building. Anyone can opt out at any time, and the opt-out reaches every copy we control.

  • Blur · on device
  • Opt-out · any time
  • Revocations · honoured

03

Your know-how, never sold to your competitors

Name the companies in your market. Baton excludes them at export, on every dataset that holds your hours, and logs the check each time.

  • Exclusion list · yours
  • Applied · at export
  • Checked · every delivery
Baton. Partner trust view Hospital group · Last 30 days
Sites1412 recording · 2 outside hours
Hours captured2,318all consented
Incidents0none reported
Revocations honoured3/3median 7 s to lock
Redaction checks18,2400 misses on audit
SiteHoursRecordersNo-go zonesRedaction checksStatus
ICU · Mumbai4122 / 233,206Recording
Operating theatres · Mumbai3682 / 352,941Recording
Ward 4B · Hyderabad2971 / 222,330Recording

+ 11 more sites

Ledger
  • Deletion confirmed by Lab A · EP-…04127
  • Export to Lab B · competitor exclusion applied
  • Consent withdrawn · EP-…04127 locked

Example data. Partner trust view for a hospital group; site names are generic.

For labs

Rights your lawyers can read. Per episode.

Try the demo

Three apps,
live today.

Click through Baton as it runs now. Every screen shows demo data only, behind the same password as this site. Features tagged Coming on this page are planned and not in the demo yet.

Demo data only. Role-based access and saved edits are still to come.

For labs

Files in your bucket,
in your schema.

Every delivery lands as plain files with a manifest. Load it with the tools you already use, and read the hands, the task and the why on every frame.

Example
load_baton.py
from lerobot.datasets.lerobot_dataset import LeRobotDataset

ds = LeRobotDataset("humanrelay/icu-line-change-v3")

for ep in ds.meta.episodes:
    start, end = ep["dataset_from_index"], ep["dataset_to_index"]
    for i in range(start, end):
        frame = ds[i]
        hands = frame["observation.hands"]  # 2 × 26 joints × xyz
        task  = frame["task"]               # "Change the line dressing"
        why   = frame["why"]                # "Second check before the flush"
s3://your-lab/humanrelay/
  • icu-line-change-v3/
  • meta/
  • info.jsonschema · fps · action space
  • stats.json
  • tasks.parquet
  • episodes/1,204 episodes
  • data/chunk-000 … 011
  • videos/ego · wrist · 30 fps
  • qa_report.jsonagreement · gold · reject rate
  • consent_manifest.jsonone record per episode

Example data. Dataset name, paths and counts are illustrative. LeRobot v3 by default; RLDS and MCAP on request.

What it feedsOne episode, three kinds of model.

  • VLA policies

    Hands, actions and language, aligned on every frame.

  • World models

    Hours of first-person video, at the volume pretraining needs.

  • Fast decision models

    Decision points, outcomes and calibrated confidence.

How to startThree laps, and the first is free.

  1. 01

    Request a sample pack

    One verified hour, free. It ships with its QA report and consent manifest.

  2. 02

    Run a pilot slice

    Name your task. We capture it at partner sites and deliver it in your schema.

  3. 03

    Recapture on your failures

    Send us your eval failures. We return the demonstrations that fix them.

Baton. by HumanRelay

Pass us the baton.

Back to humanrelay.com