Sell2Labs

Selling to Labs

How to run a trajectory capture programme

The highest-value thing most organisations could build

Every organisation with skilled people using software is sitting on an asset it is not recording: the actual sequence of decisions, actions and recoveries that constitute doing the work. Agentic training needs exactly this and cannot scrape it, because it does not exist anywhere in public. A capture programme is the one way a data owner can manufacture a top-tier asset rather than discovering one — and it can be started in a fortnight.

What a trajectory is

Not a screen recording, and not a log. A trajectory is a structured sequence of states and actions with enough context to reconstruct why each action was taken:

{
  "task_id": "claim-4417",
  "goal": "Adjudicate submitted claim; escalate if documentation incomplete",
  "operator_role": "senior claims assessor",
  "steps": [
    {
      "i": 0,
      "observation": {"screen": "s3://…/0.png", "dom_digest": "…",
                      "app_state": {"queue": "intake", "claim_id": "4417"}},
      "narration": "Checking the referral date first — if it postdates the
                    procedure the rest doesn't matter.",
      "action": {"type": "click", "target": "#referral-date", "value": null},
      "result": {"ok": true, "state_delta": {"panel": "referral"}}
    }
  ],
  "outcome": {"status": "escalated", "verified_by": "supervisor",
              "ground_truth": "correct"}
}

Three fields in there are what separate a valuable corpus from an unsellable one: narration, result and outcome. Everything else can be reconstructed. Those cannot.

Consent, before anything is recorded

Retrofitting consent is the most expensive mistake available in this programme, because it cannot be done. Set it up first:

What to record at each step

LayerCaptureWhy it earns its storage
ScreenFrame or screenshot at each action, plus a viewport sizeThe only faithful record for computer-use training; useless alone
StructureDOM digest, accessibility tree, or API-level stateText-native models train on this far better than on pixels
ActionType, target, arguments, timestampThe label. Fuzzy targets ("clicked the button") are worthless — record the selector
Tool callsEvery call with full arguments and full responseWhere agentic capability actually lives
Environment stateThe application's own state before and afterMakes the step verifiable rather than merely observed
NarrationOne line of why, from the operator, at the momentThe single highest-value field, and the one everyone omits
OutcomeTerminal status plus an independent correctness judgementTurns demonstrations into trainable and gradeable data

Narration is the whole game

A click stream teaches imitation. A click stream with reasons teaches a policy. Operators will not write essays and should not be asked to — one line, typed or spoken, at each non-obvious step, with an explicit "no comment" option for the obvious ones. Prompt for it in the tool rather than asking people to remember. Where speech is easier than typing, capture audio and transcribe; the audio itself never needs to be delivered.

Keep the failures

The instinct to deliver only clean, successful runs destroys most of the value of a capture programme. A model that has only seen success has never seen recovery, and recovery is the capability buyers are short of.

Record and keep: dead ends, wrong turns with the correction that followed, tasks abandoned as impossible, and tasks completed incorrectly with the error identified afterwards. Label them — failed, recovered_at_step_n, abandoned, completed_incorrect — so the buyer can filter rather than guess. A corpus where 30% of trajectories contain a recovery is worth more than a pristine one, and is a differentiator you can state in the listing.

Paying operators

A two-week start

  1. Days 1–3: pick one task type, write the consent, get it signed. One task type only.
  2. Days 4–7: instrument. A browser extension or a wrapped desktop session capturing screen, structure, action and tool calls, with a narration prompt bound to a hotkey.
  3. Days 8–10: run five operators for a day. Read every trajectory yourself. You will find the schema is wrong in two places; that is the point of the pilot.
  4. Days 11–14: fix the schema, write the datasheet, define the outcome judgement and who makes it. Then scale.

A hundred well-captured trajectories with narration, verified outcomes and clean consent is a saleable asset and a credible sample. Ten thousand click-streams without reasons or outcomes is a storage bill.

If your application has observable end states, the same instrumentation produces RL environments as well as demonstrations — two products from one programme. See the gym series.

Read next: Non-exclusive, exclusive, first-look — how to structure the sale once you have something to sell.

List a dataset All writing