Selling to Labs
How to run a trajectory capture programme
Every organisation with skilled people using software is sitting on an asset it is not recording: the actual sequence of decisions, actions and recoveries that constitute doing the work. Agentic training needs exactly this and cannot scrape it, because it does not exist anywhere in public. A capture programme is the one way a data owner can manufacture a top-tier asset rather than discovering one — and it can be started in a fortnight.
What a trajectory is
Not a screen recording, and not a log. A trajectory is a structured sequence of states and actions with enough context to reconstruct why each action was taken:
{
"task_id": "claim-4417",
"goal": "Adjudicate submitted claim; escalate if documentation incomplete",
"operator_role": "senior claims assessor",
"steps": [
{
"i": 0,
"observation": {"screen": "s3://…/0.png", "dom_digest": "…",
"app_state": {"queue": "intake", "claim_id": "4417"}},
"narration": "Checking the referral date first — if it postdates the
procedure the rest doesn't matter.",
"action": {"type": "click", "target": "#referral-date", "value": null},
"result": {"ok": true, "state_delta": {"panel": "referral"}}
}
],
"outcome": {"status": "escalated", "verified_by": "supervisor",
"ground_truth": "correct"}
}
Three fields in there are what separate a valuable corpus from an unsellable
one: narration, result and outcome.
Everything else can be reconstructed. Those cannot.
Consent, before anything is recorded
Retrofitting consent is the most expensive mistake available in this programme, because it cannot be done. Set it up first:
- Operator consent, explicit and specific: that their work will be recorded, that the recording may be licensed for training AI models, that it may be sold to third parties, and how long it is retained. Voluntary, with a real ability to decline, and separate from their employment terms.
- Third-party data — the customer, patient, or counterparty whose case is being worked on. Either the recording happens on synthetic or historical anonymised cases, or the underlying consent has to cover this use. Decide which before day one, because it changes the tooling.
- Assignment of the output. The recording is a work product; the paperwork should say so explicitly and by name, especially for contractors — see chain of title.
- A withdrawal path that is real: per-operator deletion, documented, testable, and possible after delivery. Buyers now ask.
What to record at each step
| Layer | Capture | Why it earns its storage |
|---|---|---|
| Screen | Frame or screenshot at each action, plus a viewport size | The only faithful record for computer-use training; useless alone |
| Structure | DOM digest, accessibility tree, or API-level state | Text-native models train on this far better than on pixels |
| Action | Type, target, arguments, timestamp | The label. Fuzzy targets ("clicked the button") are worthless — record the selector |
| Tool calls | Every call with full arguments and full response | Where agentic capability actually lives |
| Environment state | The application's own state before and after | Makes the step verifiable rather than merely observed |
| Narration | One line of why, from the operator, at the moment | The single highest-value field, and the one everyone omits |
| Outcome | Terminal status plus an independent correctness judgement | Turns demonstrations into trainable and gradeable data |
Narration is the whole game
A click stream teaches imitation. A click stream with reasons teaches a policy. Operators will not write essays and should not be asked to — one line, typed or spoken, at each non-obvious step, with an explicit "no comment" option for the obvious ones. Prompt for it in the tool rather than asking people to remember. Where speech is easier than typing, capture audio and transcribe; the audio itself never needs to be delivered.
Keep the failures
The instinct to deliver only clean, successful runs destroys most of the value of a capture programme. A model that has only seen success has never seen recovery, and recovery is the capability buyers are short of.
Record and keep: dead ends, wrong turns with the correction that followed,
tasks abandoned as impossible, and tasks completed incorrectly with the error
identified afterwards. Label them — failed,
recovered_at_step_n, abandoned,
completed_incorrect — so the buyer can filter rather than guess.
A corpus where 30% of trajectories contain a recovery is worth more than a
pristine one, and is a differentiator you can state in the listing.
Paying operators
- Pay for the time, not the trajectory. Per-trajectory rates produce rushed, shallow, suspiciously uniform work — and the uniformity is detectable in the delivered corpus.
- Pay a premium for narration and for failures. Whatever you pay for is what you get; if failures pay the same as successes, you will receive successes.
- Do not tie pay to the buyer's acceptance. It creates an incentive to produce what looks good rather than what is true, which is the fidelity problem in the rubric.
- Record who did what. Not names in the corpus — a pseudonymous operator id, with roles and credentials documented separately. Buyers price on who produced the work and cannot take it on trust.
A two-week start
- Days 1–3: pick one task type, write the consent, get it signed. One task type only.
- Days 4–7: instrument. A browser extension or a wrapped desktop session capturing screen, structure, action and tool calls, with a narration prompt bound to a hotkey.
- Days 8–10: run five operators for a day. Read every trajectory yourself. You will find the schema is wrong in two places; that is the point of the pilot.
- Days 11–14: fix the schema, write the datasheet, define the outcome judgement and who makes it. Then scale.
A hundred well-captured trajectories with narration, verified outcomes and clean consent is a saleable asset and a credible sample. Ten thousand click-streams without reasons or outcomes is a storage bill.
If your application has observable end states, the same instrumentation produces RL environments as well as demonstrations — two products from one programme. See the gym series.
Read next: Non-exclusive, exclusive, first-look — how to structure the sale once you have something to sell.