Build an RL Gym · Part 6 of 6

Publishing an environment

The last mile is the one that gets skipped

You have a deterministic environment over a real application, a reward that survives adversarial agents, a generated task set with a human-authored held-out split, and a verifier measured against people. What remains is making it usable by someone who is not you, and answering the questions a buyer asks before they will touch it.

Package for the interfaces that already exist

Do not invent an interface. The ecosystem has converged on a small number, and supporting them is mostly adapter code over the harness API from part two:

Surface	What you provide	Why bother
Gymnasium-style class	`reset()` / `step()` over the harness	The lingua franca; everything else wraps it
Container image	One image, one env var for the seed, a documented port	Lets a trainer run you without installing anything of yours
Verifier protocol	A callable taking a trajectory and returning per-criterion scores	Trainers need the reward decomposed, not a scalar
Task manifest	JSONL of task ids, seeds, difficulty coordinates, split	Reproducibility, and it is what buyers diff between versions

claims-env-1.2.0/
  README.md                 # run it in three commands
  ENVCARD.md                # the document below
  Dockerfile                # pinned base, no :latest
  env/
    gym_adapter.py          # Gymnasium-style class over the harness API
    verifier.py             # per-criterion scores, not a scalar
    rubric.yaml             # published, versioned
  tasks/
    train.jsonl  eval_generated.jsonl  eval_human.jsonl
  reports/
    determinism.json        # the two tests from part 2, with output
    baselines.json          # constant / lazy / verbose / thrash agents
    verifier_agreement.json # measured against human graders
    diversity.json          # effective task count vs nominal
  exploits.md
  LICENCE.txt  PROVENANCE.md  CHECKSUMS.sha256

The environment card

A datasheet for environments. Ten sections, all short:

What the agent is asked to do, in the language of the job rather than of RL.
Observation format — exactly what is emitted, and what is deliberately withheld.
Action space, complete, including the behaviour on malformed actions.
Reward — components, weights, penalties, and what each is trying to prevent.
Verification — the four layers, plus the measured false-pass rate.
Task construction — generator, difficulty axes, drop rates at each pipeline stage.
Splits — what is held out, how it was authored, and the rule for using it.
Determinism — what is fixed, what is not, and the reset guarantee.
Baselines — the adversarial agents and a frontier-model reference score.
Provenance and rights — the section below.

Licensing: three layers, and they differ

An environment is not one artefact. Sell it as one licence and you will be granting rights you do not have.

Layer	Owned by	Question to answer
The harness, adapters, verifier, generator	You	Permissive open source, or licensed? Both are defensible; decide deliberately
The wrapped application	Possibly someone else	May you redistribute it inside an image? This is the question that ends projects
Fixtures and task content	Depends on origin	Synthetic (clean) or derived from production data (carries every obligation that data carries)

The clean configuration — permissive harness, application you own or an open-source equivalent, fully synthetic fixtures — is worth engineering toward from the start, because it is the only one that sells without a legal negotiation attached.

The provenance questions a buyer will ask

What is the underlying application and what licence covers redistributing it?
Where did the task content come from — generated, authored, or derived from real records? If derived: whose records, under what basis?
Who wrote the human-authored evaluation set, what were their qualifications, and does their work belong to you?
Has the task content been screened against public benchmarks? Show the overlap report.
What is the false-pass rate, measured how, against whom?
What exploits are known and open? "None" is not a credible answer; a log is.
How will this be maintained when the wrapped application changes?

Those seven map almost exactly onto the dimensions in env-assay, which exists because nobody grades environments and they are being traded anyway. If you can answer all seven with an artefact, you are in a very small group.

Versioning, because environments drift

Semantic versions with a stated contract: patch = fixture or bug fix that cannot change scores; minor = new tasks or a new observation field; major = anything that changes what a score means.
Never mutate a published task set. Add a new split; leave the old one addressable. Scores reported against v1.1 must remain reproducible after v1.2 ships.
Publish a score delta on every minor release — the same baseline agents re-run, so a buyer can see what moved.
Pin the wrapped application version in the image tag. An environment that silently follows upstream is not an environment, it is a moving target.

The environments that get bought are not the most impressive. They are the ones where a buyer's engineer can run three commands, get a number, and reproduce it tomorrow.

That is the series. The environment we built — a claims workflow, wrapped, generated, verified and packaged — is the shape of the asset most organisations running real software could produce and currently do not.

List an environment All writing