The Mix Report

What we actually know about frontier data mixes

Edition 1Updated quarterly

Nobody publishes current standings on what is in frontier training mixes, because almost nobody discloses. Two projects disclose completely. The rest has to be triangulated from model cards, papers, licensing announcements and job ads. This is the first pass, and it will be wrong in places — corrections get published alongside it.

The disclosure gradient

"Open model" is not a single thing. For a data seller, the only distinction that matters is whether the training corpus is itemised, because an itemised corpus is the only place you can see the shape of the slot you would occupy.

The gradient is not evenly distributed across training stages, either. Even partial disclosers say more about pretraining scale than about what went into post-training — which is precisely the stage where purchased data lands. That asymmetry is not an accident. Pretraining sources are largely public web; the post-training mix is where competitive advantage and commercial contracts sit.

The one corpus you can read end to end

Dolma 3, the corpus behind Olmo 3, is the anchor for everything in this report, because it is the only frontier-scale mix where the pool, the sampled mix and the stage boundaries are all stated. As released:

Two things in that table deserve more attention than they get.

First, the ratio between pool and mix. A 9.3T pool yields a 5.9T pretraining mix. Roughly a third of what was collected, cleaned and tokenised was not used. Selection is doing as much work as collection. If you are selling volume, you are selling into the part of the pipeline that already throws a third of its input away.

Second, the size of the midtraining slot. 100B tokens is under two per cent of the pretraining mix, sampled from a 2.2T candidate pool, and it is where the maths, science, code and instruction-following capability work happens. That is the realistic destination for a licensed dataset. A seller sizing their asset against 5.9T concludes they are irrelevant; sizing it against 100B — against a pool of 2.2T candidates competing for those 100B slots — is the correct comparison. Midtraining is where licensed data lands takes this apart properly.

NVIDIA's Nemotron-CC sits in the same tier for a narrower purpose: a documented, reproducible pipeline for turning Common Crawl into a pretraining-grade English corpus, with the classifier ensemble and the synthetic rephrasing steps described well enough to rebuild. It is the best public evidence of how much engineering separates "web text" from "pretraining data" — and therefore of how little raw scraped text is worth on its own.

Reading the partial disclosers

Model cards from the large open-weight releases follow a recognisable pattern: a total token count, a list of source categories, a statement that data was filtered for quality and safety, and no proportions. Three things are worth extracting from them anyway.

• Category presence. Categories that appear in every card — code, maths, multilingual web, curated books and papers — are commodity. Categories that appear in one card and not others are where that lab thinks it has an edge.
• Total token count over time. Successive releases from the same family show whether the lab is still scaling tokens or has moved to scaling quality and stages. When token counts flatten while benchmarks keep moving, the gains are coming from midtraining and post-training — which is a buying signal.
• The safety and licensing language. Cards increasingly distinguish "publicly available", "licensed from third parties" and "created by annotators". The presence of the middle category is the only direct confirmation many labs give that they buy data at all.

Inferring the closed mixes

For the closed labs, there are four public channels and each has a characteristic bias:

1. Announced licensing deals. Publishers announce; labs confirm tersely. This over-represents deals where the seller wants the publicity — news and forum content — and under-represents quiet acquisitions of specialist corpora. Prices, where disclosed, are collected in What one token costs.
2. Litigation and regulatory filings. The most reliable and the most lagged. What surfaces in discovery is usually two to four years old by the time it is public.
3. Hiring. Job ads for domain experts, annotation programme managers and "data partnerships" roles are a leading indicator of where a lab thinks it is short. A posting for physician annotators precedes a medical-capability push by a year or more.
4. Published evaluations. Where a lab reports results tells you what it optimises; where it declines to report tells you more. Sustained silence on a capability that competitors report is either indifference or a known weakness, and the saturation board is our attempt to separate the two.

The honest summary: for closed labs we can establish which capability areas are being bought into with reasonable confidence, and almost never at what proportion. Anyone claiming to know the percentage of a frontier mix that is licensed is estimating from the same four channels.

What a seller should take from this

Three conclusions, in descending order of how much they change behaviour.

Size your asset against the stage it fits, not against the model. "We have 40 million tokens and they train on trillions" is the wrong comparison, made constantly. The right one is against the stage pool: tens of billions of candidate tokens competing for a midtraining slot, or tens of thousands of trajectories competing for a post-training mix.

Assume the buyer already has the commodity categories. Everything that appears in every disclosed mix is already in the buyer's pool, deduplicated, filtered and free. If your corpus is largely web text, encyclopedic text or public code, the disclosure record says it has been collected several times over. The value is in what does not appear in any published mix.

Disclosure pressure runs downhill to you. Regulatory summary obligations mean buyers increasingly have to describe their training data in public. A buyer who must publish a summary cannot accept a corpus whose provenance you cannot document — regardless of how good the data is. Chain of title, in practice covers what that means, and it is now the most common reason a promising deal stops.

Method, and how to correct this

Everything above is desk research over public sources: released corpora and their documentation, model cards, papers, company announcements and public filings. No lab confirmed anything for us. Figures quoted from Dolma 3 are as published by Ai2 and are approximate by their own account; token counts vary with tokeniser and with release revision, so treat every number here as an order of magnitude with a citation attached, not as a measurement.

This table is maintained. Each quarter it gains the mixes disclosed since, and a changelog of what we got wrong. If you know a disclosed mix we have missed, or a number we have misread, send it — corrections are published with attribution next to the original claim.

Read next: How a mix actually gets chosen — what the search is optimising when your dataset is one of its candidates.