Agent-Mimicked Synthetic Data Will Never Beat the Real Thing

The argument for synthetic evaluation data is genuinely seductive. If a model can write prose no human can pick out of a line-up, why can't a good enough agent simulate a user well enough to test on? The answer is not that the mimicry is bad. It is that passing for real and being calibrated to real are different properties, and only one of them is what an evaluation is for.

Two numbers carry the case. Organic replay, re-running an evaluation against real deployment traffic, forecasts production misbehaviour at a correlation of 0.91. Agent-mimicked synthesis stalls at a 49.5 percent discriminator win-rate: a coin flip, and it gets there only when handed real ground truth as scaffolding.

Read those together and the second one stops sounding like a win. A pipeline that can fool a discriminator half the time has demonstrated that its outputs look real. It has not demonstrated that the rate at which a behaviour shows up in its scenarios tells you anything about the rate it will show up in production. That second property is the whole job.

Where the title overstates it

"Never" is a headline, and the note is more careful than its title. Synthetic data legitimately wins on several axes, and the honest version of this argument concedes them up front:

• Contamination resistance. Freshly generated items cannot have leaked into a training corpus, which benchmark items demonstrably do.
• Rare and high-severity coverage. Genuinely dangerous scenarios are rare in real traffic and ethically fraught to collect; targeted generation can manufacture them.
• Training, not just testing. The largest alignment successes of recent years run on model-generated critiques and preference labels.
• Cold start. For a first-of-kind capability, organic replay produces no data at all, because the deployment whose logs you would want does not exist yet. Here synthetic is not the better instrument, it is the only one.

Those cluster on one axis: coverage and throughput. The organic advantage clusters on another: calibration and validity. The two are not competing for the same job.

The dependency that settles it

The deepest point in the note is not a horse race. Synthetic realism can only ever be validated against organic data. Discriminator win-rates are defined by picking the real transcript. Simulation calibration is scored against realised production rates. Organic data is the measuring stick, so it cannot be fully replaced by the thing it measures without the measurement quietly losing its meaning.

That is the sense in which the title is right. Not that synthetic data is bad, but that it can never be the thing that tells you whether your map still matches the territory.

The full treatment, with the sources, the counterweight and a note on epistemic status, is in the research note.