{"slug":"research-environment-and-token-cost","name":"How the Environment Affects Agent Performance and Token Cost","rules":{"start":"open","maxDepth":11,"allowRewind":true,"allowReplay":true},"nodes":{"open":{"short":"the note","pose":{"base":"centre"},"beat":{"layout":"center","marker":"experiment · interactive","title":["How the Environment Affects","Agent Performance and Token Cost"],"glass":1,"lede":"A pilot running one coding agent against one fixed task under six escalating levels of project context. Every environment finished, the richest ones spent up to 36 percent fewer tokens, and caching served 88 to 96 percent of the input.","caption":"11 beats · 11 min to read in full"},"prompt":"Where do you want to start?","choices":[{"label":"From the top","to":"premise"},{"label":"the scaffold","to":"the-scaffold-is-nearly-free-beca"},{"label":"Straight to the end","to":"close"}]},"premise":{"short":"the premise","pose":{"base":"flooded"},"beat":{"layout":"left","marker":"01 · experiment","title":["We ran a coding agent against","the same software task under six"],"lede":"We ran a coding agent against the same software task under six escalating levels of project context, from an empty workspace up to a full XO project scaffold, holding the model, the prompt, the task, and the codebase fixed.","caption":"Condensed from the note."},"prompt":"Where next?","choices":[{"label":"background","to":"background"},{"label":"Wrap it up","to":"close"}]},"background":{"short":"background","pose":{"base":"drained"},"beat":{"layout":"right","marker":"02 · background","title":["Background","experiment"],"lede":"A theme runs through our work on agentic systems: the environment an agent works inside matters more than the agent that executes the work.","caption":"Condensed from the note."},"prompt":"Keep going?","choices":[{"label":"what we measured","to":"what-we-measured"},{"label":"Wrap it up","to":"close"}]},"what-we-measured":{"short":"what we measured","pose":{"base":"recede"},"beat":{"layout":"left","marker":"03 · what we measured","title":["What we","measured"],"lede":"The design is a grid. We hold the task and the codebase constant and vary one thing: how much project context the workspace carries when the agent starts.","rows":[{"title":"Task success. ","note":"The share of automated acceptance checks that pass, from 0 to 1."},{"title":"Total input tokens. ","note":"Everything the agent reads and processes: the raw throughput of the run."},{"title":"Effective tokens. ","note":"The cache-adjusted cost, which charges cached input at the cache rate rather than the full rate. Concretely it is uncached input plus 0.1 times cached input plus output."},{"title":"Orientation. ","note":"How much the agent explores to get its bearings, counted as blind shell probes: directory listings, greps, and file reads run to reconstruct the project's layout."}],"caption":"Condensed from the note."},"prompt":"What next?","choices":[{"label":"context did not","to":"context-did-not-cost-success-and"},{"label":"Wrap it up","to":"close"}]},"context-did-not-cost-success-and":{"short":"context did not","pose":{"base":"drained"},"beat":{"layout":"right","marker":"04 · context did not","title":["Context did not cost success,","and it lowered the token bill"],"lede":"On the easy task, every one of the six environments finished the job with a perfect score. That is the first thing to settle: adding project context did not slow the agent down or confuse it. Nothing was lost.","figure":{"kind":"bars","categories":["E0 Empty","E1 +README","E2 +AGENTS","E3 +PROJECT","E4 Full XO","E5 +XO+Memory"],"series":[{"label":"measured","tone":"value","values":[1.87,1.4,1.27,1.95,1.8,1.2],"display":["1.87M","1.40M","1.27M","1.95M","1.80M","1.20M"]}],"caption":"Total input tokens per environment on the easy task, T01, where every run scores 1.0. README, the AGENTS contract, and the full XO scaffold with seeded memory each spend fewer tokens than the empty workspace, down to 36 percent fewer."},"caption":"Condensed from the note."},"prompt":"Where next?","choices":[{"label":"the scaffold","to":"the-scaffold-is-nearly-free-beca"},{"label":"Wrap it up","to":"close"}]},"the-scaffold-is-nearly-free-beca":{"short":"the scaffold","pose":{"base":"drained"},"beat":{"layout":"left","marker":"05 · the scaffold","title":["The scaffold is nearly free,","because it is cached"],"lede":"The obvious worry about scaffolding is that it just stuffs the context window with files the agent has to pay to read. Under prompt caching, that worry does not hold.","caption":"Condensed from the note."},"prompt":"Keep going?","choices":[{"label":"why it works","to":"why-it-works-token-cost-is-an-or"},{"label":"Wrap it up","to":"close"}]},"why-it-works-token-cost-is-an-or":{"short":"why it works","pose":{"base":"centre"},"beat":{"layout":"right","marker":"06 · why it works","title":["Why it works: token cost","is an orientation tax"],"lede":"What is the agent actually spending tokens on, if not on reading the scaffold? On finding its way around.","caption":"Condensed from the note."},"prompt":"What next?","choices":[{"label":"scope","to":"scope-and-limitations"},{"label":"Wrap it up","to":"close"}]},"scope-and-limitations":{"short":"scope","pose":{"base":"flooded"},"beat":{"layout":"left","marker":"07 · scope","title":["Scope","and limitations"],"lede":"This is a pilot, and we would rather report it honestly than oversell it.","rows":[{"title":"Small samples. ","note":"Each cell is a single run, except the empty T01 cell which has two. These are directional findings, not tight confidence intervals. The exact percentages will move as we add replications."},{"title":"One agent, one codebase. ","note":"We ran OpenAI Codex against a single repository, a snapshot of xo-cowork-api. We have not yet checked how the effect transfers to other agents or other codebases."},{"title":"The token savings are task-specific. ","note":"The reductions in section 3 come from one easy task. Broader runs in our rig show that richer context does not always cut tokens, so we treat the savings as a real but bounded result, not a universal…"},{"title":"A success effect we chose not to claim. ","note":"An earlier look suggested structured context also lifted task success on the harder task: the bare workspaces scored partial while the XO-context ones scored complete."}],"caption":"Condensed from the note."},"prompt":"Where next?","choices":[{"label":"what this means","to":"what-this-means-for-xo-projects"},{"label":"Wrap it up","to":"close"}]},"what-this-means-for-xo-projects":{"short":"what this means","pose":{"base":"drained"},"beat":{"layout":"right","marker":"08 · what this means","title":["What this means","for XO Projects"],"lede":"The practical reading is simple. The leverage is in the environment, not in a cleverer prompt or a bigger model.","caption":"Condensed from the note."},"prompt":"Keep going?","choices":[{"label":"appendix","to":"appendix-per-cell-data"},{"label":"Wrap it up","to":"close"}]},"appendix-per-cell-data":{"short":"appendix","pose":{"base":"recede"},"beat":{"layout":"left","marker":"09 · appendix","title":["Appendix:","per-cell data"],"lede":"Every number above is recomputed directly from the per-run telemetry records. Tokens are rounded. The headline cost results come from T01, where all runs succeed so cost is comparable across conditions.","panelRows":[{"title":"T01","note":"Env E0 Empty · Score 1.00 · Input tokens 1.87M · Effective (paid) tokens 275K"},{"title":"T01","note":"Env E1 +README · Score 1.00 · Input tokens 1.40M · Effective (paid) tokens 278K"},{"title":"T01","note":"Env E2 +AGENTS · Score 1.00 · Input tokens 1.27M · Effective (paid) tokens 216K"},{"title":"T01","note":"Env E3 +PROJECT · Score 1.00 · Input tokens 1.95M · Effective (paid) tokens 323K"}],"caption":"Condensed from the note, which carries 6 rows."},"prompt":"What next?","choices":[{"label":"Close the note","to":"close"}]},"close":{"short":"the end","pose":{"base":"finale"},"beat":{"layout":"center","title":["That was one path.","The note has the rest."],"glass":1,"lede":"This walk is one route through How the Environment Affects Agent Performance and Token Cost. The full note carries every chapter, the tables, the code, and the source it was adapted from.","links":[{"href":"/research/environment-and-token-cost","label":"Read the full note"},{"href":"/research","label":"All research","tone":"ghost"}]}}}}