{"slug":"research-relevance-not-volume","name":"Relevance, Not Volume","rules":{"start":"open","maxDepth":10,"allowRewind":true,"allowReplay":true},"nodes":{"open":{"short":"the note","pose":{"base":"centre"},"beat":{"layout":"center","marker":"experiment · interactive","title":["Relevance,","Not Volume"],"glass":1,"lede":"Two operating contracts for a coding agent, matched to the same ~14 KB. The generic one left conformance at its 8% floor; the one carrying a single project-specific rule lifted it to 100% for Codex and 80% for Claude.","caption":"10 beats · 8 min to read in full"},"prompt":"Where do you want to start?","choices":[{"label":"From the top","to":"premise"},{"label":"why two studies","to":"why-two-studies-disagreed-about"},{"label":"Straight to the end","to":"close"}]},"premise":{"short":"the premise","pose":{"base":"flooded"},"beat":{"layout":"left","marker":"01 · experiment","title":["The last piece left us","with a narrow target"],"lede":"The last piece left us with a narrow target. On tasks like these a bare agent already has the engineering judgment to do the work; the only thing it can't guess is a project's own arbitrary conventions, such as the exact shape of an error…","caption":"Condensed from the note."},"prompt":"Where next?","choices":[{"label":"the only lever","to":"the-only-lever-there-is"},{"label":"Wrap it up","to":"close"}]},"the-only-lever-there-is":{"short":"the only lever","pose":{"base":"drained"},"beat":{"layout":"right","marker":"02 · the only lever","title":["The only","lever there"],"lede":"We established last time that the two agents we tested, dropped into a real codebase with no documentation, already nail the functional work (idempotency, replay-safety, deduplication) and do it identically.","caption":"Condensed from the note."},"prompt":"Keep going?","choices":[{"label":"a clean test","to":"a-clean-test-same-length-one-sen"},{"label":"Wrap it up","to":"close"}]},"a-clean-test-same-length-one-sen":{"short":"a clean test","pose":{"base":"centre"},"beat":{"layout":"left","marker":"03 · a clean test","title":["A clean test: same length,","one sentence apart"],"lede":"The clean way to ask whether more context helps is to add context that does not address the task and see if it helps anyway. So we built two versions of the same AGENTS.md operating contract:","rows":[{"title":"Generic. A real, sensible contract","note":"how the project is laid out, how to run it, house style, a boot ritual. Everything except the one rule this task needs. ~14.2 KB."},{"title":"Generic + the rule. ","note":"Byte-for-byte the same contract, padded back to the same length, with one paragraph added: the actual convention the task hinges on (say, list endpoints return a body of total, offset and items,…"}],"caption":"Condensed from the note."},"prompt":"What next?","choices":[{"label":"the result","to":"the-result-volume-is-inert-relev"},{"label":"Wrap it up","to":"close"}]},"the-result-volume-is-inert-relev":{"short":"the result","pose":{"base":"drained"},"beat":{"layout":"right","marker":"04 · the result","title":["The result: volume is inert,","relevance is everything"],"lede":"Here is the conformance rate (how often the agent produces the project's required format) across three conditions: no docs at all, the generic 14 KB contract, and the same-length contract with the rule.","figure":{"kind":"bars","categories":["no docs","generic doc at 14.2 KB","the same doc plus the rule at 14.7 KB"],"series":[{"label":"Codex","tone":"value","values":[4,8,100],"display":["4%","8%","100%"]},{"label":"Claude","tone":"value","values":[4,8,80],"display":["4%","8%","80%"]}],"max":100,"caption":"Adding a document does nothing, adding the rule does everything. The generic and rule-bearing docs are the same length."},"caption":"Condensed from the note."},"prompt":"Where next?","choices":[{"label":"why two studies","to":"why-two-studies-disagreed-about"},{"label":"Wrap it up","to":"close"}]},"why-two-studies-disagreed-about":{"short":"why two studies","pose":{"base":"drained"},"beat":{"layout":"left","marker":"05 · why two studies","title":["Why two studies","disagreed about AGENTS.md"],"lede":"This also speaks to a public disagreement. Within a few months, two careful studies of AGENTS.md files reached opposite conclusions: one found the files cut an agent's effort substantially; the other found they barely help, sometimes hurt,…","caption":"Condensed from the note."},"prompt":"Keep going?","choices":[{"label":"what \"relevant\"","to":"what-relevant-actually-means-her"},{"label":"Wrap it up","to":"close"}]},"what-relevant-actually-means-her":{"short":"what \"relevant\"","pose":{"base":"centre"},"beat":{"layout":"right","marker":"06 · what \"relevant\"","title":["What \"relevant\"","actually means here"],"lede":"It is worth being precise, because \"relevant context\" can sound like a tautology. In this setting it has a sharp definition: context is relevant when it supplies something the agent both needs and cannot derive. That is a small set.","caption":"Condensed from the note."},"prompt":"What next?","choices":[{"label":"scope and limits","to":"scope-and-limits"},{"label":"Wrap it up","to":"close"}]},"scope-and-limits":{"short":"scope and limits","pose":{"base":"flooded"},"beat":{"layout":"left","marker":"07 · scope and limits","title":["Scope","and limits"],"rows":[{"title":"One repository, one convention block, five tasks. ","note":"The flip is large and clean and holds for both agents, but it is a focused result on project wire-formats, not a general law about all documentation."},{"title":"\"Relevance\" is doing real work in this claim","note":"and we have defined it narrowly (needed and underivable). Broaden the definition and the result will soften, which is the point, not a loophole."},{"title":"Codex went to 100%, Claude to 80%. ","note":"Supplying the rule is not a magic switch; one agent applied it slightly less reliably than the other."},{"title":"Relevance is necessary, not sufficient. ","note":"Putting the right rule in front of the agent is most of the battle, but not all of it. Where you put it turns out to matter too."}],"caption":"Condensed from the note."},"prompt":"Where next?","choices":[{"label":"necessary","to":"necessary-but-not-sufficient"},{"label":"Wrap it up","to":"close"}]},"necessary-but-not-sufficient":{"short":"necessary","pose":{"base":"drained"},"beat":{"layout":"right","marker":"08 · necessary","title":["Necessary,","but not sufficient"],"lede":"So the first half of the practitioner's rule is, on these tasks, settled: do not measure context by the pound.","caption":"Condensed from the note."},"prompt":"Keep going?","choices":[{"label":"Close the note","to":"close"}]},"close":{"short":"the end","pose":{"base":"finale"},"beat":{"layout":"center","title":["That was one path.","The note has the rest."],"glass":1,"lede":"This walk is one route through Relevance, Not Volume. The full note carries every chapter, the tables, the code, and the source it was adapted from.","links":[{"href":"/research/relevance-not-volume","label":"Read the full note"},{"href":"/research","label":"All research","tone":"ghost"}]}}}}