Docs lift
Docs lift is the improvement a model shows when given an API's documentation, compared with the closed-book baseline — the same tasks run without the docs. It isolates what the documentation itself contributes: performance a model already had from training earns nothing, and only the gain the docs produce counts. High lift means the docs genuinely teach a model to operate the API.
Lift is the honest unit for a documentation benchmark because it separates the docs doing work from the brand doing work. A model that answers correctly from memory tells you about the model's training; a model that answers correctly only once it has read the docs tells you about the docs. Only the second is a property the API's team controls.
The producer-side implication is encouraging: documentation is the controllable variable. An obscure API with documentation that reliably lifts models into operating it correctly is, for an arriving agent, in better shape than a household name whose live docs contribute nothing beyond what the model remembered.
Docs lift is why the Discry Score is built on doc-dependent tasks: the closed-book pass excludes what models already knew, so what gets graded is the contribution the documentation made. Discry tested the consequence directly — after contamination exclusion, how much models already know about an API and its score are decorrelated, so fame earns nothing and obscurity costs nothing.
What does an AI agent make of your API?
Find out in about a minute — no signup.
Discry your API — free