Loading rankings…
Loading rankings…
Measurement design
Usage volume vs efficiency. Token counts, cost, activity, context reuse, Yield, privacy, provenance, gaming resistance, and measurement limitations compared.
Published: September 4, 2026
If you are building a leaderboard for AI users (operators), the first question is: what should it measure? The answer determines what the leaderboard rewards and what it ignores. A leaderboard that measures token volume rewards spending. A leaderboard that measures efficiency rewards token-cascade behavior. A leaderboard that measures task correctness rewards output quality. Each choice has trade-offs.
This article compares the most common measurement options and explains why SigArena uses Yield (Υ) — while being explicit about what Yield does and does not measure.
Total input + output + cache tokens consumed.
Pros
Easy to measure. Correlates with activity and spend.
Cons
Rewards spending, not skill. A user who burns tokens inefficiently ranks highly. No efficiency signal.
Total dollar spend on AI API calls.
Pros
Directly tied to budget. Useful for cost management.
Cons
Depends on pricing tiers, model selection, and provider. Not portable across tools. Rewards spending.
Session count, message count, or time spent.
Pros
Simple. Shows engagement.
Cons
Does not distinguish productive use from idle churn. A user with many low-quality sessions ranks highly.
Output tokens divided by input tokens.
Pros
Captures how much an operator produces relative to what they feed in.
Cons
Ignores context reuse. An operator who reuses cached context efficiently may have a low ratio because cache reads are not counted as input.
Cache-read tokens relative to fresh input.
Pros
Rewards operators who build on prior context rather than re-prompting from scratch.
Cons
Does not account for output volume. High reuse with low output is not necessarily efficient.
(cache_read × output) / input². Token-cascade efficiency.
Pros
Combines context reuse and output production into a single metric. Rewards operators who reuse cached context and produce substantial output relative to fresh input. Does not reward raw spending.
Cons
Measures a token-flow relationship, not productivity, quality, or skill. Can be gamed by inflating cache reads or output. Does not capture task correctness or business value.
SigArena ranks operators by Yield (Υ) = (cache_read × output) / input² because it captures a token-cascade behavior that raw volume and cost do not: how effectively an operator reuses cached context and produces output relative to fresh input. An operator who burns millions of tokens without reusing context will not rank highly, even if their total spend is the largest.
This does not mean Yield is "better" than volume in general. It means Yield answers a different question. Volume answers "who spends the most?" Yield answers "who ranks highest under the stated token-cascade metric?" Neither answers "who is the best developer?"
Choosing a metric is necessary but not sufficient. A leaderboard also needs to address privacy, provenance, gaming resistance, and measurement limitations.
Does the metric require prompt content or code?
Yield and raw token metrics use only token counts. They do not require prompt text, response text, source code, or transcripts. This is a privacy boundary, not a guarantee of zero metadata risk.
Can the telemetry be verified?
Signed, server-verifiable token telemetry provides provenance. Without it, self-reported token counts are unverifiable. TTEOP (Token Telemetry Evaluation Operator Protocol) defines the telemetry envelope and provenance levels.
Can the metric be manipulated?
Every metric can be gamed. Token volume can be inflated by running pointless sessions. Yield can be inflated by manufacturing cache reads or output. Signed telemetry and outlier detection reduce but do not eliminate gaming. No metric is fully gaming-proof.
What does the metric not capture?
Token-cascade metrics do not measure correctness, novelty, user satisfaction, economic value, code quality, safety, talent, effort, intelligence, productivity, professional skill, task correctness, employee performance, or business value. They measure a token-flow relationship.
Yield and related token-cascade metrics quantify a token-flow relationship. They do not measure:
A high Yield means an operator ranks highly under the stated metric and time window. It does not mean they are a better developer, a more productive worker, or a more valuable employee. Conflating token-cascade efficiency with any of these is a category error.
The SigArena AI User Leaderboard applies Yield to rank operator profiles. See who ranks highest and get your own rank.
View the AI User Leaderboard →