Ranked Retrieval Evaluation Workbench

How to use Ranked Retrieval Evaluation Workbench

Enter each query’s authored document grades and ordered retrieved IDs. Choose a cutoff and an explicit unjudged policy, then inspect rank-by-rank gains, exact precision/recall/AP fractions, reciprocal rank and approximate normalized discounted gain.

1. Use the example as the schema; replace its labels and entered model assumptions.
2. Enter each query’s authored document grades and ordered retrieved IDs. Choose a cutoff and an explicit unjudged policy, then inspect rank-by-rank gains, exact precision/recall/AP fractions, reciprocal rank and approximate normalized discounted gain.
3. Calculate, inspect the complete witnesses and limitations, then copy or download the JSON report.

Good to know

• Use 1–20 uniquely named queries, each with 0–100 unique document judgments and 0–100 ranked IDs. Grades are plain integers from 0 through 5 and cutoff is 1–100. The shared 30,000-character JSON cap may constrain combined maximum sizes. Ranked duplicates and repeated judgments are refused.
• Grade above zero counts as relevant. AP@k divides by all authored relevant judgments; short-run P@k still divides by k. No-positive queries have null recall/AP/NDCG and are excluded from those means. The full ranking must be judged unless as-zero is explicitly chosen.
• Binary fractions are exact; exponential-gain NDCG and DCG use finite binary64 logarithms, without a claim of exact real arithmetic. This evaluates your supplied judgments, not web search traffic, online experiments, automated relevance labels or a universal measure of answer quality.
• Local calculation with no upload, input persistence or outside lookup. Model JSON is limited to 30,000 characters and 10,000 values. Decimal quantities are strings with up to six places; count fields use plain JSON integers. Exact fractions are capped at 8,192 bits. The complete JSON output is limited to 1,000,000 characters; approximate numbers preserve their stored values. Unsupported states fail explicitly.