How It Works

The Rating System

Caliber assigns credit-rating-style grades from Aaa to C using a two-stage evaluation of gates and weighted scores.

Rating Bands

Aaa90–100
Exceptional definition quality; highly reliable and unambiguous.
Aa80–89
Very strong; minor weaknesses.
A70–79
Strong; some limitations.
Baa60–69
Adequate; moderate ambiguity or risk.
Ba50–59
Speculative; notable weaknesses.
B35–49
Weak; high risk of poor resolution.
Caa20–34
Very weak; high likelihood of problematic resolution.
Ca10–19
Highly unreliable; severe structural issues.
C0–9
Structurally broken or guaranteed to fail resolution.

Two-Stage Evaluation

Stage 1: Gates

Pass/fail checks that must all pass. If any gate fails, the market receives a score of 0 regardless of other criteria.

Source Reachability

All provided source URLs must be accessible and return valid responses.

Source Blocklisted

No provided sources may appear on the known-unreliable source blocklist.

Stage 2: Scores

Weighted criteria scored 0–100. Each score is multiplied by its weight, and the sum produces the final score.

final_score = Σ (criterion_score × weight)

Score Criteria

Source Countstatic

Sufficient independent sources are provided for reliable resolution.

Weight20%
Source Agreementstatic

The minimum agreement threshold is achievable given the source count.

Weight20%
Source Relevancyllm

Sources are relevant and authoritative for the market's question.

Weight20%
Source Historystatic

Sources have a track record of reliability and availability.

Weight10%
Prompt Subjectivityllm

The market prompt is clear, objective, and unambiguous.

Weight10%
Temporal Soundnessllm

The resolution window is logically sound for the market's question.

Weight10%
static — rule-based, deterministicllm — semantic analysis

Example Calculation

  source_count:         40 × 0.20 =  8.0
  source_agreement:    100 × 0.20 = 20.0
  source_history:       60 × 0.10 =  6.0
  prompt_subjectivity:  75 × 0.10 =  7.5
  temporal_soundness:   85 × 0.10 =  8.5
  source_relevancy:     90 × 0.20 = 18.0
  ─────────────────────────────────────
  final_score:                      68.0  →  Baa

Try it yourself

Submit a market definition and see how Caliber rates it in real time.

Rate a Market