Rating Criteria

Source Relevancy

score20%llm

Description

Evaluates whether the provided sources are likely to contain the information needed to accurately resolve the prompt. Uses LLM semantic analysis.

How It Works

An LLM analyzes the sources against the prompt requirements:

  1. 1.Assesses if sources are authoritative (primary or secondary)
  2. 2.Checks if sources directly address the information requested
  3. 3.Evaluates likelihood that data will be published and accessible
  4. 4.Verifies sources align with the type of information required

Scoring Thresholds

ScoreAssessment
80-100Highly relevant and authoritative
60-79Moderately relevant but may lack authority
0-59Insufficiently relevant or authoritative

What Makes a Source Relevant

  • >Official sources (government, organization websites)
  • >Established news outlets with fact-checking standards
  • >Specialized data providers for the relevant domain
  • >Direct results pages rather than aggregator articles

What Reduces Score

  • >Generic news sites for specialized data
  • >User-generated content platforms
  • >Sources that cover topic tangentially
  • >Paywall or login-required content