Rating Criteria

Prompt Subjectivity

score10%llm

Description

Analyzes how clear, specific, and unambiguous the prompt is for extracting information from the provided sources. Uses LLM semantic analysis.

How It Works

An LLM evaluates the prompt against each source URL:

  1. 1.Considers whether the prompt clearly defines what to extract
  2. 2.Assesses if an LLM could unambiguously interpret the request
  3. 3.Scores each source-prompt pair from 0-100
  4. 4.Averages scores across all sources

Scoring Thresholds

ScoreAssessment
80-100Precise and unambiguous
60-79Some ambiguity but mostly clear
0-59Significant ambiguity

What Reduces Score

  • >Vague or subjective terms ("best", "significant", "many")
  • >Missing specific criteria or thresholds
  • >Unclear time references
  • >Multiple possible interpretations