Perplexity Open-Sources WANDR Benchmark for AI Research Agents
Perplexity has open-sourced WANDR, an internal benchmark created to advance the research capabilities of Perplexity Computer. It evaluates two key abilities of AI agents: finding a broad range of relevant entities and supporting every claim with sources.
The dataset includes 500 research tasks across three difficulty levels and requires collecting 170,495 records with links to supporting evidence. Its hierarchical structure and independently verifiable elements help identify not only the overall quality of an answer, but also the specific point where the agent made an error.
WANDR is based on anonymized real-world scenarios—from competitor and market analysis to literature reviews, product comparisons, and expert searches. The system reloads cited pages and checks claims against their content, allowing it to work with facts that change over time.
Why it matters
- —WANDR enables the breadth of search and depth of fact-checking by research agents to be measured separately.
- —The open dataset can help developers identify weaknesses in AI systems more precisely instead of relying on an overall response score.
- —Validating claims against the current content of sources makes it suitable for tasks involving facts that change over time.
Key facts
- WANDR contains 500 research tasks across three difficulty levels.
- Completing the tasks requires generating 170,495 source-backed entries.
- The benchmark was created from anonymized real-world usage scenarios.
- Soft scoring awards partial credit, while strict scoring requires all components to be complete and correct.
- Each cited page is fetched again to verify claims against the supporting evidence.
The full text is in the original source. Here we provide a brief summary and key facts.