Exa ATLAS and Tavily Test Different Kinds of Search Completeness

Finding one useful company fact and completing a whole set of company records are different jobs. Two new vendor-owned evaluations make that distinction concrete: Exa’s October 8 ATLAS preview examines wide, multi-step research, while Tavily’s October 9 Company Search Benchmark focuses on usefulness from one search response.

Neither measures consumer SEO traffic. Their scores should not be combined into a universal ranking of search providers. A buyer needs to choose the task first, then ask which evaluation resembles it.

A small example exposes the denominator

Consider a fictional reference set of four companies, each requiring two facts. A system returns three correct companies, with both facts correct for two of them and one fact correct for the third. It invents no extra companies. This creates three different completeness results from exactly the same output.

The calculations use F1 = 2 × precision × recall / (precision + recall). We exclude identity columns from the illustrative fact-cell count to keep the arithmetic explicit; this is not a reproduction of ATLAS’s complete grading implementation. It shows why “found the right businesses” is weaker than “returned complete correct records.”

What ATLAS counts

Exa’s ATLAS preview reports 547 tasks and distinguishes discovery, item, and row F1. Discovery concerns entities; item scoring concerns correct cells; row scoring requires complete correct rows. Its answer-key construction uses agents, verification, and repair. Exa is both the benchmark publisher and a vendor evaluated by it. Those facts belong beside any reported result.

A multi-step research system may search again when a company is missing a field. That is a useful behavior if the user asked for an exhaustive list and can tolerate the time and cost. It is not the same operating condition as a single search response used to answer a quick question.

What Tavily counts

Tavily’s Company Search Benchmark compares nine providers through a shared answer-generation and judging process. It weights Complete, Partial, Related, and Irrelevant answers at 1, 0.85, 0.15, and 0. Scores average those weights over queries. Tavily describes human validation for financial figures and is also a comparable vendor.

The released dataset viewer exposes a test split labeled with about 1,000 rows and fields including qid, category, recency_aware, query and scorecard_text. The visible records include both ordinary company questions and questions with false premises. This is an inspection of the published viewer, not a full audit of every scoring card or a rerun of the leaderboard.

A weighted usefulness score has a different interpretation from a percentage of entirely correct answers. For example, if every answer were rated Partial under Tavily’s weights, the average would be 0.85 even though none was Complete. That is a calculation explaining the scale, not a description of any provider’s actual result.

Choose the evaluation that matches the purchase

• For one-shot company enrichment, check whether the question wording, entities, markets, and freshness requirements resemble your requests.
• For exhaustive discovery, check the definition of the target set and whether missing entities are penalized.
• For complete records, inspect how partially correct rows, unknown cells, and unsupported values are handled.
• For production decisions, hold the answer model and prompt constant, and include time, retrieval charges, and human review alongside quality.

The decisive comparison is a usable answer at the required scope. A high discovery score may be enough to generate leads for later verification. It is insufficient when every row is sent directly to a customer-facing directory. Conversely, a detailed multi-hop evaluation may overstate the complexity of a simple lookup workload.

Our local business-data report makes branch identity and field freshness concrete. The retrieval and citation analysis separates API outcomes from the displayed sources, while the cross-engine source analysis addresses a different task entirely. This article explains evaluation units; it does not establish an independently tested provider winner.