- Last run
- Sep 29, 2026
- Benchmarks
- 5
Benchmarks for web search APIs and research agents
Tako Search is on par with the best web search APIs on SimpleQA and FRAMES. On research questions, Tako leads: 63.9% on FinSearchComp T3 (next best 46.4%) and 66.4% on VerticalRTK - Research (next best 55.0%). On questions that need premium data, it leads by more: 86.3% on VerticalRTK - Data Connectors (next best 55.0%).
Search API
Tako Search returns web results and structured data in one response. SimpleQA and FRAMES ask about facts that are widely published and don’t change, which general web search indexes well; we run them to check that Tako stays at parity with web search APIs on accuracy. FinSearchComp needs exact figures, often from filings and market data rather than articles.
SimpleQA
Tako Search scored 93.9% on SimpleQA, on par with Exa (93.7%) and Parallel (93.7%).
SimpleQA is OpenAI’s 2024 benchmark of 4,326 short fact-seeking questions, each with one indisputable answer.
In which year, month, and day was the Nokia 8110 announced?
SimpleQA accuracy by search API
Methodology
- Run date
- Jul 21, 2026
- Send each question to every provider’s search API and keep the results it returns.
- Have one answer model write an answer from each provider’s results alone.
- Have a separate judge model grade each answer against the ground truth.
FRAMES
Tako Search scored 49.3% on FRAMES, on par with Exa (47.8%).
FRAMES is Google’s 2024 benchmark of 824 questions that each combine facts from several Wikipedia articles, so an answer needs more than one source.
If my future wife has the same first name as the 15th first lady of the United States' mother and her surname is the same as the second assassinated president's mother's maiden name, what is my future wife's name?
FRAMES accuracy by search API
Methodology
- Run date
- Jul 21, 2026
- Send each question to every provider’s search API and keep the results it returns.
- Have one answer model write an answer from each provider’s results alone.
- Have a separate judge model grade each answer against the ground truth.
FinSearchComp
Tako Search scored 21.4% on FinSearchComp T3, ahead of Exa (19.0%), and tied with Exa on T2 at 54.6%.
FinSearchComp is a 2025 benchmark of 635 expert-written financial questions in three tiers. T2 asks for historical facts that don’t change, such as a company’s total assets on a date. T3 needs multi-step reasoning, aggregation, and calculation. We report T2 and T3.
From May 19th to May 23rd, 2025, what was the closing price (unadjusted) of Tesla in the U.S. stock market that week? How much higher/lower was it than that of Amazon? Keep 2 decimal places.
FinSearchComp T3 accuracy by search API
FinSearchComp T2 accuracy by search API
Methodology
- Run date
- Jul 21, 2026
- Send each question to every provider’s search API and keep the results it returns.
- Have one answer model write an answer from each provider’s results alone.
- Have a separate judge model grade each answer against the ground truth.
VerticalRTK - Data Connectors
With Tako Search as its only tool, an agent scored 86.3% on VerticalRTK - Data Connectors, ahead of the same agent with Exa (55.0%), Parallel (50.0%), or Firecrawl (35.0%), and answered fastest.
VerticalRTK - Data Connectors is VerticalRTK’s set of 80 fact-retrieval questions, 20 each in companies, macroeconomics, government spending, and sports: the verticals where data connector products overlap most. Each question asks for a figure from one licensed or public dataset, so a provider answers correctly only when its data reaches that figure.
No provider used an integration that needs you to bring your own license. Firecrawl costs use its Standard annual plan’s credit price.
What were Alfa Laval AB (publ)'s total order intake and Energy order intake for its fiscal quarter ended June 30, 2026, each in billions of Swedish krona?
Search API accuracy and median latency in the same agent
Search API accuracy and average cost per question in the same agent
Methodology
- Run date
- Sep 29, 2026
- Questions
- 80
- Harness
- Artificial Analysis Stirrup
- Give one agent, Artificial Analysis Stirrup, a single search API as its only tool, with a budget of three searches per question.
- Configure each search API as the Artificial Analysis Search Index methodology does.
- Grade each answer against the set’s reference figures, and record each run’s latency and cost.
The questions and grading rules are public in the VerticalRTK repository, so you can rerun this benchmark.
Answer API
Trivia about the past says little about the questions agents need answered for real research. We built VerticalRTK to test domain questions, including ones whose answers change daily, and score Tako’s Answer API against search APIs on it.
VerticalRTK - Research
Tako’s Answer API scored 66.4% on the 131 questions of VerticalRTK - Research, 11.4 points ahead of the next-best API, Exa (55.0%).
VerticalRTK - Research is the research set of VerticalRTK, Tako’s own benchmark, built and maintained by us. Its questions are in finance, macroeconomics, sports, weather, and commodities. Every fast-set question records when its answer was true, and time-sensitive answers are re-sourced before each run.
As of market close on July 9, 2026, which of the 'Magnificent Seven' had the highest market cap, and how much bigger was it than the second-place company?
VerticalRTK - Research accuracy by search API
Methodology
- Run date
- Jul 21, 2026
- Harness
- VerticalRTK repository
- Re-source every time-sensitive answer before the run, so the ground truth matches the run date.
- Send each question to every provider’s search API, and have one answer model write an answer from each provider’s results.
- Have a separate judge model grade each answer against the ground truth.
The queryset and harness are public, so you can rerun this benchmark.
Agent
The Answer Agent researches a question and writes the answer. We score it against other research agents on both FinSearchComp tiers: T2 asks for historical facts that don’t change, and T3 needs multi-step reasoning, aggregation, and calculation.
FinSearchComp T2
The Answer Agent scored 89.1% on FinSearchComp T2, ahead of the next-best agent, Parallel (Ultra), at 73.1%. It answered in 42.4 seconds, under half the time of any other agent.
FinSearchComp is a 2025 benchmark of 635 expert-written financial questions in three tiers. T2 asks for historical facts that don’t change, such as a company’s total assets on a date. T3 needs multi-step reasoning, aggregation, and calculation. We report T2 and T3.
What were the research and development expenses of Uber for the full year 2019? (answer in millions of dollars, rounded to nearest integer)
FinSearchComp T2 accuracy and time to answer by agent
Methodology
- Run date
- Jul 21, 2026
- Give each research agent every question, and let it search and reason end to end.
- Time each run from the question to the final answer.
- Grade each final answer against the ground truth.
FinSearchComp T3
The Answer Agent scored 63.9% on FinSearchComp T3, ahead of the next-best agent, Parallel (Ultra), at 46.4%. It answered in 77.9 seconds, under half the time of any other agent.
From May 19th to May 23rd, 2025, what was the closing price (unadjusted) of Tesla in the U.S. stock market that week? How much higher/lower was it than that of Amazon? Keep 2 decimal places.
FinSearchComp T3 accuracy and time to answer by agent
Methodology
- Run date
- Jul 21, 2026
- Give each research agent every question, and let it search and reason end to end.
- Time each run from the question to the final answer.
- Grade each final answer against the ground truth.
VerticalRTK - Data Connectors
The Answer Agent scored 93.8% on VerticalRTK - Data Connectors, ahead of Exa Connect (81.3%), Parallel Task (73.8%), and Firecrawl Agent (72.5%), at the lowest latency and cost of the four.
VerticalRTK - Data Connectors is VerticalRTK’s set of 80 fact-retrieval questions, 20 each in companies, macroeconomics, government spending, and sports: the verticals where data connector products overlap most. Each question asks for a figure from one licensed or public dataset, so a provider answers correctly only when its data reaches that figure.
No provider used an integration that needs you to bring your own license. Firecrawl costs use its Standard annual plan’s credit price.
What were Alfa Laval AB (publ)'s total order intake and Energy order intake for its fiscal quarter ended June 30, 2026, each in billions of Swedish krona?
Agent accuracy and median latency
Agent accuracy and cost per 1,000 questions
Methodology
- Run date
- Sep 29, 2026
- Questions
- 80
- Run each provider’s agent with its default or recommended setting: Exa Connect (Exa Agent on Auto), Parallel Task Core, and Firecrawl Agent Default.
- Select as many relevant data connectors as each agent allows, and ask it to answer in text only.
- Grade each answer against the set’s reference figures, and record each run’s latency and cost.
The questions and grading rules are public in the VerticalRTK repository, so you can rerun this benchmark.
Why is Tako more accurate on research questions?
Because Tako returns the figure itself, from the source that publishes it. Tako gives agents direct access to structured data from authoritative sources, returned with web results in one response. An agent gets the figure it needs with a citation, instead of searching and reading pages to find it, so its answers are more accurate and faster to reach.
On questions whose answers are widely published, like SimpleQA and FRAMES, that advantage is small: most good search APIs find the page. It shows on research questions, where the answer needs exact figures, several steps of reasoning, or data that changes daily.
Start building with Tako
Web results and structured data in one API call. Your first 2,000 requests are free.