SEO Data Bench/Datasets/seo-api-evaluation-v1
SEO API evaluation cohort v1: reproduce the benchmark
The declared cohort of 50 queries and 20 domains, the routes it was sent through, the exact commands, and the per-call record format, so anyone can repeat the run and diff it against ours.
Written for people asking: How do I benchmark SEO data APIs against each other on the same queries?
Cohort v1 is 50 US-English buyer queries for data APIs and 20 vendor domains, frozen on 15 September 2026. It was sent on the same day through four keyword-volume routes (DataForSEO, Serpstat, SE Ranking, Google Ads Keyword Planner), five Google SERP routes (Cloro live render, DataForSEO, Serpstat, SerpApi, Treg routed) and three backlink routes (Serpstat, SE Ranking, Moz), one call per query or domain per route, one worker per provider. Every raw response is published as one JSON file per call. Re-running the whole cohort needs the Treg CLI and about $2.50 in API charges at list prices.
What is in the cohort
- 50 queries. The buyer vocabulary of the data-API market: enrichment APIs, SEO data APIs, ad-library APIs and agent tooling, taken from a keyword export on 14 September 2026. Head terms such as “serp api” (five figures of monthly Google volume) sit next to long-tail terms such as “pay per call enrichment api” that no provider reports, on purpose: coverage of the long tail is the property an agent workload actually tests.
- 20 domains. The vendors compared across this site network plus treg.to: Hunter, Apollo, People Data Labs, Clay, Explorium, FullEnrich, Findymail, Icypeas, Prospeo, Lusha, Semrush, Ahrefs, DataForSEO, SerpApi, Serpstat, SE Ranking, Moz, Apify, ScrapeCreators and Treg.
- Market. United States (Google location code 2840), English, desktop, page 1, depth 10.
The cohort is frozen. A different query set is a new version, not an edit.
Routes and how each was called
Every route runs through the Treg CLI so that authentication, retries and timing are identical. Direct providers use the provider’s own endpoint; Treg’s routed endpoint picks a provider per call and names it in the response. Google Ads Keyword Planner is called with the account’s own credentials and is free.
| Task | Route | Request shape | List price on run date |
|---|---|---|---|
| Keyword volume | DataForSEO keywords_data/google_ads/search_volume/live |
one call, all 50 keywords | $0.09 per call |
| Keyword volume | Serpstat getKeywordsInfo, se=g_us |
one call, all 50 keywords | $0.0005 per keyword |
| Keyword volume | SE Ranking keywords/export, source=us |
one call, all 50 keywords | $0.00179 per keyword |
| Keyword volume | Google Ads generateKeywordHistoricalMetrics |
one call, all 50 keywords | free with own account |
| SERP | Cloro monitor/google, gl=US |
one call per query, live render | $0.0028 per call |
| SERP | DataForSEO serp/google/organic/live/advanced, depth 10 |
one call per query | $0.002 per call |
| SERP | Serpstat getKeywordTop, se=g_us |
one call per query, stored index | $0.0005 per call |
| SERP | SerpApi engine=google, gl=us, hl=en, num=10 |
one call per query | $0.015 per call |
| SERP | Treg routed google.serp.organic |
one call per query | billed at the serving provider’s rate |
| Backlinks | Serpstat getSummaryV2 |
one call per domain | $0.0025 per call |
| Backlinks | SE Ranking backlinks/summary, mode=domain |
one call per domain | $0.0179 per call |
| Backlinks | Moz url_metrics |
one call per domain | $0.0133 per call |
Not measured: Ahrefs (the account’s plan has no API access), Majestic and Semrush (no credential). They are absent from every table, not scored as failures.
Reproduce it
treg auth login
scripts/run-seo-cohort.py evidence/fixtures/seo-cohort-v1.json evidence/seo-runs/$(date +%F)
scripts/score-seo-cohort.py evidence/fixtures/seo-cohort-v1.json evidence/seo-runs/$(date +%F)The runner keeps one worker per provider so our own concurrency does not inflate anyone’s latency, skips a call whose output file already exists (so an interrupted run resumes), and stores wall-clock latency and exit code with every response. The scorer is deterministic: no language model touches a number.
Downloads: cohort v1 · runner · scorer · raw responses under /evidence/seo-runs/2026-09-15/ (see the evidence page for the file naming).
Record format
Each raw file is {"_run": {...}, "response": {...}}. The _run header carries task (kw, serp, bl), provider, the q or d it was called with, label, latency_ms, exit and a UTC ts. The response is the provider’s body untouched. For Treg routed calls the body includes a _treg block naming served_by and charged_micro.
Known limitations of run 2026-09-15
- DataForSEO’s bulk keyword call returned “too many requests” on the first attempt because the SERP calls to the same provider were running at the same time. It was retried after the run; both records are kept and the benchmark uses the retry.
- Google Ads Keyword Planner returned 49 keywords for 50 sent; it merged “backlink api” into “backlinks api” as close variants. That keyword is absent for Google Ads, not zero.
- Serpstat’s SERP is a stored index, so long-tail queries it has never crawled return “keyword not found”. These are counted as no data, separately from errors.
- One run. Latency numbers are one day’s network conditions; a second dated run is scheduled and will be published beside this one.
Spotted an error or a changed price? Email corrections@seodatabench.com with the page URL and a source. Corrections are logged on the corrections page.