Crawler APIs

Dashboard-aligned crawler APIs for Overview, Live, Pages, Access, and the same safe actions shown in the crawler dashboard.

Requires authentication

60 req/min reads30 req/min actions

Overview

The crawler API now follows the same mental model as the crawler dashboard: Overview, Live, Pages, Access, and the same safe actions already visible in-product.

Public crawler APIs do not expose internal implementation terms likecitation-correlation, intelligence, or standalone raw events as first-class public concepts. Those can still power the dashboard internally, but the public contract stays aligned to what users actually see.

The legacy /get-crawler endpoint still exists as a compatibility route. New integrations should use the endpoints below.

Primary Endpoint

GET/crawler/overview

Returns the top-level crawler dashboard state: hero metrics, chart series, platform breakdown, top pages, and recent preview rows.

Query Parameters

brand_idstringrequired

Brand ID from /get-brands

range_presetstring

"24h", "7d", "30d", "90d", or "custom"

Default: 30d

start_datestring

Required when range_preset=custom. Format YYYY-MM-DD

end_datestring

Required when range_preset=custom. Format YYYY-MM-DD

compare_tostring

"none" or "previous_period"

Default: none

Endpoint Suite

Use the crawler suite the same way you use the dashboard: fetch an overview, drill into Live or Pages, open a page/path/bot drawer, inspect Access, then use the safe actions if needed.

GET/crawler/overview

Hero stats, chart, setup state, top pages, and recent preview.

GET/crawler/live

Live tab views: activity, pages, and sessions with dashboard filters.

GET/crawler/pages

Pages tab lenses: pages, paths, and bots with dashboard sorting and pagination.

GET/crawler/page-details

Page drawer sections: verdict, pipeline, health, traffic, diagnostics.

GET/crawler/path-details

Path drawer sections: verdict, pipeline, top pages, top bots, diagnostics.

GET/crawler/bot-details

Bot drawer sections: verdict, pipeline, top pages, top paths, diagnostics.

GET/crawler/access

Access tab data: findings, bot access matrix, robots.txt, llms.txt, submit-to-search.

POST/crawler/access/preview-fix

Preview an access fix without applying it.

POST/crawler/verification-ping

Send the synthetic verification ping shown in the dashboard.

POST/crawler/submit-to-search

Submit URLs to AI search via Bing IndexNow.

GET/crawler/submit-to-search/status

Read submit-to-search status and the current summary.

Filters and Ranges

The crawler read endpoints share the same date model as the dashboard. Userange_preset for the built-in windows:24h, 7d,30d, 90d, orcustom with start_date andend_date.

Filter vocab stays intentionally narrow and dashboard-shaped:intent supports all,interaction, search, andtraining. Page status uses the visible dashboard states:converting, cited,crawled, healthy, andcold.

Safe Actions

The public crawler actions match the dashboard's safe action boundary: verification ping, preview access fix, and submit to search. There is no public endpoint to auto-apply crawler fixes.

POST /crawler/access/preview-fix is preview-only. Fix application remains internal.

Code Example

Overview Example

1curl -H 'Authorization: Bearer $TRAKKR_API_KEY' \
2  'https://api.trakkr.ai/crawler/overview?brand_id=75ffdeb9-0924-4ff9-8ded-c2470d73d224&range_preset=30d&compare_to=previous_period'

1{
2  "brand_id": "75ffdeb9-0924-4ff9-8ded-c2470d73d224",
3  "range": {
4    "range_preset": "30d",
5    "start": "2026-03-18",
6    "end": "2026-04-17",
7    "days": 30,
8    "compare_to": "previous_period"
9  },
10  "state": {
11    "has_tracking_id": true,
12    "has_historical_data": true,
13    "tracking_id": "75ffdeb9-0924-4ff9-8ded-c2470d73d224"
14  },
15  "hero": {
16    "total_visits": 1842,
17    "total_visits_change": 23.5,
18    "unique_pages": 47,
19    "unique_pages_change": 12.0,
20    "avg_daily_visits": 61.4,
21    "top_crawler": "ChatGPT"
22  },
23  "chart": {
24    "series": [
25      { "date": "2026-04-10", "count": 72, "by_crawler": { "ChatGPT": 38, "Perplexity": 21 } }
26    ],
27    "distribution": [
28      { "crawler": "ChatGPT", "count": 892, "percentage": 48.4 }
29    ],
30    "bot_type_summary": { "training": 621, "search": 504, "interaction": 717, "agent": 0, "other": 0 }
31  },
32  "breakdown": {
33    "distribution": [
34      { "crawler": "ChatGPT", "count": 892, "percentage": 48.4 }
35    ],
36    "top_crawler": "ChatGPT",
37    "search_bot_visits": 821,
38    "crawl_efficiency_pct": 71.3
39  },
40  "top_pages": [
41    { "url": "https://example.com/pricing", "count": 312, "crawlers": ["ChatGPT", "Perplexity"] }
42  ],
43  "recent_preview": [
44    {
45      "id": "0",
46      "url": "https://example.com/pricing",
47      "platform": "ChatGPT",
48      "intent": "interaction",
49      "visited_at": "2026-04-17T13:11:00Z",
50      "status_code": 200
51    }
52  ]
53}

Overview Example

1curl -H 'Authorization: Bearer $TRAKKR_API_KEY' \
2  'https://api.trakkr.ai/crawler/overview?brand_id=75ffdeb9-0924-4ff9-8ded-c2470d73d224&range_preset=30d&compare_to=previous_period'

1{
2  "brand_id": "75ffdeb9-0924-4ff9-8ded-c2470d73d224",
3  "range": {
4    "range_preset": "30d",
5    "start": "2026-03-18",
6    "end": "2026-04-17",
7    "days": 30,
8    "compare_to": "previous_period"
9  },
10  "state": {
11    "has_tracking_id": true,
12    "has_historical_data": true,
13    "tracking_id": "75ffdeb9-0924-4ff9-8ded-c2470d73d224"
14  },
15  "hero": {
16    "total_visits": 1842,
17    "total_visits_change": 23.5,
18    "unique_pages": 47,
19    "unique_pages_change": 12.0,
20    "avg_daily_visits": 61.4,
21    "top_crawler": "ChatGPT"
22  },
23  "chart": {
24    "series": [
25      { "date": "2026-04-10", "count": 72, "by_crawler": { "ChatGPT": 38, "Perplexity": 21 } }
26    ],
27    "distribution": [
28      { "crawler": "ChatGPT", "count": 892, "percentage": 48.4 }
29    ],
30    "bot_type_summary": { "training": 621, "search": 504, "interaction": 717, "agent": 0, "other": 0 }
31  },
32  "breakdown": {
33    "distribution": [
34      { "crawler": "ChatGPT", "count": 892, "percentage": 48.4 }
35    ],
36    "top_crawler": "ChatGPT",
37    "search_bot_visits": 821,
38    "crawl_efficiency_pct": 71.3
39  },
40  "top_pages": [
41    { "url": "https://example.com/pricing", "count": 312, "crawlers": ["ChatGPT", "Perplexity"] }
42  ],
43  "recent_preview": [
44    {
45      "id": "0",
46      "url": "https://example.com/pricing",
47      "platform": "ChatGPT",
48      "intent": "interaction",
49      "visited_at": "2026-04-17T13:11:00Z",
50      "status_code": 200
51    }
52  ]
53}