What does the crawler data say about site architecture? | Trakkr Research
It says AI crawlers are biased toward easy-to-hit pages and often do not traverse deeply enough to rescue weak structure. The architecture has to expose useful answers early.
Methodology: Built from 575,788 AI crawler visits across 84 brands and 314,501 unique URLs observed between June 11, 2025 and February 1, 2026.
Direct Answer
Mostly, AI crawlers are biased toward easy-to-hit pages and often do not traverse deeply enough to rescue weak structure. The architecture has to expose useful answers early.
What this means
Operators must adjust site hierarchy and internal linking strategies because relying on deep crawls will fail to index critical content, directly impacting visibility in AI search features.
Evidence table
| Metric | Value | Why it matters |
|---|---|---|
| One-visit sessions | 88.5% | Most AI crawler visits are one-and-done sessions. |
| Unique URLs | 314,501 | Distinct pages visited by AI crawlers. |
| GPTBot homepage rate | 2.75% | GPTBot rarely starts on the homepage. |
Frequently Asked Questions
How often do AI crawlers visit multiple pages in a single session?
Most AI crawler visits are one-and-done sessions, with one-visit sessions accounting for 88.5% of the total.
Do AI crawlers typically start their crawl from the homepage?
GPTBot rarely starts on the homepage, showing a GPTBot homepage rate of only 2.75%.
How many distinct pages did AI crawlers visit in the study?
The crawlers visited 314,501 unique URLs.
What to do next
Related pages
Continue through the same study cluster.
- can you treat ai crawler traffic like normal seo bot traffic - Related answer page
- what is the main operational takeaway from the crawler study - Related answer page
- gptbot controls more than half of observed ai crawler traffic - Related fact page
- crawler share tracker - Related tracker page
Data & Sources
- When AI Comes to Your Website - Flagship study behind this page
- Page JSON - Machine-readable companion file