Query transformation severity tracker | Trakkr Research
Query transformation severity tracker for the study How AI Translates Your Questions.
Methodology: Built from 11,521 captured prompt-to-query pairs observed in OpenAI web search calls, with 100% search-query coverage in the sampled dataset.
Summary
The benchmark data indicates a consistent pattern where AI models actively restructure user prompts rather than passing them directly to search systems. Exact matches are statistically negligible at 0.17 percent, while expanded queries account for 55.5 percent of the dataset, demonstrating a tendency to lengthen and constrain search parameters. Complete rewrites occur in 31.85 percent of cases, resulting in an average Jaccard similarity of 25.24 percent between the original prompt and the final query.
Benchmark rows
| Metric | Value | Context |
|---|---|---|
| Exact match rate | 0.17% | Only 20 of 11,521 pairs matched exactly. |
| Average similarity | 25.24% | Average Jaccard similarity between prompt and search query. |
| Complete rewrites | 31.85% | 3,670 pairs fell into the complete rewrite bucket. |
| Expanded queries | 55.5% | AI made the query longer in 6,392 cases. |
Ranked view
| Item | Value | Detail |
|---|---|---|
| Expanded queries | 55.5% | More than half of queries grow longer and more constrained. |
| Complete rewrites | 31.85% | Nearly one third of pairs keep very little original phrasing. |
| Average similarity | 25.24% | Prompt and search query often overlap far less than teams assume. |
| Exact matches | 0.17% | Literal user phrasing is almost never the final retrieval query. |
Related pages
Continue through the same study cluster.
- how aggressive are ai query rewrites - Related answer page
- does ai add current year terms to searches - Related answer page
- average prompt to query similarity is only twenty five percent - Related fact page
Data & Sources
- How AI Translates Your Questions - Flagship study behind this page
- Page JSON - Machine-readable companion file