Not sure where to start? Take a guided tour.
AI Site Grader
AI Query Scanner
llms.txt GeneratorDoes AI search your exact words?
No. AI almost never searches the exact wording a user provides. Out of 11,521 prompt-query pairs, the exact match rate was only 0.17%, with an average Jaccard similarity of 25.24%.
The key numbers.
- Prompt-query pairs
- Captured prompt-to-query transformations.
- 11,521
- Exact match rate
- Only 20 of 11,521 pairs matched exactly.
- 0.17%
- Average similarity
- Average Jaccard similarity between prompt and search query.
- 25.24%
Teams that optimize only for the literal prompt phrasing miss the actual query shapes the model uses to find evidence.
The evidence.
| Measure | Result |
|---|---|
| Prompt-query pairs Captured prompt-to-query transformations. | 11,521 |
| Exact match rate Only 20 of 11,521 pairs matched exactly. | 0.17% |
| Average similarity Average Jaccard similarity between prompt and search query. | 25.24% |
Population and method
Built from 11,521 captured prompt-to-query pairs observed in OpenAI web search calls, with 100% search-query coverage in the sampled dataset.
Where the evidence stops
This study reflects captured search-query transformations rather than every hidden retrieval step a model may perform.
Some inserted entities reflect category heuristics or market leaders rather than final recommendation outcomes.
The underlying dataset is concentrated in commercial and research prompts rather than all possible conversational traffic.
What to do with this
- Write for the query the model is likely to search, not just the wording a human might type first.
- Add current-year, format, and category framing where it genuinely helps the page answer the query.
- Make sure competitor and category language is present when the model is likely to inject it during retrieval.
Operators must measure and optimize for the transformed search queries rather than raw user inputs to ensure their content is retrieved during the AI search process.
Questions about this answer.
Cite this answer.
Source study: Query Translation“No. AI almost never searches the exact wording a user provides. Out of 11,521 prompt-query pairs, the exact match rate was only 0.17%, with an average Jaccard similarity of 25.24%.”
Mack Grenfell · Does AI search your exact words?
How AI Translates Your Questions · Trakkr Research · Updated 22 January 2026