Not sure where to start? Take a guided tour.
AI Site Grader
AI Query Scanner
llms.txt GeneratorWhich metrics best summarize cross-model disagreement?
The clearest summary metrics are average agreement, perfect agreement, and the share of high-divergence prompts. In this study those land at 43.3%, 4.0%, and 14.6% respectively.
The key numbers.
- Average agreement
- Mean cross-model agreement rate.
- 43.3%
- Perfect agreement
- Only a small share of prompts produce unanimous outcomes.
- 4.0%
- High divergence rate
- Prompts in the 0-25% agreement bucket.
- 14.6%
Together, they show the center, the extreme, and the risk tail of cross-model recommendation behavior.
The evidence.
| Measure | Result |
|---|---|
| Average agreement Mean cross-model agreement rate. | 43.3% |
| Perfect agreement Only a small share of prompts produce unanimous outcomes. | 4.0% |
| High divergence rate Prompts in the 0-25% agreement bucket. | 14.6% |
Population and method
Built from 797,644 valid comparisons across 44,088 reports and 8 models, covering 6,439,133 model responses in the observed window.
Where the evidence stops
Agreement is measured across recommendation outputs, not across hidden reasoning or retrieval context.
Average agreement can hide large differences between query classes and model pairs.
The study measures overlap, not which answer was objectively “right”.
What to do with this
- Track visibility across multiple models instead of using one platform as a proxy for the whole market.
- Prioritize query classes where disagreement is highest because that is where share can move fastest.
- Treat consensus as a benchmark, but treat divergence as the operating reality.
This answer matters because it turns a study finding into an operating rule teams can use when they decide what to publish, refresh, or measure next.
Questions about this answer.
Cite this answer.
Source study: Model Divergence“The clearest summary metrics are average agreement, perfect agreement, and the share of high-divergence prompts. In this study those land at 43.3%, 4.0%, and 14.6% respectively.”
Mack Grenfell · Which metrics best summarize cross-model disagreement?
Same Question, Different AI, Different Answers · Trakkr Research · Updated 11 March 2026
Apply the finding
Overlap explains agreement, while add AI share of voice to the cross-model scorecard shows which brands own the category across the same prompt set.
Google Search should not be folded into an assistant proxy. Keep AI Overview results in a separate measurement stream and compare it with other surfaces only after each is measured.