Optimize
Optimize scans your site and reports what stops AI from reaching your pages, reading them, and using them as evidence. It appears as Site Optimization in the sidebar and opens at /optimize.
Open it after a site change, or when citations are flat and you want to know whether the problem is your writing or your delivery. A page can rank in search while a bot that runs no JavaScript receives an empty shell.
Optimize needs a Growth plan or higher. A brand viewer can read a scan but cannot start one. The page has three tabs: Findings, Pages and History.
What a scan does
The scan starts from your site, follows what it can discover, and stops at your page budget. It runs in four phases.
Per page it records the status code, HTTPS, indexing directives, canonical, title and description, headings, internal links, alt text, readability, text volume, answer structure, freshness, structured data and response time. The crawl fetches without running JavaScript, so what it reads is close to what ChatGPT, Claude and Perplexity receive. At the domain level it checks robots.txt, the sitemap, llms.txt, mcp.json and security.txt.
The scan makes no model calls. The written read on top of it, and the deeper analysis of a single page, are separate steps you ask for.
Settings in the page header holds the page budget (100, 250 or 500 pages, capped at 250 on Growth and 500 on Scale), priority URLs, exclude patterns, and a weekly scheduled scan. Scan now starts a manual run, limited to one per brand every 24 hours. Export gives you Full report (PDF), Checks (CSV) or Pages (CSV).
Reading Findings
Findings reads in decision order: what is wrong, where the site loses AI, the work, then the record. A first run, a running scan and a stopped scan use the same sections in the same order, so nothing rearranges when data lands.
The verdict
The first band is labeled Priority finding. One sentence, one supporting line of evidence, and one action beside it. The sentence is chosen in a fixed order:
- a robots-level block on AI crawlers, which outranks everything else
- a written headline, but only when it carries a real number
- the biggest measured loss at Available, Reached or Understood
- the first row of the checks table, in that row's own words
- nothing is blocking AI on this site
The band waits for the checks and the journey probe, so it never announces a clean site while data is still loading. Where a written read exists, a second button opens it.
The six stages
A strip of six cells sits under the verdict, one per stage of the site journey. It uses plain labels for the stages that Pages documents in full.
| Cell | Journey stage |
|---|---|
| AI can reach | Available |
| Bots fetch | Reached |
| AI can read | Understood |
| Matches asks | Relevant |
| AI cites | Selected |
| People arrive | Visited |
Every cell counts against the pages measured at that stage, never the whole site, because coverage differs per stage. An unmeasured stage shows an en dash and names what would light it, such as crawler tracking or Search Console. Clicking a measured cell scopes the work list to that stage.
One cell is marked the Bottleneck: the measured stage losing the most pages against its own base. Relevant can never be it, because it is inferred from detected questions and tracked prompts rather than observed.
The work list
One table, ordered by what to fix first. The named slices are What to fix first (the default, holding open and in-progress checks), All checks, Fixed and Ignored. The columns are Work, Evidence, Severity, Pages and Points, with one action button per row.
Three registers do different jobs, and they never borrow each other's words.
| Register | What it does | Example |
|---|---|---|
| Work | Gives an instruction, always a verb | Add schema to the pages AI fetches |
| Evidence | States the measured fact and its base | 74 pages of the 100 crawled carry no JSON-LD |
| Wrong if | Names the case where the finding is a false alarm | Wrong if these templates are deliberately excluded from rich results |
Open a row for the consequence, the caveat, Where it shows up, and How to fix with steps for your detected platform. Copy for a ticket and Send to dev put that brief on your clipboard or into an email. Open these pages sends the affected URLs to the Pages tab, pre-filtered. Ignore stays separate from Fixed.
Where a brand has prompt coverage, a switch beside the filters swaps the table for a per-prompt view. Below it, Notes on this scan collects facts that are not problems: what the crawl missed, and things that look wrong but are deliberate. The last band is the domain's record, which names absent things rather than hiding them: Platform, robots.txt, Sitemap, llms.txt, Rendering, CDN and Bot visits.
The Checks score
The score is deliberately quiet. It sits in the footer row of the checks table and links to History.
Each check on a page counts 1 for a pass, 0.5 for a warning and 0 for a fail, weighted by how much it matters. Serving full content to non-JavaScript bots carries the heaviest weight, then answer structure and content freshness. Three failures cap a page whatever else passes: no usable content caps it at 0, a noindex directive at 10, and a page not on verified HTTPS at 40. The site score averages the page scores, weighted by each page's worth in observed bot fetches, citations and search clicks.
Points is the same score read backwards: the gap between your score and 100, split across the checks in proportion to the share of crawled pages each one fails on. The column sums to that gap, and the footer says what clearing the rows on screen would take the score to.
Render dependency
Many React, Vue and client-rendered Next.js sites send a small HTML shell and add the useful text after JavaScript runs. A browser handles that. AI crawlers do not, with Gemini the exception because it renders through Google's system.
When the crawl's plain fetch misses body text, the check Serve full content to non-JS bots fires and names the pages, and the Rendering fact reads Pre-rendered for bots, Partly rendered for bots or Empty shell for bots. Opening a page shows What bots receive: the same page fetched as GPTBot and through a rendered browser, with the passages a bot loses marked section by section.
The fix is server-side rendering, static generation, or a trusted prerender path, and it comes first: structured data cannot help a crawler that received none of it. JavaScript rendering covers the options, and AI Pages serves clean HTML to crawlers when you cannot change the main site.
Site-level files and access
Four checks describe the domain rather than individual pages.
| File or control | What Optimize checks | Honest reading |
|---|---|---|
robots.txt | Whether AI user agents may fetch the site | A block can be policy. If accidental, nothing else helps until it changes. |
sitemap.xml | Whether URLs are discoverable and still resolve | Discovery builds the page list. It does not promise selection. |
llms.txt | Whether the root file exists and is really text | No AI platform reads it today. Publishing one is a bet. |
mcp.json | Whether a machine-interface descriptor is present | An early signal for agent access, not evidence of citations. |
A valid file closes one access or discovery gap. It is not an outcome.
Fixing from inside Optimize
Eight checks can be drafted and applied for you: titles and descriptions, missing meta descriptions, schema, canonical tags, Open Graph, Twitter cards, llms.txt, and the robots.txt AI block. Those rows read Fix it for me; everything else reads How to fix.
The drawer shows the change as a diff with the evidence behind it, and a chip for each connected destination that supports it: WordPress, Webflow, Shopify or GitHub. GitHub opens a pull request rather than writing to the site. An applied change is re-crawled and marked verified once it is live, and it can be rolled back. The filter band slices the table by pipeline stage, from Untouched to Verified. Drafting is capped at 25 proposals per brand in a rolling 24 hours, and destinations are connected under Sites.
Pages and History
The Pages tab lists every URL this scan touched, with What's wrong, Type, Score, Words, crawls in the last 30 days, the top crawler and an issue count, sliced by All pages, Needs work and Passing. Inspect URL in the header runs a deeper analysis of one address.
That tab is scoped to one scan. The Pages page at /pages is the durable record for a URL, joining the same page identity with crawler activity, citations, traffic, work and results over time. Use the tab to work through a scan, and the Pages page to follow one URL.
The History tab charts the Checks score over time and lists every past scan with what changed. Pick two to compare. Opening an older scan puts the page into that scan's state, with a line at the top saying so.
Common questions
How is this different from an SEO audit?
An SEO audit asks whether a search engine can index and rank a page. Optimize asks whether AI can reach the page, get its useful text, find a direct answer, and use it as evidence. Keep both.
Where did the Issues tab go?
It became the checks table inside Findings. Old ?tab=issues links still open Findings, and an ?issue= link still expands and scrolls to the matching row.
How often should I scan?
Scan after a meaningful site change and compare in History. A weekly scheduled scan catches drift between releases. Manual scans are limited to one per brand every 24 hours.
Can I scan a competitor's site?
No. Optimize is for sites you control. Use Competitors and Citations to study the external pages AI uses for your prompts.
Why did my scan fail?
The usual causes are bot protection at your CDN or firewall, a timeout on a slow server, a DNS or certificate problem, or a page that returned no content. The failure screen names which one it hit. For a firewall block, see Trakkr behind a WAF.
Does publishing llms.txt mean AI will cite me?
No. No AI platform reads that file today. It is a bet on a possible standard, so Optimize says as much on the row rather than awarding you a tick.
What if a check is wrong for my site?
Open the row and read the Wrong if line. If the stated exception fits, ignore the check with that reason. The table exists to make a justified false alarm visible, not to force every site into one template.