AI Site Grade
wfw.com — AI Site Grade
WFW.com blocks OAI-SearchBot while serving JavaScript shells to all AI crawlers, rendering thousands of articles invisible.
WFW.com's selective bot blocking and JavaScript-rendered pages create a critical AI visibility gap, despite proactive Anthropic verification.
- Findings
- 9
- Evidence checks
- 26
- Completed
- 30 May 2026
Analysis
WFW.com — AI-Visibility Audit
OAI-SearchBot is actively blocked by Sucuri's WAF (403), while every other major AI crawler — GPTBot, ClaudeBot, PerplexityBot, Google-Extended, ChatGPT-User, anthropic-ai, Applebot-Extended, Bytespider — receives a full 200 with identical content to a browser. This selective blocking means OpenAI's search and retrieval products (ChatGPT web search, SearchGPT) cannot access wfw.com, creating a blind spot in the most consumer-facing AI discovery channel.
Crawler Access
The site sits behind Sucuri/Cloudproxy (WAF + CDN), with strong security headers including HSTS, X-Frame-Options SAMEORIGIN, and a CSP. The robots.txt is a bare Yoast-generated file with no AI-bot-specific directives — no Disallow for GPTBot, ClaudeBot, or any other crawler. The only disallowed paths are seven PDF files from 2023. llms.txt returns 404, meaning no AI-friendly content map exists. The sitemap index is healthy (11 sub-sitemaps, thousands of URLs), but the list_known_urls tool could not parse it — the sitemap is present but the domain's root-level sitemap discovery failed, suggesting a redirect chain issue (wfw.com redirects to www.wfw.com).
Content & Schema Posture
The homepage and all key pages are JavaScript-rendered shells — the plain GET returns under 50 words of visible text on most pages. The homepage yielded only 13 words; the About Us page returned just "1982 Founded"; the Data Centres article returned 4 words. This is a critical problem: AI crawlers that do not execute JavaScript (including GPTBot, ClaudeBot, and Google-Extended) receive near-empty pages. JSON-LD schema is present (Organization, WebSite, BlogPosting, BreadcrumbList) but contains stale references to a dev2 subdomain (dev2.wfw.temporarywebsiteaddress.com) for the logo URL — a development/staging URL leaking into production schema on every page. The Insights page schema description contains Lorem Ipsum placeholder text. No FAQ schema, no HowTo, no Article with full body content in schema.
Cold-Knowledge Gap
The LLM cold-knowledge snapshot correctly identifies WFW as a London-founded (1982) sector-focused firm in shipping, aviation, and energy finance with offices in maritime hubs. However, the site's actual positioning emphasizes energy, infrastructure, and transport as three equal pillars — the cold model still frames it primarily as a "shipping and aviation finance boutique." The site's current flagship product, GARI (Global Aviation Resource Index), a proprietary legal analytics tool, is entirely absent from the model's prior knowledge. The model also does not know about WFW's 2026 content activity — Middle East crisis guidance, data centre legal frameworks, or the "Shaping 2026" Germany report.
External Signals
External search results for the firm returned zero organic results across multiple queries — no press mentions, no directory listings, no Reddit threads surfaced. This is unusual for a global law firm with 19 offices and likely reflects either low off-domain citation volume or the search tool's limitations. The DNS TXT records reveal an anthropic-domain-verification token, confirming WFW has proactively registered with Anthropic's crawler verification system — a deliberate AI-visibility step that contrasts with the lack of an llms.txt or any robots.txt AI-bot rules.
Surprising Findings
The dev2 staging domain in production schema is a technical debt risk — every page's Organization and Article schema points logo images to dev2.wfw.temporarywebsiteaddress.com. If that staging environment goes offline, all schema logos break. The Lorem Ipsum in the Insights page schema description suggests the page was launched without QA on structured data. The GARI tool (/gari) is a single-page app that shows "Loading..." to crawlers — its canonical URL points to gari.wfw.com (a subdomain), creating a cross-domain canonical signal. The site has ~3,000+ press releases and articles in the sitemap, but most article pages return only 2-4 words of visible text to crawlers, making the entire content library effectively invisible to non-JS bots.
Findings
OAI-SearchBot blocked by Sucuri WAF High
OAI-SearchBot receives a 403 error from the Sucuri WAF, while all other major AI crawlers get full access. This blocks OpenAI's search and retrieval products from indexing the site.
What to change: Allow OAI-SearchBot through the WAF by whitelisting its user-agent or IP ranges.
Key pages render as JavaScript shells with minimal text High
Homepage, About Us, and article pages return fewer than 50 words of visible text when fetched without JavaScript. AI crawlers that do not execute JS receive near-empty content.
What to change: Implement server-side rendering or dynamic rendering to serve meaningful HTML content to crawlers.
Production schema references dev2 staging domain High
JSON-LD schema on every page uses `dev2.wfw.temporarywebsiteaddress.com` for logo URLs, a development/staging domain that could break if taken offline.
What to change: Replace all staging URLs in schema with production URLs (e.g., `www.wfw.com`).
Insights page schema contains Lorem Ipsum placeholder text Medium
The `description` field in the schema on the Insights page is filled with Lorem Ipsum text, indicating the page was launched without QA on structured data.
What to change: Update the schema description on the Insights page with a meaningful summary.
llms.txt file returns 404 Medium
The site does not provide an `llms.txt` file, missing an opportunity to guide AI crawlers to key content.
What to change: Create an `llms.txt` file listing important pages and summaries for AI crawlers.
GARI tool renders as loading screen to crawlers Medium
The Global Aviation Resource Index page at `/gari` shows only 'Loading...' to crawlers, and its canonical URL points to a subdomain, creating cross-domain canonical confusion.
What to change: Server-side render the GARI content or provide a static summary for crawlers.
LLM cold knowledge omits current positioning and GARI Medium
The model's prior knowledge frames WFW primarily as a shipping and aviation finance boutique, missing the current three-pillar focus on energy, infrastructure, and transport, and the GARI tool.
What to change: Publish more content and structured data emphasizing the current practice areas and GARI to improve LLM knowledge.
Low external citation volume in search results Low
Multiple web searches for the firm returned zero organic results, indicating low off-domain mentions or limited indexing.
Sitemap discovery fails due to redirect chain Low
The root domain wfw.com redirects to www.wfw.com, causing sitemap discovery tools to fail when accessing the root-level sitemap.
What to change: Ensure the sitemap is accessible from both the root and www subdomain, or use a redirect that preserves the sitemap path.
What's working
- Anthropic domain verification token present — DNS TXT records include an `anthropic-domain-verification` token, indicating proactive registration with Anthropic's crawler verification system.
- Robots.txt does not block any AI crawlers — The robots.txt file has no Disallow directives for GPTBot, ClaudeBot, or other AI crawlers, allowing them to crawl the site freely.
- Sitemap index contains thousands of URLs — The sitemap index is present and includes 11 sub-sitemaps with thousands of URLs, providing a comprehensive content map.
- JSON-LD schema present on key pages — Organization, WebSite, BlogPosting, and BreadcrumbList schema are present on the homepage and other pages, providing structured data to crawlers.
- Strong security headers in place — The site uses HSTS, X-Frame-Options SAMEORIGIN, and a Content Security Policy, ensuring secure communication.
- GARI proprietary analytics tool available — The Global Aviation Resource Index is a unique, proprietary legal analytics tool that differentiates WFW in the market.
Track wfw.com across AI search
This is one snapshot. Open the interactive report to inspect evidence, or grade another site free.