widefitshoes.co.uk — AI Site Grade

Analysis

Cloudflare JS Challenge Blocks Every AI Crawler — Site Is Invisible to All Bots

Every single AI crawler — GPTBot, ClaudeBot, PerplexityBot, Google-Extended, OAI-SearchBot, ChatGPT-User, Applebot-Extended, Bytespider, anthropic-ai — receives a 403 from Cloudflare's JS challenge wall on every URL tested (homepage, /products, /collections, /robots.txt, /llms.txt). The browser baseline also gets 403. The site is a Shopify storefront (23.227.38.65, Cloudflare DNS) that requires JavaScript execution and cookie acceptance before any content is served. No AI crawler can render the JS challenge, so none ever sees a single byte of product content.

Crawler Access

The robots.txt (retrieved via Wayback Machine) explicitly allows GPTBot, ChatGPT-User, and PerplexityBot with Allow: / directives — the site operator intended these bots to crawl freely. But the Cloudflare "Under Attack" / JS challenge mode (cType: managed) overrides this entirely. The llms.txt returns the same 403 wall. The sitemap index exists (4 sub-sitemaps for products, pages, collections, blogs) but is also inaccessible to crawlers. The site has zero organic search visibility — a site:widefitshoes.co.uk search returns no results.

Content & Schema

The archived homepage (Wayback Machine, March 2026) reveals a mature Shopify store with 1,740+ words of visible content, ShoeStore and WebSite JSON-LD schema, a physical address (33 Kenton Park Parade, London HA3 8DN), phone number (0208 907 1742), and price range (£50–£250). The site carries major brands (New Balance, Nike, Skechers, Ecco, DB Shoes, Padders, HJ Hall) and offers widths from 2E to 8E. The about page claims the business has been operating since 1965 and is "approved by over 9000 health professionals." The blog has 20+ articles, the most recent from January 2025, covering foot conditions (bunions, diabetes, arthritis, plantar fasciitis, lymphoedema, gout, swollen feet). Product pages include width selectors and Trustpilot badges. However, the ShoeStore schema's GeoCoordinates has empty latitude/longitude values, and no Product schema was found on the archived product listing pages.

Cold-Knowledge Gap

The LLM's cold knowledge accurately describes the brand as a UK-based wide-fit specialist stocking New Balance, Skechers, Hotter, Clarks, and FitFlop — but the actual site does not carry Hotter, Clarks, or FitFlop. The live site's brand roster is New Balance, Nike, Skechers, Ecco, DB Shoes, Padders, Roamers, I-Runner, Dek, Tredd Well, and HJ Hall. The model also mentions a "Wide Fit Shoes" own-label budget line — no evidence of this exists in the archived site content. The model cites "mixed" Trustpilot reviews; the site prominently displays Trustpilot badges on every product but no aggregate rating schema was found.

External Signals

The domain has zero indexed external mentions across search engines. No Trustpilot profile surfaced, no Reddit threads, no press coverage. The DNS TXT records show verification tokens for Google, Apple, Pinterest, and Klaviyo (email marketing), plus a Shopify SPF record — indicating a mature ecommerce setup that has simply been locked behind Cloudflare's strictest protection mode.

Findings

1. Cloudflare JS challenge blocks every AI crawler High

   Every AI crawler (GPTBot, ClaudeBot, PerplexityBot, Google-Extended, OAI-SearchBot, ChatGPT-User, Applebot-Extended, Bytespider, anthropic-ai) receives a 403 from Cloudflare's JS challenge wall on all tested URLs. The site requires JavaScript execution and cookie acceptance, which no crawler can perform.

   What to change: Disable JS challenge mode for known AI crawler user agents, or serve a static HTML version of the site to bots.

2. Robots.txt allowances overridden by Cloudflare High

   The robots.txt explicitly allows GPTBot, ChatGPT-User, and PerplexityBot with Allow: / directives, but Cloudflare's JS challenge blocks them regardless.

   What to change: Configure Cloudflare to bypass JS challenge for verified bot user agents listed in robots.txt.

3. llms.txt returns 403 High

   The llms.txt file, intended to provide AI-friendly content, returns a 403 due to the Cloudflare wall.

   What to change: Ensure llms.txt is accessible to AI crawlers by excluding it from JS challenge.

4. Sitemap inaccessible to crawlers High

   The sitemap index and sub-sitemaps exist but are blocked by Cloudflare, preventing crawlers from discovering URLs.

   What to change: Allow crawler access to sitemap files by excluding them from JS challenge.

5. Zero organic search visibility High

   A site:widefitshoes.co.uk search returns no results, and no external mentions exist across search engines, Trustpilot, or social media.

   What to change: Resolve crawler access issues to allow indexing; build external backlinks and reviews.

6. Empty GeoCoordinates in ShoeStore schema Medium

   The ShoeStore JSON-LD schema includes GeoCoordinates with empty latitude and longitude values, which may confuse AI parsers.

   What to change: Populate GeoCoordinates with the actual store location (33 Kenton Park Parade, London HA3 8DN).

7. Missing Product schema on product pages Medium

   Archived product listing pages lack Product JSON-LD schema, which is critical for AI visibility of individual items.

   What to change: Add Product schema with name, description, price, availability, and brand to all product pages.

8. Missing AggregateRating schema despite Trustpilot badges Medium

   Trustpilot badges are displayed on product pages, but no AggregateRating schema is present to convey review data to AI crawlers.

   What to change: Add AggregateRating schema with review count and average rating from Trustpilot.

9. Cold knowledge brand roster differs from actual inventory Medium

   LLM cold knowledge lists Hotter, Clarks, and FitFlop as carried brands, but the site actually stocks New Balance, Nike, Skechers, Ecco, DB Shoes, Padders, Roamers, I-Runner, Dek, Tredd Well, and HJ Hall.

   What to change: Ensure accurate brand information is prominently displayed and structured on the site to correct AI knowledge.

10. No external signals or backlinks Medium

    The domain has zero indexed external mentions, no Trustpilot profile, no Reddit threads, and no press coverage, limiting AI visibility.

    What to change: Build external backlinks through PR, partnerships, and review platforms; claim Trustpilot profile.

What's working

• Rich homepage content with 1,740+ words — The homepage contains over 1,740 words of descriptive content about wide-fit shoes, including brand mentions and width options, which is valuable for AI understanding.
• ShoeStore and WebSite JSON-LD schema present — The homepage includes ShoeStore and WebSite schema markup, providing structured data about the business type and site.
• Physical address and phone number provided — The site lists a physical address (33 Kenton Park Parade, London HA3 8DN) and phone number (0208 907 1742), aiding local SEO and credibility.
• Blog with 20+ articles on foot conditions — The blog covers foot conditions like bunions, diabetes, arthritis, and plantar fasciitis, providing authoritative content that can attract AI attention.
• Mature Shopify setup with DNS verification tokens — DNS TXT records show verification for Google, Apple, Pinterest, and Klaviyo, indicating a well-configured ecommerce platform.
• Robots.txt explicitly allows key AI bots — The robots.txt file contains Allow directives for GPTBot, ChatGPT-User, and PerplexityBot, showing intent to be crawled.
• Sitemap index with sub-sitemaps exists — A sitemap index file references four sub-sitemaps for products, pages, collections, and blogs, indicating a structured site architecture.
• Trustpilot badges displayed on product pages — Product pages show Trustpilot badges, signaling social proof to users (though not to crawlers due to missing schema).