desmondanddempsey.com — AI Site Grade

Analysis

I have enough data to write the audit now. Let me compile the findings.

---

Cloudflare JS Challenge Blocks Every AI Crawler — and Every Browser

The live site at desmondanddempsey.com returns a 403 Cloudflare JS challenge wall to all traffic — every AI crawler (GPTBot, ClaudeBot, PerplexityBot, Google-Extended, OAI-SearchBot, Bytespider, Applebot-Extended, anthropic-ai) and every browser UA alike. The compare_bot_access test confirmed all 11 UAs receive the same "Verifying your connection..." HTML shell with zero visible content. No AI crawler has accessed a single page of real content in the current state.

Crawler Access

The site has no robots.txt accessible — Cloudflare blocks it. No llms.txt exists (returns 403). No sitemap is reachable. The DNS resolves to 23.227.38.32 (Shopify's IP range), and the site runs on Cloudflare with managed challenge mode enabled. The X-Frame-Options: SAMEORIGIN and Permissions-Policy headers are set, but no Strict-Transport-Security header was observed. The Cloudflare challenge requires JavaScript execution and cookie acceptance — a hard wall for all text-based crawlers.

Cold-Knowledge Gap

The LLM's prior knowledge about Desmond & Dempsey is surprisingly detailed — it knows the brand was founded in 2014 by Molly and Joel, named after their grandfathers, produces hand-painted prints on organic cotton, is B Corp certified, and makes products in Portugal. However, none of this information is currently machine-readable on the live site. The Wayback snapshot confirms the site is a Shopify storefront with rich product content (pyjama sets at £95–£170, silk ranges, monogramming, 90-day returns), but the live site serves zero structured data to any crawler.

Schema Posture

The homepage and all key pages (About Us, Sustainability, Shipping & Returns) contain zero JSON-LD schema of any type — no Organization, no Product, no FAQPage, no BreadcrumbList. The meta description on the homepage reads "Shop our range of cotton pyjamas and luxury silk sleepwear with unique prints for women and men. Free UK & US Shipping." but this text is only served to browsers that pass the JS challenge. The site has no blog (the /blogs/journal path returns 404 in the Wayback Machine).

External Signals

The brand has virtually no indexed external presence on DuckDuckGo — searches for the brand name, founders, products, and reviews all returned zero results. No Trustpilot, Reddit, or press mentions surfaced. The DNS TXT records show Google site verification, Klaviyo, Facebook domain verification, and Microsoft verification — indicating active marketing stack usage — but none of this translates to discoverable off-domain signals that AI engines could cite. The Wayback Machine shows the site has been captured over 2,000 times since 2014, confirming it is an established brand that has recently become completely opaque to automated access.

Findings

1. Cloudflare JS challenge blocks all AI crawlers and browsers High

   The live site returns a 403 Cloudflare JS challenge wall to every user agent, including all major AI crawlers (GPTBot, ClaudeBot, PerplexityBot, etc.). No crawler can access any page content.

   What to change: Disable the JS challenge for known AI crawler user agents by configuring Cloudflare WAF to allow GPTBot, ClaudeBot, and other listed bots, or serve a static HTML version to those UAs.

2. No accessible robots.txt file High

   The robots.txt endpoint returns a 403 error, so crawlers cannot discover allowed or disallowed paths.

   What to change: Serve a robots.txt file that allows AI crawlers to access the site, and ensure it is not blocked by Cloudflare.

3. No llms.txt file Medium

   The llms.txt endpoint returns a 403, so LLMs have no machine-readable guide to the site's content.

   What to change: Create an llms.txt file that lists key pages and provides a brief overview of the brand for LLM consumption.

4. No accessible sitemap High

   The sitemap endpoint returns a 403, preventing crawlers from discovering the site's URL structure.

   What to change: Serve a sitemap.xml that lists all important pages, and ensure it is not blocked by Cloudflare.

5. Zero JSON-LD schema on any page High

   The homepage, About Us, Sustainability, and Shipping & Returns pages contain no structured data markup (Organization, Product, FAQPage, etc.).

   What to change: Add JSON-LD schema for Organization, Product (on product pages), and BreadcrumbList to all relevant pages.

6. No indexed external presence on DuckDuckGo High

   Searches for the brand name, founders, products, and reviews returned zero results on DuckDuckGo, indicating no indexed external signals.

   What to change: Build external backlinks through press coverage, influencer partnerships, and listings on review platforms like Trustpilot to improve off-domain signals.

7. Blog path returns 404 Medium

   The /blogs/journal path returns a 404, so there is no blog content for crawlers to index.

   What to change: Restore or create a blog with articles about the brand, products, and sustainability to provide crawlable content.

8. Missing Strict-Transport-Security header Low

   The site does not send an HSTS header, which is a security best practice and can affect trust signals.

   What to change: Add a Strict-Transport-Security header with a max-age of at least one year.

What's working

• LLM prior knowledge contains rich brand details — The LLM knows Desmond & Dempsey was founded in 2014 by Molly and Joel, produces hand-painted prints on organic cotton, is B Corp certified, and manufactures in Portugal. This provides a foundation for AI visibility if the site becomes accessible.
• Established Wayback Machine history with over 2,000 captures — The site has been archived over 2,000 times since 2014, confirming it is an established brand with historical content that could be referenced.
• Active marketing stack with DNS verification records — DNS TXT records show Google site verification, Klaviyo, Facebook domain verification, and Microsoft verification, indicating an active marketing and analytics setup.