thedonkeysanctuary.org.uk — AI Site Grade

Analysis

Cloudflare JS Challenge Blocks Every AI Crawler From All Site Content

The Donkey Sanctuary's live site is entirely inaccessible to AI crawlers. Every single bot tested — GPTBot, ClaudeBot, PerplexityBot, Google-Extended, ChatGPT-User, OAI-SearchBot, Applebot-Extended, and anthropic-ai — receives a 403 response from Cloudflare's JS challenge page, returning zero words of visible content. The homepage, robots.txt, llms.txt, and sitemap.xml all return the same Cloudflare "Just a moment..." wall. No bot reaches a single page of real content.

Crawler Access

The robots.txt (captured via Wayback Machine from January 2025) is a Drupal 7-era file that blocks AhrefsBot, Baiduspider, MJ12bot, Yandex, and dozens of other scrapers with Disallow: /, but has no rules for GPTBot, ClaudeBot, PerplexityBot, Google-Extended, or any other AI-specific user-agent. The llms.txt endpoint returns a 403 Cloudflare challenge — it does not exist as a real file. The sitemap index (sitemap.xml) points to two sub-sitemaps (sitemap-main.xml and shop/media/sitemap.xml), but both are also behind the Cloudflare wall. The site runs on nginx behind Cloudflare with aggressive JS challenge enforcement, hosted on AWS (IP 52.208.5.150). The homepage meta robots tag served by Cloudflare is noindex,nofollow, though the real homepage (via Wayback) uses index, follow.

Cold-Knowledge Gap

LLM cold knowledge about The Donkey Sanctuary is strong and accurate: founded 1969 by Dr. Elisabeth Svendsen, world's largest donkey sanctuary, 6,000+ donkeys cared for, operations in 30+ countries, campaigns against the donkey skin trade (ejiao), sponsor-a-donkey program, Sidmouth visitor centre. This knowledge aligns closely with the homepage content captured via Wayback — the site's mission ("world leaders in donkey welfare"), impact figures (4.6 million donkeys in 2024), and key campaigns (End the skin trade) are all present. The gap is not in accuracy but in freshness: the cold knowledge describes the brand well, but AI models cannot verify or update that knowledge against the live site because they cannot access it.

Schema Posture

The homepage contains a well-structured JSON-LD schema block with Organization, WebPage, and WebSite types. The Organization schema includes sameAs links to Facebook, Instagram, YouTube, TikTok, and LinkedIn. The WebSite schema includes a SearchAction potential action. However, the schema is missing NGO or Charity types, FaqPage markup despite FAQ-style content, and Event schema for the Sidmouth visitor centre. No Article or NewsArticle schema appears on the news section. The schema URLs point to Wayback Machine proxy URLs rather than the live domain, which would confuse crawlers that do reach the archived content.

External Signals

No external search results were retrievable via the tool, but the cold knowledge indicates strong reputational signals: praise for ethical stance against the donkey skin trade, transparent financial reporting, and large-scale international operations. The site's own content (via Wayback) shows a resource hub with extensive factsheets for donkey owners and veterinary professionals, a news section with recent articles, and a donation funnel with specific impact metrics (74p of every £1 goes to helping donkeys). The site runs on Drupal (evident from the robots.txt paths and theme name "rutherford"), which carries JS-rendering risk for some interactive elements.

Findings

1. Cloudflare JS challenge blocks all AI crawlers from all site content High

   Every AI crawler tested receives a 403 response from Cloudflare's JS challenge page, returning zero words of visible content. The homepage, robots.txt, llms.txt, and sitemap.xml are all behind the same wall.

   What to change: Remove the JS challenge for known AI crawler user-agents, or serve a static HTML version of the homepage and key pages to bots. Add a robots.txt that allows AI crawlers and a valid llms.txt file.

2. Robots.txt has no rules for AI crawlers and blocks many scrapers High

   The robots.txt (from January 2025) blocks AhrefsBot, Baiduspider, MJ12bot, Yandex, and others with Disallow: /, but has no directives for GPTBot, ClaudeBot, PerplexityBot, Google-Extended, or other AI-specific user-agents.

   What to change: Update robots.txt to explicitly allow AI crawlers (e.g., GPTBot, ClaudeBot, PerplexityBot, Google-Extended) and remove blanket disallows for legitimate bots.

3. llms.txt endpoint returns 403 and does not exist Medium

   The llms.txt file returns a Cloudflare 403 challenge, indicating it is not published. AI crawlers cannot discover a structured summary of the site's content.

   What to change: Create and serve a valid llms.txt file that lists key pages and content summaries for AI crawlers.

4. Sitemap and sub-sitemaps are inaccessible behind Cloudflare wall High

   The sitemap.xml points to two sub-sitemaps, but both are behind the Cloudflare JS challenge and return 403 to crawlers.

   What to change: Ensure sitemap.xml and sub-sitemaps are accessible to crawlers without JS challenges.

5. Homepage meta robots tag served by Cloudflare is noindex,nofollow High

   The Cloudflare-served homepage includes a meta robots tag of noindex,nofollow, which would prevent indexing if a crawler could reach it. The real homepage (via Wayback) uses index, follow.

   What to change: Remove the noindex,nofollow meta tag from the Cloudflare response for the homepage.

6. JSON-LD schema URLs point to Wayback Machine proxy instead of live domain Medium

   The schema.org URLs in the homepage JSON-LD use Wayback Machine proxy URLs (web.archive.org) rather than the live domain, which would confuse crawlers that do reach the archived content.

   What to change: Update the JSON-LD schema to use the live domain URLs (https://www.thedonkeysanctuary.org.uk/) instead of Wayback Machine proxy URLs.

7. Missing NGO or Charity schema types on homepage Medium

   The homepage JSON-LD uses Organization, WebPage, and WebSite types but does not include NGO or Charity schema types, which are more specific for a charitable organization.

   What to change: Add NGO or Charity schema types to the JSON-LD to better describe the organization.

8. Missing FaqPage and Event schema despite relevant content Low

   The site contains FAQ-style content and a Sidmouth visitor centre, but no FaqPage or Event schema markup is present.

   What to change: Add FaqPage schema to FAQ sections and Event schema for visitor centre events.

9. No Article or NewsArticle schema on news section Medium

   The news section lacks Article or NewsArticle schema markup, reducing the chance of appearing in Google News or AI-generated news summaries.

   What to change: Add Article or NewsArticle schema to news articles.

10. Drupal site carries JS-rendering risk for interactive elements Low

    The site runs on Drupal (evident from robots.txt paths and theme name), which may use JavaScript for some interactive elements, potentially hiding content from crawlers that do not execute JS.

    What to change: Ensure critical content is server-side rendered or statically available without JavaScript.

11. No external search results found for the domain or brand Medium

    Multiple web searches for the domain and brand returned zero results, indicating poor external visibility or indexing issues.

    What to change: Investigate why the site is not indexed by search engines; ensure noindex is removed and sitemaps are accessible.

What's working

• LLM cold knowledge about the charity is strong and accurate — LLMs have accurate knowledge about The Donkey Sanctuary: founded 1969, world's largest donkey sanctuary, 6,000+ donkeys, operations in 30+ countries, campaigns against donkey skin trade, sponsor-a-donkey program, Sidmouth visitor centre.
• Homepage includes well-structured JSON-LD schema with Organization, WebPage, WebSite — The homepage JSON-LD includes Organization, WebPage, and WebSite types with sameAs links to social media and a SearchAction potential action.
• Homepage content via Wayback shows strong mission, impact figures, and campaigns — The archived homepage clearly states the mission ('world leaders in donkey welfare'), impact figures (4.6 million donkeys in 2024), and key campaigns (End the skin trade).
• Site includes extensive resource hub with factsheets for donkey owners and vets — The site (via Wayback) shows a resource hub with factsheets for donkey owners and veterinary professionals, providing valuable content.
• Donation funnel includes specific impact metrics (74p of every £1) — The donation page highlights that 74p of every £1 goes to helping donkeys, a strong trust signal.
• Cold knowledge indicates strong reputational signals: ethical stance, transparent finances, international operations — LLMs know the charity is praised for its ethical stance against the donkey skin trade, transparent financial reporting, and large-scale international operations.