royalautogroup.com — AI Site Grade

Analysis

Akamai WAF Blocks Every Crawler Except Anthropic — Site Invisible to AI Search

The site's Akamai edge firewall returns 403 Access Denied to every user-agent tested — GPTBot, ClaudeBot, PerplexityBot, Google-Extended, OAI-SearchBot, ChatGPT-User, and even a standard browser — while anthropic-ai alone receives a 200 with full HTML content (357KB). This creates a bizarre single-crawler funnel: Claude can read the site, but no other AI engine can reach it.

Crawler Access

The robots.txt (served only to anthropic-ai; all others get 403) contains explicit disallow rules for GPTBot, OAI-SearchBot, ChatGPT-User, Claude-User, Claude-SearchBot, CCBot, and PerplexityBot — but only for static assets (/*.js, /*.css, /*.json, /api/, /apis/, /pixall/). The homepage and content paths are not disallowed in the robots.txt. The actual blocking happens at the Akamai WAF layer, which rejects all traffic except anthropic-ai. The llms.txt exists and returns 200 to anthropic-ai with 92KB of structured site links and descriptions — a well-formed file that no other AI crawler can access. The sitemap.xml contains hundreds of URLs including inventory pages, service pages, and promotions, but is also only reachable by anthropic-ai.

Cold-Knowledge Gap

LLM knowledge of "Royal Auto Group" returns a blank: the model states it has no specific, verified information and describes the name as generic. The actual site reveals a multi-franchise dealership in Cortland, NY selling Chevrolet, Subaru, Nissan, and Buick new vehicles plus used inventory, with a full service center. The site uses the Dealer.com (DDC) platform and has Google Search Console verification (n7-VCBgM5DfXiX9pspGjSTbWkbVFgCnS0Bo2-acoYEI) and Bing verification — suggesting the owner expects organic search traffic that the WAF is blocking.

Schema Posture

The homepage HTML (visible only to anthropic-ai) contains no JSON-LD schema of any kind — no AutoDealer, LocalBusiness, Product, or Organization markup. The page uses CSS classes like multi-franchise, chevrolet, subaru, nissan and has <meta name="robots" content="index, follow"> on interior pages, but the structured data layer is entirely absent. The about/index.htm page returns a 404 (custom error page), meaning the dealership has no dedicated about page in the sitemap.

External Signals

Web searches for "Royal Auto Group Cortland", "royalautogroup.com", and the dealership name return zero indexed results across multiple queries. No reviews, Reddit threads, press mentions, or dealer directory listings surfaced. The site is effectively invisible to web search engines — the Akamai WAF blocks Googlebot and Bingbot at the edge, preventing any indexing. The only external signals are DNS records pointing to Akamai and Microsoft 365 email (Outlook protection).

Findings

1. Akamai WAF blocks all crawlers except Anthropic High

   The Akamai edge firewall returns 403 Access Denied to GPTBot, ClaudeBot, PerplexityBot, Google-Extended, OAI-SearchBot, ChatGPT-User, and standard browsers, while only anthropic-ai receives a 200 response with full HTML content.

   What to change: Reconfigure the Akamai WAF to allow access from major AI crawlers (GPTBot, ClaudeBot, PerplexityBot, Google-Extended, OAI-SearchBot) and web search bots (Googlebot, Bingbot) to the homepage and content paths.

2. robots.txt only accessible to Anthropic crawler High

   The robots.txt file returns 403 to all user-agents except anthropic-ai, which receives a 200 with 3229 bytes containing disallow rules for static assets and API paths for GPTBot, OAI-SearchBot, ChatGPT-User, Claude-User, Claude-SearchBot, CCBot, and PerplexityBot.

   What to change: Make robots.txt publicly accessible and remove disallow rules for AI crawlers on content paths, or ensure the WAF does not block access to robots.txt.

3. Sitemap XML only accessible to Anthropic crawler High

   The sitemap.xml file returns 403 to all user-agents except anthropic-ai, which receives a 200 with 61396 bytes containing hundreds of URLs including inventory, service, and promotion pages.

   What to change: Make sitemap.xml publicly accessible so all crawlers can discover the site's URL structure.

4. llms.txt only accessible to Anthropic crawler High

   The llms.txt file returns 403 to all user-agents except anthropic-ai, which receives a 200 with 92039 bytes of structured site links and descriptions.

   What to change: Make llms.txt publicly accessible so all AI crawlers can benefit from the structured site information.

5. No JSON-LD schema on homepage High

   The homepage HTML contains no JSON-LD structured data for AutoDealer, LocalBusiness, Product, or Organization markup, missing a key opportunity for AI visibility.

   What to change: Add JSON-LD schema markup for AutoDealer, LocalBusiness, and Product to the homepage and inventory pages.

6. About page returns 404 Medium

   The about/index.htm page returns a 404 error, meaning the dealership has no dedicated about page despite being listed in the sitemap.

   What to change: Create a proper about page with dealership history, location, and team information, and ensure it returns a 200 status.

7. Zero web search indexing High

   Web searches for the dealership name, domain, and location return zero indexed results, indicating the site is invisible to search engines due to the WAF blocking Googlebot and Bingbot.

   What to change: Allow Googlebot and Bingbot through the Akamai WAF to enable search engine indexing.

8. Cold knowledge gap for LLMs Medium

   LLM knowledge of 'Royal Auto Group' returns no specific information, describing the name as generic, despite the site being a multi-franchise dealership with Chevrolet, Subaru, Nissan, and Buick.

   What to change: Ensure the site is accessible to AI crawlers and add structured data to help LLMs learn about the dealership.

What's working

• Well-formed llms.txt with structured site links — The llms.txt file is well-formed, contains 92KB of structured site links and descriptions, and is accessible to the Anthropic crawler, providing a strong foundation for AI visibility.
• Comprehensive sitemap.xml with hundreds of URLs — The sitemap.xml contains hundreds of URLs including inventory, service, and promotion pages, providing a complete site map for crawlers.
• Anthropic crawler receives full HTML content — The Anthropic crawler receives a 200 response with 357KB of full HTML content on the homepage, including inventory and service information.
• Google Search Console and Bing verification present — The site has Google Search Console and Bing verification, indicating the owner expects organic search traffic and has taken steps to manage indexing.