bobbyrahal.com — AI Site Grade

Analysis

Akamai Blocks Every Crawler Except Anthropic

The entire bobbyrahal.com domain is behind an Akamai WAF that returns HTTP 403 to every browser and nearly every AI crawler — including GPTBot, ClaudeBot, Google-Extended, PerplexityBot, OAI-SearchBot, and ChatGPT-User — but anomalously allows anthropic-ai through to an nginx origin serving full HTML. This creates a bizarre single-bot visibility funnel: only Claude's training crawler sees the site's actual content, while every other major AI model is locked out at the edge.

Crawler Access

The robots.txt file (accessible only via anthropic-ai; returns 403 to all other user-agents) explicitly disallows GPTBot, OAI-SearchBot, ChatGPT-User, Claude-User, and Claude-SearchBot from /api/, /apis/, /pixall/, and all .js/.css/.json files — but does not block those bots from the main HTML pages. The Akamai layer makes that distinction moot: those bots never reach the origin to read the robots.txt directives. The sitemap.xml (511 KB, containing hundreds of URLs) is also only served to anthropic-ai. No llms.txt exists (404). The site runs on Dealer.com (DDC) platform, a common automotive dealer CMS.

Cold-Knowledge Gap

LLM knowledge about "Bobby Rahal" describes the racing driver and team owner — the 1986 Indianapolis 500 winner, co-owner of Rahal Letterman Lanigan Racing. The site itself is a multi-franchise automotive dealership group (Acura, Mercedes-Benz, Honda, BMW, Toyota, Lexus, pre-owned) based in Wexford, Pennsylvania, with locations across central PA. The cold knowledge contains zero awareness of the dealership business. An AI model asked about "Bobby Rahal" will describe a motorsports figure, not a car-selling enterprise — a complete brand identity mismatch.

Content & Schema Posture

The homepage title is "New and Used Vehicles | Bobby Rahal Automotive Group." The about page is titled "Origin Of Bobby Rahal Automotive Group." The site uses meta name="robots" content="index, follow" on key pages, but those directives are irrelevant since Googlebot never reaches the pages. No JSON-LD schema was detected in any fetched page snippet. The site has a blog (/blog/index.htm), a community involvement page (/community-involement.htm), and a Hershey Theatre page (/hershey-theatre.htm), but all are invisible to search engines.

External Signals

Web search returns zero indexed results for bobbyrahal.com, the dealership group name, or any combination of "Bobby Rahal" with automotive/dealership terms. No Wayback Machine snapshot exists. The domain has no external mentions, reviews, or press linking to it. The only discoverable digital footprint is the DNS records and the Akamai error page. The site is effectively a dark web property for every AI crawler except Claude's training pipeline.

Findings

1. Akamai WAF blocks all crawlers except anthropic-ai High

   The Akamai WAF returns HTTP 403 to every browser and nearly every AI crawler, including GPTBot, ClaudeBot, Google-Extended, PerplexityBot, OAI-SearchBot, and ChatGPT-User. Only the anthropic-ai user-agent is allowed through to the nginx origin, creating a single-bot visibility funnel.

   What to change: Reconfigure the Akamai WAF to allow common AI crawler user-agents (GPTBot, Google-Extended, ClaudeBot, etc.) and search engine bots to access the site, or remove the WAF entirely if not needed.

2. robots.txt returns 403 to all user-agents except anthropic-ai High

   The robots.txt file is only accessible via the anthropic-ai user-agent; all other user-agents receive a 403 response. This means the robots.txt directives are effectively invisible to other bots.

   What to change: Ensure robots.txt is publicly accessible without authentication or WAF filtering.

3. Sitemap.xml only served to anthropic-ai High

   The sitemap.xml (511 KB, containing hundreds of URLs) is only returned when requested with the anthropic-ai user-agent. All other crawlers receive a 403, preventing search engines from discovering the site's URL structure.

   What to change: Make sitemap.xml publicly accessible to all crawlers.

4. Zero search engine indexing for bobbyrahal.com High

   Web searches for bobbyrahal.com, the dealership group name, and related terms return zero indexed results. The site is completely absent from search engine indexes.

   What to change: Resolve the WAF blocking issue to allow search engine bots to crawl and index the site.

5. No llms.txt file present Medium

   The site does not serve an llms.txt file (returns 404), missing an opportunity to provide AI crawlers with structured guidance and content summaries.

   What to change: Create and publish an llms.txt file at the root to guide AI crawlers.

6. LLM knowledge mismatch: racing driver vs. dealership group High

   LLM knowledge about 'Bobby Rahal' describes the racing driver and team owner, with no awareness of the Bobby Rahal Automotive Group dealership business. An AI model asked about 'Bobby Rahal' will describe a motorsports figure, not a car-selling enterprise.

   What to change: Improve the site's visibility and indexing so that AI models can learn about the dealership group from the site itself.

7. No JSON-LD schema detected on key pages Medium

   No JSON-LD structured data was found in the fetched page snippets, missing opportunities to provide rich context to AI crawlers and search engines.

   What to change: Add JSON-LD structured data (e.g., LocalBusiness, AutomotiveDealer) to all key pages.

8. No Wayback Machine snapshot exists Low

   The site has no archived snapshot in the Wayback Machine, indicating a lack of historical web presence or external linking.

9. No external mentions or reviews found Medium

   Web searches for the dealership group name, reviews, and location combinations returned zero results, indicating no external digital footprint.

   What to change: Build external backlinks and encourage online reviews to improve off-site signals.

What's working

• Anthropic-ai crawler is allowed through the WAF — The anthropic-ai user-agent is permitted through the Akamai WAF and can access the full HTML content of the site, including the sitemap and all pages. This ensures Claude's training pipeline can read the site.
• Comprehensive sitemap.xml with hundreds of URLs — The sitemap.xml is 511 KB and contains hundreds of URLs, providing a complete inventory of the site's pages for crawlers that can access it.
• Robots.txt disallows sensitive paths for AI bots — The robots.txt explicitly disallows GPTBot, OAI-SearchBot, ChatGPT-User, Claude-User, and Claude-SearchBot from accessing /api/, /apis/, /pixall/, and all .js/.css/.json files, which is a good security practice.
• Key pages use meta robots index, follow — The homepage and about page include <meta name="robots" content="index, follow">, indicating the site intends to be indexed.
• Blog and community pages provide rich content — The site includes a blog, community involvement page, and Hershey Theatre page, offering valuable content that could attract AI crawlers if accessible.