berkshirehathawayautomotive.com — AI Site Grade

Analysis

Anthropic-ai Is the Sole AI Crawler Allowed Through a 403 Wall

The entire berkshirehathawayautomotive.com domain is locked behind an Akamai WAF that returns 403 for every browser and every major AI crawler — except anthropic-ai, which receives a full 200 response with 348KB of HTML served from an nginx origin. This creates a bizarre asymmetry: Claude can read the site, but GPTBot, Google-Extended, PerplexityBot, and OAI-SearchBot all get blocked identically to a human browser.

Crawler Access

No robots.txt or llms.txt exists — both return 403 from Akamai. The sitemap.xml is also blocked. The compare_bot_access test on the homepage showed 10 of 11 tested user-agents blocked with the same Akamai 403 page. Only anthropic-ai passed through to the nginx backend. The site runs on AWS (EC2 IP 34.231.3.11) behind Akamai CDN, with a DDC (Dealer.com) platform stack visible in the HTML class attributes. The homepage title reads "Berkshire Hathaway Automotive | Automotive Dealership Group."

Cold-Knowledge Gap

The LLM model knows the brand as a 80+ dealership group formed from the 2015 Van Tuyl Group acquisition, serving the Sun Belt with major OEM brands. The actual site, however, is a single-page JS shell that even the anthropic-ai bot can only partially render — the 348KB response is dominated by bundled JavaScript and CSS assets (Dealer.com's v9-global-0011-v2 framework). No dealership listings, no location data, no inventory, no About page, no contact information is served as static HTML. The /dealerships, /about, and /locations paths all return 404. The site has zero discoverable URLs in any sitemap.

External Signals

Search engines have zero indexed pages from this domain. DuckDuckGo returned zero results for any query containing the brand name, the domain, or the Van Tuyl acquisition. The site has no external backlink footprint visible to standard search. The only way the brand is known to AI systems is through pre-training data about Berkshire Hathaway's 2015 acquisition — not from anything the site itself publishes.

Schema Posture

The homepage contains zero JSON-LD schema of any type. No Organization, AutoDealer, LocalBusiness, or BreadcrumbList markup exists. The page has no meta description, no Open Graph tags, and no canonical URL. The heading structure is a single H1: Access Denied for blocked users, while the anthropic-ai bot sees a title tag but no visible H1 in the static HTML — the actual content is rendered client-side by JavaScript that the bot fetch cannot execute.

Findings

1. Akamai WAF blocks all AI crawlers except anthropic-ai High

   The site's Akamai WAF returns 403 for every major AI crawler (GPTBot, Google-Extended, PerplexityBot, OAI-SearchBot) and all browsers, but allows anthropic-ai through with a 200 response. This creates a single-crawler monopoly and prevents other AI systems from accessing the site.

   What to change: Remove the Akamai WAF block for all legitimate AI crawlers, or implement a robots.txt that allows them while still blocking malicious traffic.

2. No robots.txt or llms.txt available High

   Both robots.txt and llms.txt return 403 from Akamai, meaning crawlers cannot discover allowed paths or AI-specific instructions. This leaves crawlers with no guidance and contributes to the site's invisibility.

   What to change: Create and serve a robots.txt that allows AI crawlers, and optionally an llms.txt to guide AI systems.

3. Sitemap.xml blocked by Akamai High

   The sitemap.xml returns 403, preventing crawlers from discovering any URLs. Combined with zero discoverable URLs, the site has no crawlable entry points.

   What to change: Serve a valid sitemap.xml listing all public pages, and ensure it is accessible to crawlers.

4. Zero indexed pages in search engines High

   Multiple web searches for the domain and brand name returned zero results. The site has no indexed pages, no backlink footprint, and is effectively invisible to search engines and AI systems that rely on search indices.

   What to change: Ensure the site is crawlable and indexable by search engines, and build external backlinks to establish authority.

5. Homepage is a JavaScript shell with no static content High

   The homepage serves 348KB of HTML dominated by bundled JavaScript and CSS, with no visible H1 or meaningful text in the static HTML. The actual content is rendered client-side, which AI crawlers that do not execute JavaScript cannot see. Key pages like /dealerships, /about, and /locations return 404.

   What to change: Implement server-side rendering or static HTML for key content such as dealership listings, locations, and contact information, so that AI crawlers can extract meaningful data without executing JavaScript.

6. No JSON-LD schema on the homepage High

   The homepage contains zero JSON-LD structured data of any type, including Organization, AutoDealer, LocalBusiness, or BreadcrumbList. This prevents AI systems from understanding the site's entity and offerings.

   What to change: Add JSON-LD schema markup for Organization, AutoDealer, and LocalBusiness to the homepage and all relevant pages.

7. Missing meta description and Open Graph tags Medium

   The homepage has no meta description and no Open Graph tags, which limits how the site appears in search results and social previews, and reduces clarity for AI systems.

   What to change: Add a descriptive meta description and Open Graph tags (title, description, image) to the homepage.

8. No canonical URL specified Low

   The homepage lacks a canonical URL tag, which can lead to duplicate content issues and confuse crawlers about the preferred URL.

   What to change: Add a canonical URL tag pointing to the preferred version of the homepage.

9. No external backlinks detected Medium

   Web searches found no external backlinks to the domain, contributing to its low authority and lack of visibility in search and AI systems.

   What to change: Develop a backlink strategy to build external links from reputable automotive and business directories.

10. Brand known to AI only through pre-training data Medium

    The LLM has knowledge of Berkshire Hathaway Automotive from pre-training data (2015 Van Tuyl acquisition), but the site itself provides no current information. This creates a gap between what AI knows and what the site offers.

    What to change: Publish up-to-date content about dealership locations, inventory, and services to ensure AI systems have accurate current information.

What's working

• Anthropic-ai crawler is allowed through WAF — The anthropic-ai crawler receives a 200 response with full HTML, enabling Claude to access the site's content. This is a positive step for AI visibility, albeit limited to one crawler.
• Homepage has a descriptive title tag — The homepage title 'Berkshire Hathaway Automotive | Automotive Dealership Group' clearly describes the business, which helps crawlers understand the site's purpose.
• Site hosted on AWS with Akamai CDN — The site uses AWS (EC2) for hosting and Akamai for CDN, providing reliable infrastructure and fast content delivery globally.