berglundcars.com — AI Site Grade

Analysis

Akamai Blocks Every AI Crawler Except Anthropic

Berglund Automotive has a two-tier access system where the Akamai WAF returns HTTP 403 to every browser and every AI crawler except anthropic-ai, which gets a full HTTP 200 with 763KB of HTML served from nginx behind the CDN. This means Claude can read the site cold, but GPTBot, Google-Extended, PerplexityBot, OAI-SearchBot, and ChatGPT-User all hit the same Akamai denial page as a human browser. The non-www domain (berglundcars.com) does not resolve at all — connection refused on all user-agents.

Crawler Access

The robots.txt (accessible only to anthropic-ai) explicitly disallows GPTBot, CCBot, OAI-SearchBot, ChatGPT-User, Claude-User, Claude-SearchBot, and PerplexityBot from accessing /api/, /apis/, /pixall/, and static assets — but does not block them from the homepage or content pages. The Akamai WAF overrides this: those bots never reach the nginx server to read the robots.txt at all. No llms.txt exists as a standard file; the URL returns a 726KB plain-text document that appears to be an auto-generated inventory dump rather than a curated AI guide. No Sitemap: directive appears in the robots.txt.

Cold-Knowledge Gap

LLM knowledge describes Berglund as a "family-owned auto dealership group based in Lynchburg, Virginia, operating since 1975" selling Ford, Chevrolet, Toyota, Honda, Nissan, Hyundai, and Chrysler/Dodge/Jeep/Ram. The actual site reveals a far larger franchise footprint: the HTML class list and page titles include INFINITI, Kia, Volvo, Buick, Mazda, BMW, Land Rover, Cadillac, Mercedes-Benz, Jaguar, GMC, and FIAT — 19 brands across locations in Roanoke, Bedford, Lynchburg, and Salem. The cold knowledge misses roughly half the brand portfolio and omits the luxury/import tier entirely.

Schema Posture

The homepage and subpages contain zero JSON-LD schema of any type. The ddc-site class and dealer.com CDN references indicate the site runs on Dealer.com's platform, which typically injects AutoDealer and Organization schema — but none was present in the fetched HTML. No LocalBusiness, AutoDealer, Product, or Vehicle structured data was detected. The meta name="robots" content="index, follow" tag is present on content pages, but Google cannot reach those pages to read it.

External Signals

The site has zero indexed pages in search engines. A site:berglundcars.com query returns no results. No external news, reviews, or Reddit threads were found in search. The Wayback Machine shows a snapshot from February 2026, indicating the site has been accessible historically but is now effectively invisible to the open web. The lastmod dates in the sitemap (May 30, 2026) suggest the content is being maintained, but only anthropic-ai can verify this.

Findings

1. Akamai WAF blocks all AI crawlers except Anthropic High

   The Akamai WAF returns HTTP 403 to every browser and AI crawler except anthropic-ai, which receives HTTP 200. GPTBot, Google-Extended, PerplexityBot, OAI-SearchBot, and ChatGPT-User all hit the same denial page.

   What to change: Reconfigure Akamai WAF to allow GPTBot, Google-Extended, and other major AI crawlers to access the site, or serve them static HTML versions.

2. Non-www domain does not resolve High

   The non-www domain (berglundcars.com) returns connection refused for all user-agents, including browsers and AI crawlers.

   What to change: Configure the non-www domain to redirect to the www version or serve the same content.

3. Zero pages indexed in search engines High

   A site:berglundcars.com search returns no results, and no external news, reviews, or social mentions were found. The site is effectively invisible to the open web.

   What to change: Allow Googlebot and other search crawlers through the WAF, and ensure content is indexable.

4. No JSON-LD structured data on any page High

   The homepage and subpages contain zero JSON-LD schema of any type, including AutoDealer, LocalBusiness, or Vehicle schema. This limits AI understanding of the business and inventory.

   What to change: Add JSON-LD structured data for AutoDealer, Organization, and Vehicle listings on relevant pages.

5. LLM cold knowledge misses half the brand portfolio High

   LLM knowledge lists only 7 brands (Ford, Chevrolet, Toyota, etc.), but the site actually sells 19 brands including INFINITI, Kia, Volvo, BMW, Mercedes-Benz, and others. The luxury/import tier is entirely absent from AI knowledge.

   What to change: Publish a comprehensive llms.txt or knowledge graph that lists all brands and locations to correct AI knowledge.

6. Robots.txt accessible only to Anthropic Medium

   The robots.txt file returns HTTP 403 to browsers and most AI crawlers; only anthropic-ai can read it. It disallows several bots from API and static paths, but the WAF blocks them before they reach it.

   What to change: Make robots.txt publicly accessible and ensure it does not block important AI crawlers from content.

7. llms.txt is an auto-generated inventory dump Medium

   The llms.txt URL returns a 726KB plain-text document that appears to be an auto-generated inventory dump rather than a curated AI guide. It is not a standard llms.txt file.

   What to change: Replace the inventory dump with a proper llms.txt file that summarizes the business, brands, locations, and key pages.

8. No Sitemap directive in robots.txt Low

   The robots.txt file does not contain a Sitemap: directive, making it harder for crawlers to discover the sitemap.

   What to change: Add a Sitemap: directive pointing to the sitemap URL.

9. Sitemap returns 403 to browsers Medium

   The sitemap.xml returns HTTP 403 when accessed by a browser, though it is accessible to anthropic-ai. This prevents search engines from discovering the sitemap.

   What to change: Allow public access to sitemap.xml so search engines can crawl it.

10. No external signals or backlinks found Medium

    No external news, reviews, or social mentions were found for the domain. The site has no backlinks or citations from other sources.

    What to change: Build external signals through PR, listings, and social media to improve AI visibility.

What's working

• Anthropic AI crawler has full access to the site — The anthropic-ai crawler receives HTTP 200 and can read all pages, including the homepage, sitemap, and subpages. This allows Claude to index the site's content.
• Large sitemap with recent lastmod dates — The sitemap.xml is 401KB and contains many URLs with lastmod dates as recent as May 30, 2026, indicating active content maintenance.
• Content pages accessible to Anthropic with rich HTML — Subpages like the visit-us page and purchase-your-next-vehicle page return 200 to anthropic-ai with 200-700KB of HTML, containing detailed content about locations and inventory.
• Robots.txt allows content paths for listed bots — The robots.txt disallows only API and static asset paths for GPTBot, CCBot, etc., but does not block them from the homepage or content pages. The WAF blocks them anyway, but the intent is permissive.
• Meta robots tag allows indexing on content pages — Content pages include <meta name="robots" content="index, follow">, indicating the site intends for pages to be indexed.