fosterswift.com — AI Site Grade

Analysis

ClaudeBot blocked, agricultural-law gap, duplicate URL rot

The most consequential finding is that ClaudeBot receives a 429 (blocked) on the homepage while GPTBot, OAI-SearchBot, Google-Extended, PerplexityBot, and ChatGPT-User all return 200 with full content — meaning any AI assistant powered by Claude cannot retrieve the site's content live, creating an asymmetric knowledge disadvantage.

Crawler Access

The robots.txt at fosterswift.com/robots.txt contains zero AI-bot-specific directives — no User-agent: GPTBot, no User-agent: ClaudeBot, no User-agent: Google-Extended. It uses a single catch-all User-agent: * rule that disallows /*.pdf and several opaque path segments (/gazebo17, /trellis19, /scullery23, etc.). The compare_bot_access test confirmed that ClaudeBot gets a 429 (rate-limit block) while Bytespider also gets a 429. All other AI crawlers pass through cleanly. The llms.txt returns a 404 — the site has no AI-friendly content map. The sitemap.xml exists and contains 2,918 URLs, but many are duplicated with trailing comma-number suffixes (e.g., /people/Thomas-Meagher-...,41 and /people/Thomas-Meagher-...), both resolving to 200 with near-identical content but different canonical URLs, which dilutes crawl budget and signals.

Cold-Knowledge Gap

The LLM's prior knowledge describes Foster Swift as a firm "particularly known for deep expertise in agricultural and cooperative law" with a standout agricultural practice group, a well-regarded newsletter, and frequent speaking at the Michigan Farm Bureau. The actual site tells a different story. The homepage and services page list "Agri-Business" as one sub-item under Business & Tax, but there is no dedicated agricultural law practice page, no agricultural law blog, no featured agricultural content, and no mention in the firm's history or leadership narrative. The cold knowledge also cites offices in Farmington Hills (the site says Southfield, relocated in 2016) and a founding year of 1969 (the site traces origins to 1902). The model is operating on stale or confounded information.

Schema Posture

Every page examined carries the same minimal Organization schema: name, url, and sameAs (LinkedIn, Facebook, Instagram). There is no LegalService subtype, no Attorney or Lawyer schema on attorney profile pages, no FAQPage schema on the blog index (which lists 11 blog categories with Q&A-style post titles), no LocalBusiness schema with address/telephone, and no BreadcrumbList. Attorney profile pages like Thomas Meagher's contain rich structured data (honors, representative matters, bar admissions) but none of it is marked up in schema — it is all plain HTML text.

External Signals

Web searches returned zero organic results for the firm name, reviews, or Reddit mentions. The only external references found are press mentions the firm itself publishes (Crain's Detroit Business, Dbusiness, Ingham County Legal News). The site's DNS shows a CSP with unsafe-inline and unsafe-eval in script-src, and the server infrastructure (IPs on ussignalcom.net) suggests a small hosting provider rather than a major CDN. The homepage delivers only 113 words of visible text — most content is hidden behind cookie-consent overlays and navigation menus that render server-side but produce a thin first-impression for crawlers.

Findings

1. ClaudeBot receives 429 rate-limit block on homepage High

   ClaudeBot is blocked with a 429 status while GPTBot, OAI-SearchBot, Google-Extended, PerplexityBot, and ChatGPT-User all return 200 with full content. This creates an asymmetric knowledge disadvantage for Claude-powered AI assistants.

   What to change: Remove the rate-limit block for ClaudeBot or whitelist its user-agent to allow access.

2. Robots.txt lacks AI-bot-specific directives Medium

   The robots.txt file contains no User-agent lines for GPTBot, ClaudeBot, Google-Extended, or other AI crawlers. It only has a catch-all rule disallowing PDFs and obscure paths.

   What to change: Add explicit User-agent directives for major AI crawlers to control access granularly.

3. llms.txt returns 404 Medium

   The site has no llms.txt file, which is a recommended standard for providing AI-friendly content summaries.

   What to change: Create an llms.txt file with a summary of the site's content for AI crawlers.

4. Sitemap contains duplicate URLs with trailing comma-number suffixes Medium

   The sitemap includes many duplicate URLs (e.g., /people/Thomas-Meagher-...,41 and /people/Thomas-Meagher-...) that resolve to near-identical content, diluting crawl budget and causing canonical confusion.

   What to change: Remove duplicate URLs from the sitemap and implement proper canonical tags.

5. No dedicated agricultural law content despite LLM knowledge High

   LLM prior knowledge describes Foster Swift as having deep expertise in agricultural and cooperative law, but the site has no dedicated agricultural law page, blog, or featured content. Agri-Business is only a sub-item under Business & Tax.

   What to change: Create a dedicated agricultural law practice page with detailed content and schema markup.

6. LLM knowledge contains outdated firm details Medium

   The LLM's prior knowledge cites an office in Farmington Hills (relocated to Southfield in 2016) and a founding year of 1969 (site traces origins to 1902), indicating stale or confounded information.

   What to change: Ensure the site prominently displays current office locations and founding history to correct LLM knowledge.

7. No LegalService or Attorney schema markup High

   Pages use only minimal Organization schema. Attorney profile pages lack Attorney or Lawyer schema, and the site has no LocalBusiness, FAQPage, or BreadcrumbList schema.

   What to change: Add LegalService, Attorney, LocalBusiness, and BreadcrumbList schema markup to relevant pages.

8. Homepage delivers only 113 words of visible text Medium

   The homepage has very little visible text, with most content hidden behind cookie-consent overlays and navigation menus, resulting in a thin first impression for crawlers.

   What to change: Increase visible text content on the homepage and ensure key information is accessible without interaction.

9. No organic search results for firm name or reviews Medium

   Web searches for the firm name, reviews, and Reddit mentions returned zero results, indicating weak external signals and low online visibility.

   What to change: Encourage client reviews on third-party platforms and improve SEO to increase organic presence.

10. Bytespider also receives 429 block Low

    Bytespider, a crawler associated with ByteDance, is blocked with a 429 status, similar to ClaudeBot.

    What to change: Review and adjust rate-limiting rules to allow Bytespider if desired.

What's working

• Sitemap.xml exists with 2,918 URLs — The sitemap is present and contains a large number of URLs, helping crawlers discover content.
• Most AI crawlers return 200 on homepage — GPTBot, OAI-SearchBot, Google-Extended, PerplexityBot, and ChatGPT-User all successfully retrieve the homepage with full content.
• Organization schema with sameAs links present on all pages — Every page includes basic Organization schema with name, URL, and social media sameAs links, providing a baseline identity signal.
• Attorney profile pages contain rich structured data in plain HTML — Pages like Thomas Meagher's include honors, representative matters, and bar admissions in readable HTML, which can be leveraged for schema markup.
• Blog index lists 11 categories with Q&A-style post titles — The blog index provides a clear structure of categories and descriptive post titles, aiding content discovery.
• Services page lists practice groups with descriptions — The services page provides an overview of practice groups, helping crawlers understand the firm's offerings.