takingshape.com — AI Site Grade

Analysis

PerimeterX Blocks All AI Crawlers From Every Page — Site Is Invisible to LLMs

The entire takingshape.com domain is locked behind PerimeterX bot detection that returns a "Please verify you are a human" page to every crawler tested — GPTBot, ClaudeBot, PerplexityBot, Google-Extended, ChatGPT-User, and even a standard browser user-agent all receive the same 81-word denial page. The site is effectively invisible to AI engines.

Crawler Access

The robots.txt contains no AI-bot-specific rules — no mention of GPTBot, ClaudeBot, Google-Extended, or any other AI crawler. The User-agent: * block disallows */on/demandware.store/*, which is the exact path pattern every request is redirected through (the site runs on Salesforce Commerce Cloud / Demandware). This means even if a bot bypasses PerimeterX, the robots.txt itself blocks the store endpoint that serves all content. The llms.txt returns a 410 Gone status. Sitemaps are also blocked by PerimeterX — the sitemap_index.xml redirects to the same denial page.

Cold-Knowledge Gap

The LLM model knows Taking Shape as "a plus-size women's fashion retailer, specializing in sizes 14–30, founded in 2006 by two sisters, with physical stores in Australia and New Zealand." This prior knowledge is richer than anything the site itself delivers to crawlers. The site's actual homepage, product pages, category pages, and sitemaps all return zero visible content to automated agents. The model's knowledge about the brand's "Curve" denim line, shapewear-integrated dresses, and "Petite Plus" range comes from off-domain sources — not from the site itself.

Schema Posture

The only structured data found on the domain lives in the llms.txt 410 error page's HTML: a single Organization schema block containing the legal name, logo URL, customer service phone number (1800 117 834), and links to Facebook and Instagram. This schema is served only on a dead-end error page that no crawler should reach. No Product, BreadcrumbList, FAQPage, or WebSite schema was detected anywhere.

External Signals

The Wayback Machine holds only a single snapshot from September 2013 — no recent archives exist. The domain resolves to Cloudflare (104.18.64.228/65.228) with a CNAME to takingshape.com.cdn.cloudflare.net. The PerimeterX protection is so aggressive that even a standard browser user-agent with no special headers is blocked. The site operates three locale variants (AU, NZ, UK) but all redirect through the same PerimeterX wall.

Findings

1. PerimeterX blocks all AI crawlers from every page High

   Every tested crawler (GPTBot, ClaudeBot, PerplexityBot, Google-Extended, ChatGPT-User, and standard browser) receives an 81-word 'Please verify you are a human' page. The site is invisible to AI engines.

   What to change: Configure PerimeterX to allow AI crawlers (GPTBot, ClaudeBot, Google-Extended, etc.) by whitelisting their user-agent strings or IP ranges.

2. Robots.txt disallows the store endpoint that serves all content High

   The User-agent: * rule disallows */on/demandware.store/*, which is the path pattern every request is redirected through (Salesforce Commerce Cloud). Even if PerimeterX is bypassed, robots.txt blocks the store endpoint.

   What to change: Remove or relax the disallow rule for */on/demandware.store/* for AI crawlers, or serve content from a separate path that is not blocked.

3. No llms.txt file available Medium

   The llms.txt endpoint returns a 410 Gone status, providing no guidance to AI crawlers about which pages to index or how to access content.

   What to change: Create an llms.txt file that lists important pages (homepage, category pages, product pages) and provides hints for AI crawlers.

4. Sitemaps blocked by PerimeterX High

   The sitemap index and individual sitemaps redirect to the same human-verification page, preventing crawlers from discovering the site's URL structure.

   What to change: Ensure sitemaps are accessible to AI crawlers by whitelisting their user-agents in PerimeterX.

5. No structured data on content pages High

   The only structured data found is an Organization schema on the 410 error page. No Product, BreadcrumbList, FAQPage, or WebSite schema exists on any accessible page.

   What to change: Add Product schema to product pages, BreadcrumbList to category pages, and WebSite schema to the homepage. Ensure these are served on pages accessible to crawlers.

6. LLM prior knowledge richer than site content delivered to crawlers Medium

   The LLM knows Taking Shape as a plus-size women's fashion retailer with specific product lines (Curve denim, shapewear-integrated dresses, Petite Plus) and store locations, but this information comes from off-domain sources, not from the site itself.

   What to change: Ensure that product details, brand history, and store information are present on the site and accessible to crawlers so that AI models can cite the site as a source.

7. No recent Wayback Machine snapshots Medium

   The Wayback Machine holds only a single snapshot from September 2013, indicating that the site has been inaccessible to archiving bots for over a decade.

   What to change: Allow archiving bots (e.g., archive.org) to access the site to build a historical record and improve external signals.

8. No AI-bot-specific rules in robots.txt Low

   The robots.txt does not mention GPTBot, ClaudeBot, Google-Extended, or any other AI crawler, missing an opportunity to explicitly allow or disallow them.

   What to change: Add explicit rules for AI crawlers (e.g., allow GPTBot, disallow others) to control access.

9. Homepage returns zero visible content to crawlers High

   The homepage, when fetched by any automated agent, returns only an 81-word denial page with no product, category, or brand information.

   What to change: Serve static HTML content to crawlers that includes brand description, product categories, and key pages.

10. All locale variants blocked by PerimeterX High

    The AU, NZ, and UK locale variants all redirect through the same PerimeterX wall, returning the same denial page.

    What to change: Apply the same PerimeterX whitelisting for AI crawlers across all locale variants.

11. No web search results for the domain Medium

    Multiple web searches for the brand and domain returned zero results, indicating poor search engine indexing likely due to the bot blocking.

    What to change: Resolve crawler access issues to allow search engines to index the site, improving organic visibility.

12. Organization schema only on error page Low

    The only structured data found is an Organization schema on the 410 error page, which is not a page that should be indexed. It contains legal name, logo, phone, and social links.

    What to change: Move the Organization schema to the homepage and add other relevant schemas (WebSite, Product) to appropriate pages.

What's working

• Organization schema present on error page — An Organization schema block exists on the llms.txt error page, containing legal name, logo URL, customer service phone, and social links. This provides basic brand information if accessed.
• Cloudflare protection in place — The site uses Cloudflare for CDN and security, which can help mitigate DDoS and other attacks, though it currently blocks all crawlers.
• Multiple locale variants available — The site supports AU, NZ, and UK locales, indicating international reach, though all are currently blocked.
• Brand known to LLM with rich prior knowledge — The LLM has prior knowledge of Taking Shape as a plus-size women's fashion retailer with specific product lines and store locations, indicating brand recognition.
• Robots.txt file exists and is accessible — The robots.txt file is accessible and returns a 200 status, providing basic crawl instructions, though it currently blocks the store endpoint.
• Sitemap index file exists — A sitemap index file is present at /AU/sitemap_index.xml, indicating an attempt to organize URLs, though it is blocked by PerimeterX.
• DNS resolves correctly — The domain resolves to Cloudflare IPs with proper A and NS records, indicating correct DNS configuration.
• Logo SVG accessible — The logo SVG at /on/demandware.static/.../ts-logo.svg is accessible and returns a 200 status, providing a visual asset.