webgains.com — AI Site Grade

Analysis

Webgains has no AI-bot rules in robots.txt, no llms.txt, and its blog listing page serves only 8 words of visible text to crawlers — yet every major AI bot gets a full 200 with real content on the homepage.

Crawler Access

All 11 tested AI crawlers (GPTBot, ClaudeBot, PerplexityBot, Google-Extended, OAI-SearchBot, ChatGPT-User, Bytespider, Applebot-Extended, anthropic-ai, Perplexity-User) receive a 200 status with identical byte-size content as a browser on the homepage. The site runs on Cloudflare (PHP 7.4.15 backend) with no UA-based blocking. However, the robots.txt at webgains.com/robots.txt contains zero AI-bot-specific directives — only a catch-all * rule that disallows /components/, /merchants/, /affiliates/, /advertiser/, /publisher/, /agency/, /docs/, and admin paths. No llms.txt exists (404). The sitemap at /public/sitemap.xml is a 1.5 MB dynamically generated file with hundreds of URLs, but the list_known_urls tool returned zero sitemap URLs — the sitemap is not being properly surfaced to discovery tools.

Content & Schema Posture

The homepage carries no FAQ schema, no Product schema, no HowTo schema — only WebSite, Organization, BreadcrumbList, and WebPage types. The Advertisers page is the only page with a FAQPage schema (4 questions about affiliate programmes). The blog listing page at /public/en/blog/ returns only 8 words of visible text to a plain GET — the actual blog posts are loaded via JavaScript, creating a thin-content problem for crawlers that do not execute JS. Individual blog posts (e.g., the agencies article) are rich at ~957 words with proper Article schema including author, publisher, and image. The homepage content is strong (~1,076 words) with clear H1/H2 structure and a "Service-First" positioning repeated across pages.

Cold-Knowledge Gap

The LLM knows Webgains as a 2005-founded European affiliate network, part of AdLINK Group, with a reputation for slower payments and tracking discrepancies. The site itself says it was founded in 2005, part of ad pepper media group since 2006, and positions itself as "the world's first B Corp certified affiliate network" (certified July 2023, score 87.8). The cold knowledge misses the entire B Corp differentiator — the single most distinctive claim the site makes. The cold knowledge also mentions "SmartLinks" and "Global Partner Network" as notable products, but the site's current product language focuses on "Resilient Tracking Suite," "Customer Journey Report," "Smart Commissions," and "Agency Certification" — a significant terminology gap. The cold knowledge references "AdLINK Group" while the site footer says "part of ad pepper media International N.V."

External Signals

External search results for Webgains reviews, Reddit mentions, and Trustpilot are effectively absent — no organic review aggregations surfaced. The site relies entirely on its own testimonials (Mark Hammond, Jake Wild, Alex Dalby) and case studies (Db Journey). The Knowledge Hub at knowledgehub.webgains.com (Confluence-based) contains substantial documentation but is not linked from the sitemap and has no schema markup. The old blog subdomain at blog.webgains.com timed out on fetch — potentially a dead or redirected subdomain that still appears in navigation links.

Surprising Findings

The blog listing page is a JS shell (8 words visible) while individual blog posts are server-rendered with full content — an inconsistent rendering strategy that likely suppresses crawler discovery of the blog archive. The publishers page also returns only 33 words of visible text, suggesting JS-dependent content loading on multiple key landing pages. The sitemap is massive (1.5 MB, dynamically generated by All in One SEO) but not discoverable via standard sitemap discovery tools — the list_known_urls call returned zero sitemap URLs despite the file existing. The site uses x-mod-pagespeed (Google PageSpeed module) and PHP/7.4.15 — a PHP version that reached end-of-life in November 2022.

Findings

1. Blog listing page renders as a JavaScript shell with only 8 words of visible text High

   The blog listing page at /public/en/blog/ returns only 8 words of visible text to a plain HTTP GET. Actual blog posts are loaded via JavaScript, making the archive effectively invisible to crawlers that do not execute JS.

   What to change: Implement server-side rendering or static generation for the blog listing page so that all post titles and excerpts are present in the initial HTML.

2. Robots.txt contains no AI-bot-specific directives Medium

   The robots.txt file at webgains.com/robots.txt has only a catch-all '*' rule disallowing several paths, but no explicit rules for GPTBot, ClaudeBot, or other AI crawlers. This leaves AI bot access uncontrolled and misses an opportunity to guide crawlers to valuable content.

   What to change: Add explicit allow/disallow rules for major AI crawlers (GPTBot, ClaudeBot, etc.) and consider allowing access to key content while blocking low-value paths.

3. No llms.txt file found Medium

   The site does not provide an llms.txt file (404), which is a recommended standard for helping LLMs discover and prioritize content. This limits the site's ability to guide AI crawlers to its most important pages.

   What to change: Create an llms.txt file listing key pages (homepage, about, advertisers, publishers, blog) to improve LLM content discovery.

4. B Corp certification is absent from LLM cold knowledge High

   The site prominently claims to be 'the world's first B Corp certified affiliate network' (certified July 2023, score 87.8), but this differentiator is entirely missing from LLM knowledge. The cold knowledge instead references outdated product names and corporate structure.

   What to change: Ensure the B Corp page is well-structured with schema markup and linked from the homepage. Consider submitting the site to LLM knowledge bases or using structured data to reinforce the certification.

5. Sitemap exists but is not surfaced to discovery tools Medium

   A 1.5 MB dynamically generated sitemap exists at /public/sitemap.xml, but the list_known_urls tool returned zero sitemap URLs, indicating the sitemap is not properly linked or referenced in robots.txt for crawler discovery.

   What to change: Reference the sitemap URL in robots.txt and ensure it is listed in the sitemap index if applicable. Consider breaking it into smaller sitemaps for better crawl efficiency.

6. Publishers page returns only 33 words of visible text High

   The publishers landing page at /public/en/publishers/ returns only 33 words of visible text, suggesting heavy reliance on JavaScript for content. This limits crawler understanding of the page's value proposition.

   What to change: Add server-rendered content to the publishers page, including key benefits and calls to action, to ensure crawlers can index the full message.

7. Blog subdomain times out on fetch Medium

   The blog subdomain at blog.webgains.com timed out during fetch, suggesting it may be dead or misconfigured. If still linked from the site, this creates a broken user and crawler experience.

   What to change: Either restore the blog subdomain with working content or redirect it to the main site's blog section. Remove any navigation links pointing to the dead subdomain.

8. Site runs on end-of-life PHP 7.4.15 Low

   The server header reveals PHP/7.4.15, which reached end-of-life in November 2022. This poses security and performance risks and may limit the ability to implement modern rendering techniques.

   What to change: Upgrade PHP to a supported version (8.0 or later) to improve security and enable better server-side rendering capabilities.

9. Knowledge Hub is not linked from sitemap and lacks schema Medium

   The Knowledge Hub at knowledgehub.webgains.com contains substantial documentation but is not included in the sitemap and has no schema markup, reducing its discoverability and AI understanding.

   What to change: Add the Knowledge Hub to the sitemap and implement appropriate schema (e.g., Article or TechArticle) on its pages.

10. Homepage lacks FAQ schema despite common questions Low

    The homepage has no FAQ schema, even though the site likely addresses common questions about affiliate marketing. Only the Advertisers page includes FAQPage schema.

    What to change: Add FAQ schema to the homepage for frequently asked questions to improve visibility in AI-generated answers.

11. No external reviews or Reddit mentions surfaced in search Medium

    Web searches for Webgains reviews, Reddit mentions, and Trustpilot returned zero results, indicating a lack of organic external signals that could boost credibility and AI visibility.

    What to change: Encourage satisfied clients to leave reviews on platforms like Trustpilot and G2. Engage in relevant online communities to build organic mentions.

12. LLM cold knowledge uses outdated product names Medium

    The LLM's knowledge references 'SmartLinks' and 'Global Partner Network,' but the site currently promotes 'Resilient Tracking Suite,' 'Customer Journey Report,' and 'Smart Commissions.' This mismatch can confuse AI-generated summaries.

    What to change: Update the site's structured data and public content to consistently use current product names. Consider publishing a product update page or press release to signal changes to LLMs.

What's working

• All 11 tested AI crawlers receive full content on homepage — Every major AI bot (GPTBot, ClaudeBot, etc.) gets a 200 status with the same byte-size content as a browser on the homepage, indicating no UA-based blocking.
• Individual blog posts are server-rendered with Article schema — Blog posts like the agencies article have ~957 words of visible text and proper Article schema including author, publisher, and image, making them well-indexable.
• Advertisers page includes FAQPage schema with 4 questions — The Advertisers page is the only page with FAQPage schema, containing 4 questions about affiliate programmes, which can help AI generate direct answers.
• Homepage has strong content with clear structure — The homepage contains ~1,076 words with clear H1/H2 headings and a consistent 'Service-First' positioning, providing a solid foundation for AI understanding.
• B Corp page provides detailed certification information — The B Corp page at /public/en/bcorp/ contains 697 words explaining the certification, score, and impact, which is a strong asset once discovered by LLMs.
• Knowledge Hub provides extensive documentation — The Knowledge Hub at knowledgehub.webgains.com contains substantial documentation (406 words on the welcome page) that can serve as a rich source for AI training if properly surfaced.
• Site uses Cloudflare for performance and security — Cloudflare provides CDN and security benefits, helping ensure fast load times and protection against attacks, which indirectly supports crawler access.
• Consistent 'Service-First' messaging across pages — The site consistently uses 'THE SERVICE FIRST AFFILIATE NETWORK' tagline and messaging across multiple pages, reinforcing brand identity for AI understanding.