User-agent: * # Public marketing site (erfbrand). Logged-in portal lives on app subdomain; use that # host's /robots.txt (Disallow: /). You cannot disallow another host's URLs from here. # # Paths that belonged to the portal before the split may 301 to the portal origin; see # X-Robots-Tag on those redirects. # Transactional / non-indexable paths on THIS host only Disallow: /offert Disallow: /offert/ Disallow: /offert/csrf-token Disallow: /calculate-pricing Disallow: /generate-pricing-pdf # Internal design previews (redesign work in progress) Disallow: /_preview Disallow: /favicon.ico Allow: / User-agent: GPTBot Allow: / User-agent: OAI-SearchBot Allow: / User-agent: ChatGPT-User Allow: / User-agent: ClaudeBot Allow: / User-agent: Claude-User Allow: / User-agent: PerplexityBot Allow: / User-agent: Perplexity-User Allow: / User-agent: Google-Extended Allow: / User-agent: CCBot Allow: / User-agent: Applebot-Extended Allow: / User-agent: Bingbot Allow: / User-agent: Googlebot Allow: / # Sitemap index and specialized sitemaps (all on this origin) Sitemap: https://erf.se/sitemap.xml Sitemap: https://erf.se/main.sitemap.xml Sitemap: https://erf.se/hemsida.sitemap.xml Sitemap: https://erf.se/ai/faq.sitemap.xml Sitemap: https://erf.se/ai/howto.sitemap.xml Sitemap: https://erf.se/ai/service.sitemap.xml Sitemap: https://erf.se/ai/blog.sitemap.xml Sitemap: https://erf.se/ai/dataset.sitemap.xml # AI crawler information # See https://erf.se/ai/ai.txt for AI-specific crawling guidelines