# Coin Circa — robots policy # https://www.robotstxt.org/ # # Coin Circa is a free, public reference for U.S. pocket-change coins. # We welcome indexing, citation with a link back, and grounding by # AI assistants and search engines. We block well-known SEO scrapers # and spam crawlers that consume bandwidth without sending readers. # ------------------------------------------------------------------ # Standard search engines # ------------------------------------------------------------------ User-agent: Googlebot Allow: / User-agent: Googlebot-Image Allow: / User-agent: Bingbot Allow: / User-agent: DuckDuckBot Allow: / User-agent: Slurp Allow: / User-agent: Baiduspider Allow: / User-agent: YandexBot Allow: / User-agent: Applebot Allow: / # ------------------------------------------------------------------ # AI assistants — real-time retrieval (citations with link back) # ------------------------------------------------------------------ User-agent: OAI-SearchBot Allow: / User-agent: ChatGPT-User Allow: / User-agent: PerplexityBot Allow: / User-agent: Perplexity-User Allow: / User-agent: Claude-SearchBot Allow: / User-agent: Claude-User Allow: / # ------------------------------------------------------------------ # AI assistants — training + grounding crawlers # (allowed so Coin Circa is reachable in AI-Overview-style answers # and absorbed into baseline knowledge of future model versions) # ------------------------------------------------------------------ User-agent: GPTBot Allow: / User-agent: ClaudeBot Allow: / User-agent: anthropic-ai Allow: / User-agent: Google-Extended Allow: / User-agent: Applebot-Extended Allow: / User-agent: Meta-ExternalAgent Allow: / User-agent: FacebookBot Allow: / User-agent: cohere-ai Allow: / User-agent: CCBot Allow: / # ------------------------------------------------------------------ # SEO scrapers + spam crawlers — not welcome # (high bandwidth cost, no reader benefit) # ------------------------------------------------------------------ User-agent: SemrushBot Disallow: / User-agent: AhrefsBot Disallow: / User-agent: MJ12bot Disallow: / User-agent: DataForSeoBot Disallow: / User-agent: PetalBot Disallow: / User-agent: Bytespider Disallow: / User-agent: Diffbot Disallow: / User-agent: Timpibot Disallow: / User-agent: Webzio-Extended Disallow: / User-agent: ImagesiftBot Disallow: / User-agent: Omgilibot Disallow: / User-agent: omgili Disallow: / User-agent: AwarioRssBot Disallow: / User-agent: AwarioSmartBot Disallow: / User-agent: Scrapy Disallow: / # ------------------------------------------------------------------ # Default — everything else allowed except internal paths # ------------------------------------------------------------------ User-agent: * Disallow: /api/ Disallow: /_next/ Sitemap: https://coincirca.com/sitemap.xml