User-agent: * Allow: / # Sitemap (index -> conflicts, pages, briefings) Sitemap: https://armedconflicts.org/sitemap.xml # RSS feed: https://armedconflicts.org/feed.xml # LLM-readable summaries: # https://armedconflicts.org/llms.txt # https://armedconflicts.org/llms-full.txt # AI usage policy: https://armedconflicts.org/ai.txt # LLM info (start here): https://armedconflicts.org/llm-info # No Crawl-delay: it is non-standard (ignored by Googlebot/Bingbot), this is # a static site with no server load to protect, and the whole point of this # workstream is to remove friction for the AI retrieval crawlers driving # citation growth (OAI-SearchBot, ChatGPT-User, Google-Extended, # Applebot-Extended, CCBot, Amazonbot, Meta-ExternalAgent, Bytespider, etc.) # — none of which have their own group below and would otherwise inherit a # delay from User-agent: * for no benefit. # AI training crawlers — allowed but please respect the source attribution User-agent: GPTBot Allow: / User-agent: ClaudeBot Allow: / User-agent: anthropic-ai Allow: / User-agent: PerplexityBot Allow: / # Search crawlers: everything except the raw data downloads. # # /data/ holds 97 .json/.csv/.geojson files, linked from all 62 record and map # pages as "Download data". They are payloads, not pages -- a .geojson cannot # carry a