# EZer - robots.txt # https://www.ezerapp.com # Last updated: 2025-12-29 # =========================================== # Default: Allow all crawlers # =========================================== User-agent: * Allow: / Disallow: /backups/ Disallow: /*.md$ # =========================================== # Traditional Search Engines # =========================================== # Google User-agent: Googlebot Allow: / User-agent: Googlebot-Image Allow: / User-agent: Googlebot-Mobile Allow: / # Bing User-agent: Bingbot Allow: / User-agent: msnbot Allow: / # Yahoo User-agent: Slurp Allow: / # DuckDuckGo User-agent: DuckDuckBot Allow: / # Yandex User-agent: Yandex Allow: / # Baidu User-agent: Baiduspider Allow: / # =========================================== # AI Search Engines & LLM Crawlers # =========================================== # OpenAI / ChatGPT User-agent: GPTBot Allow: / User-agent: ChatGPT-User Allow: / # Anthropic / Claude User-agent: Claude-Web Allow: / User-agent: ClaudeBot Allow: / User-agent: anthropic-ai Allow: / # Google AI (Bard/Gemini) User-agent: Google-Extended Allow: / # Perplexity AI User-agent: PerplexityBot Allow: / # Microsoft Copilot User-agent: Copilot Allow: / # Meta AI User-agent: FacebookBot Allow: / User-agent: Meta-ExternalAgent Allow: / # Cohere User-agent: cohere-ai Allow: / # Apple (Siri/Spotlight) User-agent: Applebot Allow: / User-agent: Applebot-Extended Allow: / # Amazon Alexa User-agent: Amazonbot Allow: / # You.com AI User-agent: YouBot Allow: / # Common Crawl (used by many AI companies) User-agent: CCBot Allow: / # =========================================== # Social Media Crawlers (for link previews) # =========================================== User-agent: Twitterbot Allow: / User-agent: LinkedInBot Allow: / User-agent: Pinterest Allow: / User-agent: Slackbot Allow: / User-agent: WhatsApp Allow: / User-agent: TelegramBot Allow: / # =========================================== # Sitemap Location # =========================================== Sitemap: https://www.ezerapp.com/sitemap.xml # =========================================== # Host directive (for search engines that support it) # =========================================== Host: https://www.ezerapp.com