# ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ # robots.txt — Gateway Consult (gatewayconsult.com.np) # Optimized for Google Mobile-First, GEO, AEO & AI Retrieval # ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ # ── GLOBAL CRAWL RULES ─────────────────────────── User-agent: * Disallow: /search Allow: /search/label/ Disallow: /*?updated-min= Disallow: /*?updated-max= Disallow: /*&updated-min= Disallow: /*&updated-max= Allow: / # ── AI SEARCH & RETRIEVAL AGENTS (Explicit Allow) ── User-agent: PerplexityBot Allow: / User-agent: OAI-SearchBot Allow: / User-agent: ChatGPT-User Allow: / User-agent: Google-Extended Allow: / # ── AI FOUNDATION MODEL CRAWLERS ────────────────── User-agent: GPTBot Allow: / User-agent: ClaudeBot Allow: / User-agent: anthropic-ai Allow: / User-agent: meta-externalagent Allow: / User-agent: Bytespider Allow: / # ── SITEMAPS ───────────────────────────────────── Sitemap: https://www.gatewayconsult.com.np/sitemap.xml Sitemap: https://www.gatewayconsult.com.np/sitemap-pages.xml