# See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file # User-agent: * Disallow: /searchv1 Disallow: /searchv2 Disallow: /a/ Disallow: /invite/ Disallow: /api/products/search User-agent: Baiduspider Disallow: /products/ Disallow: /tags/ Disallow: /nodes/ User-agent: YandexBot Disallow: / User-agent: bingbot Disallow: /tags/ Disallow: /api/ User-agent: SemrushBot Disallow: / User-agent: PetalBot Disallow: / User-agent: AhrefsBot Disallow: / User-agent: DotBot Disallow: / User-agent: Neevabot Disallow: / User-agent: SeekportBot Disallow: / User-agent: fr_bot Disallow: / Sitemap: https://www.desertcart.com.kw/sitemaps/kw/sitemap-index.xml.gz