User-agent: Scrapy Disallow: / User-agent: ia_archiver Disallow: / User-agent: Googlebot-Image Disallow: / user-agent: googlebot allow: / User-agent: * Disallow: /wp-admin/ User-agent: HypeStat Disallow: / Sitemap: https://www.omofon.com/post-sitemap.xml Sitemap: https://www.omofon.com/post-sitemap2.xml Sitemap: https://www.omofon.com/page-sitemap.xml Sitemap: https://www.omofon.com/category-sitemap.xml