User-agent: Baiduspider Disallow: User-agent: 360Spider Disallow: User-agent: Sogou web spider Disallow: User-agent: Googlebot Disallow: User-agent: Bingbot Disallow: User-agent: googlebot-image Disallow: User-agent: googlebot-mobile Disallow: #Disallow: Sitemap: http://www.gwdang.com/sitemap_index.xml.gz # Sitemap: http://www.pingluntuan.com/sitemap_static.xml Crawl-delay: 30 Visit-time: 0100-1000 Request-rate: 10/1m 0100 - 2359 User-agent: Slurp