# www.robotstxt.org/ # Allow crawling of all content User-agent: * Disallow: /admin/ Disallow: /bigtree/ Disallow: /usershop/ Disallow: /user/ Disallow: /site/blog-and-community/tags/