# All robots are allowed to crawl User-agent: * Allow: / Disallow: /archive/cat/ Disallow: /archive/go/