# Tells Scanning Robots Where They Are And Are Not Welcome # User-agent: can also specify by name; "*" is for everyone # Disallow: if this matches first part of requested path, # forget it User-agent: * Disallow: /documents/ Disallow: /Documents/ Disallow: /Cool/ Disallow: /cool/ Disallow: /COOL/ Disallow: /Archive/ Disallow: /archive/