# # robots.txt for http://publishingcourses.stanford.edu/ # # $Id: robots.txt, 2001/04/25 judith blankman Exp $ # # applies to all robots and disallows files that have an expiration date in the meta tag User-agent: * Disallow: /Out-Of-Date # applies to all robots and disallows indexing of files in various folders User-agent: * Disallow: /dev Disallow: /_private Disallow: /archive Disallow: pow(old) Disallow: /pow2001