# robots.txt for dwheeler.com # See http://www.robotstxt.org for more about robots.txt files. # Ignore "frozen" files - they are merely copies of the real thing. # Also ignore secure-programs/old; they're old versions, and at least # one robot (Teleport Pro/1.28) seems to go completely out of control # if presented directly with a directory to index. # I _could_ just block Teleport Pro, but I don't really want indexing # to lead people to old versions instead of current material anyway. User-agent: * Disallow: /frozen Disallow: /secure-programs/old Disallow: /chessclub/bin Disallow: /chessclub/samples Disallow: /chessclub/cgi-bin Disallow: /chessclub/matches Disallow: /archive