# robots.txt for http://www.pa.msu.edu/ User-agent: * Disallow: /old/ # This area is for storage, of no interest to search engines Disallow: /test/ # This area is for testing, of no interest to search engines Disallow: /test1/ # This area is for testing, of no interest to search engines Disallow: /test2/ # This area is for testing, of no interest to search engines Disallow: /test_cpw/ # This area is for testing, of no interest to search engines Disallow: /new/ # This area is for testing, of no interest to search engines Disallow: /textonly/ # This area currently undergoing testing, of no interest to search engines # # The following areas are for image storage, # not for text files anyone would want to do searches on/for Disallow: /graphics/ # image storage Disallow: /images/ # image storage Disallow: /bullets/ # image storage Disallow: /buttons/ # image storage Disallow: /backgrounds/ # image storage # Disallow: /wwwstat/ # This area is for web server statistics, which (while acceptible for access # otherwise) end up generating a lot of false positive hits, so shouldn't be # available to web crawlers and robots Disallow: /stats/ # This area is just an alias for the 'wwwstat' area, disallowed above # Disallow: /cgi-bin/ # This area is for programs, not text files Disallow: /services/computing/cgi-local/ # This area is for programs, not text files # Disallow: /helpdesk/mail/ # This area is for internal problem resolution, of no interest to search engines Disallow: /ftp/pub/incoming/ # This area is for incoming files only; not to be catalogued by search engines # Disallow: /reference/load/ # This area has system load graphs - no need to search on their contents (it would # # probably just confuse the search engine anyway Disallow: /reference/old/ # This area is for storage, of no interest to search engines # Disallow: /committees/comp_ops/mail_lists.html # if this should be stumbled upon, it should still not be indexed # Disallow: /ftp/pub/zepf/ # restricted at Steve Zepf's request # # The following SOAR areas were originally private, with .htaccess files, but since the Sun web server doesn't # enforce the .htaccess restrictions, block robot access at least Disallow: /soar/private/ Disallow: /soar/private/backup/ Disallow: /soar/local/ Disallow: /soar/SWG/discuss/ Disallow: /soar/SWG/docs-draft/ Disallow: /soar/SWG/old/ Disallow: /soar/SWG/docs-int/ Disallow: /soar/SWG/business/ Disallow: /soar/SWG/minutes-int/ Disallow: /soar/board/discuss/ Disallow: /soar/board/business/ Disallow: /soar/board/minutes-int/ Disallow: /soar/archives/old/ Disallow: /soar/archives/internal/ Disallow: /soar/operations/discuss/ Disallow: /soar/operations/docs-draft/ Disallow: /soar/operations/business/ Disallow: /soar/operations/minutes-int/ Disallow: /soar/operations/docs-int/ Disallow: /soar/operations/docs-int/ftp/ Disallow: /soar/upcoming/internal/ Disallow: /soar/consortium/discuss/ Disallow: /soar/consortium/minutes-int/ Disallow: /soar/policy/discuss/ Disallow: /soar/policy/minutes-int/ Disallow: /soar/webnotice/NoticesDocs/ Disallow: /soar/webnotice/ # # appears to be at least semi-private Disallow: /cmp/birge-group/internalstuff