# robots.txt for http://www.xperiweb.net # 04/31/04 # archive.org User-agent: ia_archiver Disallow: / #surveybot User-agent: SurveyBot Disallow: / # all robots excluding archive.org will spider the domain User-agent: *