# Exclusion definitions for "GoogleBot" and other search engines / web crawlers # ____________ ºº][TML ßrowser # | robots.txt | # ¯¯¯¯¯¯¯¯¯¯¯¯ CyBeR CeNtRuM # ______________________________________________________________________________________ # Version Date By Description # -------------------------------------------------------------------------------------- # 1·0·0 2006-04-14 HBr/CC Created. # 1·0·1 2006-05-01 HBr/CC Allow normal access for the 'Alexa' crawler. # 1·0·2 2006-06-26 HBr/CC Exclusion of audio & video samples. # 1·2·0 2007-05-04 HBr/CC Exclusion of test environment. # 1·2·1 2007-06-01 HBr/CC Exclusion of /filedate.php*. # ¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯ User-Agent: Googlebot-Image Disallow: /images/schoonmoeder.png User-agent: * # attention all robots: # web-author ¦ content-provider: Crawl-delay: 20 Disallow: /*.css$ Disallow: /*.dat$ Disallow: /*.doc$ Disallow: /*.eml$ Disallow: /*.eot$ Disallow: /*.ico$ Disallow: /*.JS$ Disallow: /*.kmz$ Disallow: /*.MIDI$ Disallow: /*.MP3$ Disallow: /*.pdf$ Disallow: /*.php$ Disallow: /*.rtf$ Disallow: /*.swf$ Disallow: /*.TTF$ Disallow: /*.wma$ Disallow: /*.wav$ Disallow: /*.wmv$ Disallow: /*.wpl$ Disallow: /*.wri$ Disallow: /*.xsl$ Disallow: /_vti_cnf/ Disallow: /audio/ Disallow: /beacons/ Disallow: /calendar/ Disallow: /cgi-bin/ Disallow: /english/booking.html Disallow: /english/error400.html Disallow: /english/error401.html Disallow: /english/error403.html Disallow: /english/error404.html Disallow: /english/error500.html Disallow: /english/error509.html Disallow: /english/membership_apply.html Disallow: /error/ Disallow: /favos/ Disallow: /filedate.php* Disallow: /icons/ Disallow: /images/ Disallow: /JavaApplets/ Disallow: /members/ Disallow: /plesk-stat/ Disallow: /statistics/ Disallow: /stats/ Disallow: /test/ Disallow: /usage/ Disallow: /video/ Disallow: /w3c/ Disallow: /WEB-INF/ #