# robots.txt for http://saas.nsw.edu.au/ # Don't visit any URL starting with "/stubabes/" blocks /stubabes/* # Don't visit any URL starting with "/index" - blocks index* # ============================================================= User-agent: * Disallow: /js # not a public document Disallow: /doc # not a public document Disallow: /css # not a public document Disallow: /img # not a public document Disallow: /lib # not a public document Disallow: /tmp # not a public document Disallow: /secure # not a public document Disallow: /awstats # not a public document Disallow: /style # css stylesheets Disallow: /jscript # javascripts Disallow: /cgi-bin/ # cgi files go here Disallow: /images/ # images only # ===============================================================