# robots.txt for http://www.webforums.ex.ac.uk/ # All spiders should avoid Everything User-agent: * Disallow: / User-Agent: OmniExplorer_Bot Disallow: /