# This says to apply these settings to ALL search engine spiders/crawlers User-agent: * Sitemap: http://www.ibexuk.com/sitemap.xml # These settings will keep spiders from indexing your unwanted pages Disallow: /images/ Disallow: /Scripts/ Disallow: /SpryAssets/