# Robots.txt file from http://www.marketing.affiliatexfiles.com # User-agent:Googlebot Disallow: all User-agent: googlebot-Image Disallow: /images/ User-agent: ia_archiver Disallow: User-agent: Mozilla/3.0 (compatible;miner;mailto:miner@miner.com.br) Disallow: User-agent: Due to a deficiency in Java it's not currently possible to set the User-agent. Disallow: User-agent: * Disallow: all