User-agent: * Disallow: /index.php? Disallow: /cgi-bin/ Disallow: /generator/ Disallow: /css/ Disallow: /flash/ Disallow: /fckeditor/ Disallow: /js/ Disallow: /pdf/ Disallow: /stage/ Disallow: /test/ Disallow: /tours/ Disallow: /lists/ Disallow: /articles/print/ User-Agent: AhrefsBot Disallow: /