User-agent: * Disallow: /includes/ Disallow: /adm/ Disallow: /config.php Disallow: /extension.inc Disallow: /common.php # Crawlers Setup User-agent: * Crawl-delay: 10 User-agent: * Disallow: /*/blackhole/*