User-agent: * Allow: / User-agent: * #Crawl-delay: 30