#Any search crawler can crawl our site User-agent: * #allow everything else allow: / Disallow: /*.pdf$ Disallow: /*.doc$ Sitemap: https://www.grandcaninehotel.com/robots.txt