Pentester Labs Recon 00 write up

The goal is to find the robots.txt The robots.txt file is used to tell the web spiders how to crawl a website. To avoid having confidential information indexed and searchable, webmasters use this file to tell spiders to avoid specific pages. This is done using the keyword disallow

Solution: Go to the said domain hackycorp.com/robots.txt, and the flag will be present