cookies.txt is a data source configuration file used to set pre-defined cookie values used by the crawler when updating a web data source. A web data source’s
cookies.txt file is generally retrieved as part of a pre-gather workflow command.
cookies.txt file is present it will be read at crawl start-up and any cookies parsed will then be used during the crawl. Any messages relating to errors parsing the
cookies.txt file will be in the main
cookies.txt file defines a cookie for the domain
www.example.com with no expiry date and a
name=value setting of
# Domain Tailmatch Path Secure Expires Name Value www.example.com TRUE / FALSE 0 id 1234