cookies.txt is a data source configuration file used to set pre-defined cookie values used by the crawler when updating a web data source. A web data source’s cookies.txt file is generally retrieved as part of a pre-gather workflow command.

If the cookies.txt file is present it will be read at crawl start-up and any cookies parsed will then be used during the crawl. Any messages relating to errors parsing the cookies.txt file will be in the main gather.log file.


Netscape cookie file format, one cookie per line. Comment lines are preceded with a hash.


The example cookies.txt file defines a cookie for the domain with no expiry date and a name=value setting of id=1234.

# Domain     Tailmatch  Path  Secure Expires Name  Value  TRUE        /    FALSE  0       id    1234