crawler.request_timeout
Timeout for HTTP page GETs (milliseconds)
Key: crawler.request_timeout
Type: Integer
Can be set in: collection.cfg
Description
This is the maximum number of milliseconds that the crawler will wait for a HTTP GET request to return. If the web server does not start to return the page in question before this timeout, the crawler assumes the page was unavailable.
| This value is multiplied by 4 to give the parser (link extraction) timeout. |