crawler.monitor_delay_type
Type of delay to use during crawl (dynamic or fixed).
Key: crawler.monitor_delay_type
Type: MonitorDelayTypes
allowed values: dynamic
or fixed
Can be set in: collection.cfg
Description
This parameter specifies what type of delay the webcrawler will use between each request to a web server. There are two types of delay supported:
-
dynamic
: (default) Calculate delay based on time taken for previous request to this server. -
fixed
: Use the same fixed delay after each request.