crawler.monitor_delay_type
Type of delay to use during crawl (dynamic or fixed).
Key: crawler.monitor_delay_type
Type: MonitorDelayTypes allowed values: dynamic or fixed
Can be set in: collection.cfg
Description
This parameter specifies what type of delay the webcrawler will use between each request to a web server. There are two types of delay supported:
-
dynamic: (default) Calculate delay based on time taken for previous request to this server. -
fixed: Use the same fixed delay after each request.