Help Center

Menu

crawler.overall_crawl_timeout

Background

This option specifies how many minutes, or hours, the crawler is allowed to run. The option crawler.overall_crawl_units specifies the unit of measure.

This parameter can be left empty, in which case the crawler will keep going until there are no URLs left in its frontier, or it reaches another limit e.g. crawler.max_files_stored

Setting the key

Set this configuration key in the search package or data source configuration.

Use the configuration key editor to add or edit the crawler.overall_crawl_timeout key, and set the value. This can be set to any valid Integer value.

Default value

crawler.overall_crawl_timeout=24

Examples

If you are testing a new web data source, then it can be useful to run a short crawl, say 10 minutes:

crawler.overall_crawl_timeout=10
crawler.overall_crawl_units=min

See also