crawler.revisit.num_times_unchanged_threshold

Background

This parameter specifies a threshold to use when looking at how many times the content of a given URL has been unchanged over the course of multiple crawls.

A particular revisit policy might decide that if the number of times a URL has been unchanged is higher than the threshold then a revisit would not be required.

Increasing this threshold would result in more revisits (HTTP HEAD and/or GET requests) being performed.

Setting the key

Set this configuration key in the search package or data source configuration.

Use the configuration key editor to add or edit the crawler.revisit.num_times_unchanged_threshold key, and set the value. This can be set to any valid Integer value.

Default value

crawler.revisit.num_times_unchanged_threshold=5