crawler.revisit.num_times_revisit_skipped_threshold

Background

This parameter specifies a threshold to use when looking at how many times we have skipped revisiting a URL (and used a copy of its content instead).

A particular revisit policy might decide that if the number of times we have skipped a revisit for a given URL is higher than the threshold then a revisit would be required. The "number of times a revisit has been skipped" value for that URL would then be reset to zero.

Setting the key

Set this configuration key in the search package or data source configuration.

Use the configuration key editor to add or edit the crawler.revisit.num_times_revisit_skipped_threshold key, and set the value. This can be set to any valid Integer value.

Default value

crawler.revisit.num_times_revisit_skipped_threshold=2