Threshold for number of times a page revisit has been skipped when deciding whether to revisit it.

Key: crawler.revisit.num_times_revisit_skipped_threshold
Type: Integer
Can be set in: collection.cfg

Table of Contents


This parameter specifies a threshold to use when looking at how many times we have skipped revisiting a URL (and used a copy of its content instead).

A particular revisit policy might decide that if the number of times we have skipped a revisit for a given URL is higher than the threshold then a revisit would be required. The "number of times a revisit has been skipped" value for that URL would then be reset to zero.

Default Value