crawler.cache.URLCache_max_size

Background

This is the maximum size of the URL cache which records all URLs seen during the crawl (whether downloaded or not). This directive may be ignored by some cache implementations.

Setting the key

Set this configuration key in the search package or data source configuration.

Use the configuration key editor to add or edit the crawler.cache.URLCache_max_size key, and set the value. This can be set to any valid Integer value.

Default value

crawler.cache.URLCache_max_size=50000000

Notes

May be ignored by some cache implementations.

See also