Help Center

Menu

crawler.frontier_hosts

Background

This option allows you to specify which hosts crawlers are running on when using a DistributedFrontier (distributed crawling). Each host will be responsible for its own subset of the web servers being crawled.

Setting the key

Set this configuration key in the search package or data source configuration.

Use the configuration key editor to add or edit the crawler.frontier_hosts key, and set the value. This can be set to any valid List<String> value.

Default value

(empty)

Examples

Specify the IP addresses of the hosts that the distributed crawl is running on.

crawler.frontier_hosts=12.67.89.8,12,67.89.9

See also