Help Center

Menu

crawler.frontier_num_top_level_dirs

Background

This optional parameter can be used to specify the number of top level directories to use when storing files that are required for a disk based frontier.

Setting the key

Set this configuration key in the search package or data source configuration.

Use the configuration key editor to add or edit the crawler.frontier_num_top_level_dirs key, and set the value. This can be set to any valid Integer value.

Default value

(empty)

Examples

If you are performing a very large crawl you may run into filesystem limits on the number of files allowed in a single directory. To get around this you might split the frontier files among 100 top level directories by setting:

crawler.frontier_num_top_level_dirs=100

See also