crawler.max_download_size

Background

This parameter specifies the maximum download size for any single file during the crawl. The value is specified in MB. The default value is 10 MB.

This download limit applies to both HTML files and binary files (e.g. PDF, Word etc.).

Setting the key

Set this configuration key in the search package or data source configuration.

Use the configuration key editor to add or edit the crawler.max_download_size key, and set the value. This can be set to any valid Integer value.

Default value

crawler.max_download_size=10