Help Center

Menu

crawler.max_parse_size

Background

The crawler will stop parsing documents larger than the specified value (in MB), and their content will be truncated. This only applies to MIME types listed in the crawler.parser.mimeTypes parameter (e.g. HTML, text, XML). Here parsing refers to link extraction from these file types.

Setting the key

Set this configuration key in the search package or data source configuration.

Use the configuration key editor to add or edit the crawler.max_parse_size key, and set the value. This can be set to any valid Integer value.

Default value

crawler.max_parse_size=10

Examples

Increase the limit to 15MB.

crawler.max_parse_size=15

See also