crawler.max_url_length

The crawler will ignore all URLs that are larger than this option’s value. The check is based on the number of characters in the URL e.g. if the limit is 256 then any URL with more than 256 characters will be ignored. Typically if the URL is too long, it is likely to be a crawler trap.

Set this configuration key in the search package or data source configuration.

Use the configuration key editor to add or edit the crawler.max_url_length key, and set the value. This can be set to any valid Integer value.