crawler.allow_concurrent_in_crawl_form_interaction

Background

This experimental option allows concurrent form interaction processes to be prevented which may work around problems in some web single sign-on environments.

When using in-crawl form interaction the web crawler normally processes required form interaction while fetching the page which requires it, which may mean that multiple in-crawl form interaction processes occur concurrently.

In some web authentication environments this may pose a problem because the web crawler shares cookies between all crawler threads. For example, if a separate SSO authentication server relies on cookies to control the post-authentication return page, two parallel logins may not behave as expected.

Please note that the locking necessary to prevent concurrent form interaction may have a detrimental effect on web crawler scalability/performance.

this option is experimental it may be subject to major changes without backwards compatibility in future versions of Funnelback.

Setting the key

Set this configuration key in the search package or data source configuration.

Use the configuration key editor to add or edit the crawler.allow_concurrent_in_crawl_form_interaction key, and set the value. This can be set to any valid Boolean value.

Default value

crawler.allow_concurrent_in_crawl_form_interaction=true

Examples

Prevent concurrent form interaction processing

crawler.allow_concurrent_in_crawl_form_interaction=false