Funnelback 15.0.0 release notes

Released: 22nd January 2016

15.0.0 - New features

  • Marketing Dashboard Administration Interface (originally called Modern administration dashboard) providing an improved administration interface for common search service management tasks. Specific improvements include:

    • Improved reporting interface, including mapping and time based comparisons.

    • Improved editing interface for best bets, including bets bet previewing.

    • Improved editing interface for curator rules, including a number of new triggers.

    • Improved editing interface for synonyms and search tuning data sets.

    • Ability to publish individual items (such as a best bet) rather than entire configuration files.

    • Better user session management, including the ability to logout without closing the web browser.

    • Introduces a 'service' concept to separate collections purely for back-end gathering purposes from those expected to receive user queries.

  • Structured administration API covering all Administration Interface (originally called Modern administration dashboard) features.

  • Content Auditor 'recommendation' reports:

    • Reading Grade scores for textual content.

    • Missing Metadata identification.

    • Duplicate Title identification.

    • Date Modification reporting.

    • Response Time reporting.

    • Undesirable Text identification.

    • Duplicate Content identification.

  • Push collections now support replicating their indexes to remote query processor servers.

  • Custom filters can now be implemented to receive a parsed HTML DOM model rather than raw bytes/characters using the JSoup Filter interface.

  • TRIMPush collections now support HP Records Manager v8 (formerly TRIM). TRIM v7 is still supported, but previous versions (v6.2, v6.0, v5) are not.

  • Form interaction now supports multiple 'in-crawl' rules, and the form action targets may be defined by regular expressions.

15.0.0 - Selected improvements and bug fixes

  • Funnelback now supports running the public search interface over HTTPS by default - Simply configure the desired HTTPS port during installation.

  • Introduced a new curator trigger which can perform a range of string and numeric comparisons on an arbitrary URL parameter.

  • Introduced a new curator trigger which can trigger when specific facet categories are selected.

  • Introduced numeric comparisons for curator segment triggers.

  • Upgraded API/library versions for several social media sources.

  • Fetching URL information is now isolated from the web server process to reduce the impact of errors during fetch operations.

  • Improved efficiency of duplicate detection within Content Auditor.

  • search.json and suggest.json now return CORS headers to allow cross-origin requests.

  • Modern UI hook scripts now include $SEARCH_HOME/lib/java/groovy and the collection’s @groovy directories on their class path when they are run, allowing for common functions to be stored there and reused.

  • Funnelback now includes a JSON Schema definition of the search.json output to allow for schema comparison between versions.

  • Upgraded filtering libraries, providing fixes for a number of PDF extraction problems and many other filtering issues.

  • Improved logging context and consistency of configuration.

  • Fixed handling of long metadata classes used for security when merging indexes.

  • Fixed handling of long metadata classes beginning with 'd' and 't'.

  • Fixed handling of long metadata classes when using search session features.

  • More robust handling of query/click logs when building search analytics.

  • Improved configuration of matrix connection when using document level security.

  • Improved management of temporary/working files when running Funnelback services.

  • License limit information is now available within the (modern) administration dashboard.

  • Reduced memory usage of some social media gathering operations.

  • Avoid double-wrapping of jsonp search results which occurred in some configurations.

  • Fixed colfield indexer option operation on push collections.

  • Improved push collection debugging and recovery features.

  • Improved a number of cases where invalid HTML was made worse during filtering/accessibility checking.

  • Updated to match new Funnelback logos and branding.

  • Removed constraint on meta collection components requiring matching ordering in metamap.cfg and xml.cfg.

  • Updated Freemarker templating library to 2.3.23.

15.0.0 - Upgrade Issues

  • Some features from Funnelback’s Classic administration dashboard have been removed where they overlap with the Administration Interface (originally called Modern administration dashboard). Services will need to be manually created for relevant collections to allow these features to be used.

  • Funnelback’s Classic search UI is no longer included with the standard Funnelback install. Where required, a separate install package is available to provide Classic Search UI compatibility under Jetty, however Classic Search UI under IIS is no longer supported.

    • As a result of the removal of Classic Search UI, certain static resources used by some Modern UI templates have moved from /search to /s/resources-global. Most references will be automatically fixed during installation, but manual fixes may be required in some cases.

    • Templates referring to search/help/simple_search will need to be updated, as this generic documentation for classic UI is no longer included. Relevant help content could be added to a collection’s web resources folder instead.

  • Administration Interface best bets are implemented using curator rules rather than the historical mechanism. Existing search templates may need to be updated to ensure the resulting adverts are styled as desired.

  • The format of synonyms.cfg files has changed to include some additional id and edit-time properties. Files using the old format will still be read, however only the new format will be written.

  • Funnelback administration no longer supports Internet Explorer version 9 or below - Please ensure all administrators have up-to-date IE versions prior to upgrading.

  • TRIMPush collections: The name of the user defined fields has changed from udfX (where X was the field number) to the actual name of the field, e.g. Cost, Category, etc. If you are using user defined fields you will need to update your metadata mappings to account for this change.

  • Late binding security check scripts invoked during query processing are no longer provided. Implementing early or late binding checks in native code triggered using the secPlugin query processor option is the recommended replacement.

  • The form Modern UI URL parameter cannot be passed blank anymore. It can still be omitted entirely (resulting in the default simple form being used) but passing it blank (i.e. http://server/s/search?collection=c&form=) will result in an 400 Bad Request HTTP response.

  • The Admin API authentication scheme has changed from HTTP Basic to a token based system. Client applications needs to be updated.

  • As Funnelback now supports a per item publication model, all curator rules will be enabled on upgrade. Any rules you do not wish to be enabled should be unpublished removing them from the live search.

  • Jetty web server context directories have been renamed from $SEARCH_HOME/web/conf/contexts-https and $SEARCH_HOME/web/conf/contexts-http to $SEARCH_HOME/web/conf/contexts-admin and $SEARCH_HOME/web/conf/contexts-public

  • Form Interaction: values are now expected to be URL-encoded in form_interaction.cfg, to allow for characters like '&' to be part of passwords. An entry like username=admin&password=&min should be written as username=admin&password=%26min instead.

  • sec.content.optimiser has been renamed to sec.seo-auditor.

  • The search help pages at /search/help/simple_search and /search/help/query_language_help are not available anymore. Form files that point to them should be updated to remove the links.

  • Any analytics database initially created in Funnelback version 9 must be updated from scratch.

  • The following features have been removed - Funnelback Shell, Store Service (superseded by push collection API), File Transfer (superseded by multi-server WebDAV support), EAS (superseded by the new Administration Interface), Late binding security (superseded by early binding security, which can support pluggable late binding checks).

  • SEO Auditor is now available only as part of the Administration Interface, and can no longer be configured for public access.

  • Gathering of TRIM v6 sources is no longer supported (superseded by v7 and v8 support).

  • The authentication mechanisms for the Admin API have been changed to require a login call rather than HTTP Basic headers. See API UI for information on handling the new authentication scheme.

Patches

Type Release version Description

3 Bug fixes

Upgrades log4j2 to version 2.17 to fix the security vulnerability where log4j2 JNDI features do not protect against attacker-controlled LDAP and other JNDI related endpoints.

3 Bug fixes

Prevents creation of objects within Freemarker template files to ensure that template editors can not cause external code to be executed.

3 Bug fixes

Fixes security issues where:

  • The default form-not-found template reflected the given form id without proper escaping.

  • The default configuration of URL previewing could be used to expose local log file content.

Please ensure any custom form-not-found.ftl templates in collections are updated to perform correct escaping if they were derived from the previously vulnerable form-not-found.default.ftl.

Please ensure that any customised value for the global default_url_renderer.permitted_url_pattern setting in global.cfg prevents access to file:// URLs.

3 Bug fixes

Fixed an issue where the user editing interface for a user with no permitted collections would be presented with all collections selected, rather than none.

3 Bug fixes

Fixes a cross site scripting vulnerability when unescaped HTML was provided to the CheckBlending macro’s linkText attribute.

3 Bug fixes

Corrected the XSS Vulnerability in Anchors.html

3 Bug fixes

Fixes a bug where configs would not be reloaded in some multi server environments.

3 Bug fixes

Fixes a bug where data loss could occur in Push collections if commits failed.

3 Bug fixes

Fixes a bug on Windows where commits could fail if index files in a snapshot are held opened.

3 Bug fixes

Fixes various DLS security flaws.

3 Bug fixes

Fixes a bug where data loss could occur in push on Windows. The problem is more likely to occur when Push is used in a meta collection.

3 Bug fixes

Fixes a race condition when saving a meta collection configuration on Windows if a component collection is updating in the background.

3 Bug fixes

Fixes a bug with Curator based Best Bets, where an OutOfMemoryError would be thrown.