Help Center

Menu

Funnelback 15.22.0 release notes

Released: 31 July 2019

Supported until: 23 October 2020

New features

Overhauled configuration editing with per-setting security

Introduced new interfaces for editing configuration settings on the global, collection and profile levels. These new screens provide easier access, link to documentation for each individual setting and list default values during configuration.

New user configuration settings allow individual users to be permitted access to each individual setting on a case by case basis, via roles if desired. This allows Funnelback administrators to grant inexperienced implementers access to the many low-risk configuration settings while retaining control of any that could affect server performance or stability.

API for collection creation and configuration

Funnelback now provides REST API endpoints for creating and configuring new collections programmatically. This allows other systems or custom scripts to create new collections as required, rather than requiring a person to do so via Funnelback’s administration interfaces.

WebDAV support for editing common configuration files

The following files have been added to the modernized configuration file editing interface available through the Configuration file manager. These files can also now be managed through a WebDAV client or through scripted WebDAV HTTP calls.

SAML compatibility with ADFS

Introduced a SAML algorithm setting auth.saml.algorithm which allows Funnelback to connect to an ADFS SAML identity provider without changing ADFS' default hash algorithm.

Improvements

  • Introduced auth.user.delay-length-seconds setting to make brute force password guessing impractical.

  • Added official support for comments within global, collection and profile configuration files.

  • To improve security the Server header is no longer returned in HTTP responses.

  • Improved the performance and memory usage of the administration API schema and documentation.

  • Reorganized release note documentation and added a release note facet category within documentation search.

  • Added support for 'training' license keys.

  • Added support for per-environment configuration settings which allows the value of a collection or profile configuration setting to vary between servers within a multi-server environment.

  • The Marketing UI is now able to embed tracking for multiple Google Analytics accounts simultaneously.

  • When Google Analytics is used within Funnelback the anonymizeIp setting is now enabled automatically.

  • Improved presentation of knowledge graph tasks within the task queue interface.

  • Added an installer setting to replace any existing admin user configuration with the latest default admin settings.

  • Added auto-scrolling to matches when using the find tool in the configuration file editor.

  • Knowledge graph no longer creates self-referencing mentions relationships.

  • Simplified uploading replacement files in the configuration file web interface.

  • Changed the knowledge graph service to run as the funnelback user rather than root on Linux.

  • Improved how padre query biased summaries are generated such that spaces are added based on the source document, thus preventing issues where extra spaces could be added or removed.

  • The API fields that are requested from Facebook can now be specified in collection.cfg using the facebook.page-fields, facebook.post-fields and facebook.event-fields configuration keys.

  • Improved the query response time when using sort options, dramatically so for title sorting in particular.

  • Added a default role 'default-super-user' which grants a user access to everything, it is recommended super users are added to this role.

Upgrades to Funnelback components

  • The default version of Handlebars used in the default template and knowledge graph widget preview has been upgraded to 4.1.2.

  • Upgraded the included JDK to 11.0.3 update 7.

  • Upgraded the included version of Jetty to 9.4.19 which solves a rare concurrency issue during startup.

  • Upgraded the included Spring Framework to 5.1.4.

  • Upgraded the RestFB library to 2.21.0.

  • Upgraded the Angular version to 7.2.10.

Bug fixes

  • Fixed the accessibility auditor URL based facets to drill-down as intended.

  • Fixed an issue with excessive constraints being applied through faceted navigation which could occur when indexing XML with external metadata.

  • Improved caching of SAML based users to make role changes take immediate effect rather than waiting for the next login.

  • Removed the possibility of removing expected directories such as a profile’s web resources directory via WebDAV.

  • Fixed WebDAV permission interpretation to match the intended behaviour where non-sensitive configuration files should not require specific permissions to be read.

  • Fixed padre sorting to cause results in tiers after the first to be sorted when a sort mode is enabled.

  • Solved an issue where updating configurations during upgrade could revoke previously granted permissions.

  • Fixed HTTP status code when loading implementer UI pages.

  • The knowledge graph widget now retains facet selections when paging through results.

  • Searching within knowledge graph can no longer return results not represented as nodes within the graph.

  • Previewing the knowledge graph widget form the administration interface now ensures that the preview is loaded with a valid node.

  • Improve handling of URLs with invalid port values when generating broken link reports.

  • Fixed output of 'compare with live' tool when used with an empty preview configuration file.

  • Fixed knowledge graph relationship editor’s copy function to use a valid name for the duplicate.

  • Knowledge graph’s label editor no longer prevents creating a label when the metadata class in question is already been labeled for a different entity type.

  • Knowledge graph’s relationship editor now correctly preserves relationship direction configuration.

  • Fixed faceted navigation on metadata when the metadata values are sourced from both XML and external metadata.

  • Made the uninstaller remove the knowledge graph service.

  • Fixed the scheduled task interface when running on Windows 2016.

  • Fixed an issue which prevented the knowledge graph restarting when expected.

  • Fixed an issue where some configuration settings (e.g. ui.modern.search_link) were returning null when empty-string was expected in the template data model.

  • Added missing translations keys and side help for knowledge graph

  • Disabled HTTP/2 in underlying HTTP library which caused socket timeout errors during crawling

  • Fixed an issue where jetty would terminate on invalid 'index.autoc' (query completion) files.

  • Directory collections no longer incorrectly strip some unicode characters.

  • Fixed an issue where jetty stopped logging after deploying knowledge graph.

  • Fixed an issue where Facebook collections did not update due to the recent Graph API changes.

  • Fixed an issue in the knowledge graph widget where the value of the targetUrl query parameter would be truncated if it contained a '?' character.

  • Fixed an issue where the update could fail during the record Accessibility Auditor history steps after migrating a collection from an older version.

  • Fixed an issue where the deprecated wcag check (replaced by Accessibility Auditor) could cause the update to fail because the 'wcag-journal.log' file could not be found.

  • Fixed an issue where the WebDAV endpoint would not re-prompt for HTTP Basic authentication when invalid credentials were provided, which could lead to a redirect loop in the Cyberduck WebDAV client.

  • Fixed an issue where the knowledge-graph.max_heap_size setting was not applied when updating knowledge graph.

  • Fixed an issue where browsers would display a mixed-content error when the knowledge graph API returns URLs with varying protocols.

  • Fixed an issue where the knowledge graph widget returned a cross-origin resource sharing policy error. The knowledge graph API now supplies Access-Control-Allow-Origin headers.

  • Fixed an issue where large (>2GB) index.dt files would cause padre-gs to fail when setting gscopes.

  • Fixed an issue where faceted navigation URLs in the knowledge graph widget search view were double prefixed with a domain.

  • Fixed an issue where knowledge graph public /type and /rels APIs were not converted to absolute URLs.

  • Fixed an issue where result URLs from filecopy collections were not resolved correctly in knowledge graph due to their relative syntax. The new knowledge graph widget parameter urlPrefix has been added to allow converting relative result URLs to absolute ones.

Important changes

  • In multi server environments, collection configuration changes in collection.cfg no longer have a separate 'publish' step to replicate between servers. Since the replication occurs immediately on saving we encourage the use of profile.cfg for all query time settings so that they can be previewed and then published to the profile’s live view when ready.

  • The padre query processor now sorts results in all result tiers rather than just the first when a sort mode is enabled.

Changes to configurations

  • On upgrade, the contents of collection.cfg.start.urls will be moved to the collection’s collection.cfg start_url setting if the file is less than 20kb in size and the current start_url value is the previous 'disabled' message, as long as no environment-specific start urls files are configured.

  • The collection.cfg file’s start_url option’s meaning has changed. The crawler now also crawls from start URLs set in this option, in addition to URLs specified in the file referenced by crawler.start_urls_file. Previously, if start_url was set to anything other than _disabledsee_start_urls_file, the crawler would crawl from the single URL specified in start_url. In general, upon upgrade, URLs in the file collection.cfg.start.urls are moved into start_url, as long as start_url is not set in collection.cfg, or when start_url is set to _disabledsee_start_urls_file. For web and matrix collections, start_url must be set inside collection.cfg to prevent this upgrade from running again. (Setting start_url without a value - e.g. start_url= 'sets' this property to empty, which can be used to prevent this upgrade task.)

  • Funnelback now supports the editing of global.cfg which is only available to users who are granted the new sec.server.config permission. On upgrade, users will not be automatically granted that permission as editing these settings was previously possible only with command-line access.

  • Users who previously had file manager permission to access the collection.cfg file are automatically granted the configuration permissions read.all-keys and edit.all-keys, sec.can-read-all-unknown-config-keys and sec.can-edit-all-unknown-config-keys.

  • Users who previously had file manager permission to access the profile.cfg file are automatically granted configuration permissions to read and edit all known profile-level keys, sec.can-read-all-unknown-config-keys and sec.can-edit-all-unknown-config-keys.

  • Users who previously had file manager permission to access the cookies.txt file are automatically granted the new sec.cookie-config permission.

  • Users who previously had file manager permission to access the custom_gather.groovy file are automatically granted the new sec.custom-gather permission.

  • Users who previously had file manager permission to access at least one of hook_extra_searches.groovy, hook_post_datafetch.groovy, hook_pre_datafetch.groovy, hook_pre_process.groovy are automatically granted the new sec.hook-script permission.

  • Users who previously had file manager permission to access the meta-names.xml file are automatically granted the new sec.meta-name permission.

  • Users who previously had file manager permission to access the workflow.cfg file are automatically granted the new sec.workflow-config permission.

  • The API fields that are requested from Facebook can now be specified in collection.cfg using the facebook.page-fields, facebook.post-fields and facebook.event-fields configuration keys. This should enable future changes to Facebook’s APIs to be handled without requiring further Funnelback patches.

  • The ui.modern.related-document-fetch.[relatedDocumentKey] key has been renamed to ui.modern.related-document-fetch.[relatedDocumentKey].type. The old keys is converted automatically upon upgrade.

Upgrade notes

  • Upgrading from versions 12.2.0 and earlier is no longer supported, first upgrade to version 15.20.0 and then upgrade to the latest version. Unless stated otherwise no future version of Funnelback will support upgrading directly from 12.2.0 or earlier.

  • The padre indexer XML parser is now less lenient when indexing numeric, date and geo location metadata. Previously, elements that where not correctly closed, such as <v>123</oops>, could have been mapped to //v. In this version of Funnelback, such mappings will no longer work. This is inline with how other metadata is mapped.

  • Centos 6 is not supported in this and subsequent versions of Funnelback.

  • Unlike Funnelback version 15.20, when upgrading a users and roles, permissions that are derived from file manager rules will now require sec.file.manager instead of sec.file.manager.edit.

  • To upgrade Handlebars to 4.1.2, update the path to resource in FTL forms from ${GlobalResourcesPrefix}thirdparty/handlebars-4.0.12/handlebars.min.js to ${GlobalResourcesPrefix}thirdparty/handlebars-4.1/handlebars.min.js. Note that previous versions of Handlebars will be removed from Funnelback in a future release.

Patches

Type Release version Description

3 Bug fixes

15.22.0.34

Fixed the security vulnerability where Spring Framework RCE may be vulnerable to remote code execution (RCE) via data binding [CVE-2022-22965]

3 Bug fixes

15.22.0.33

Upgrades log4j2 to version 2.17 to fix the security vulnerability where Apache Log4j2 Thread Context Lookup Pattern is vulnerable to remote code execution in certain non-default configurations.

3 Bug fixes

15.22.0.32

Upgrades log4j2 to version 2.15 to fix the security vulnerability where log4j2 JNDI features do not protect against attacker-controlled LDAP and other JNDI related endpoints.

3 Bug fixes

15.22.0.31

Fixes a cross-site scripting vulnerability in Freemarker templates.

3 Bug fixes

15.22.0.30

Reduces memory usage when returning search results as XML.

3 Bug fixes

15.22.0.29

Fixes an issue where sessions are not terminated on logout events triggered by perl pages.

3 Bug fixes

15.22.0.29

Fixes an XXE issue where input to the webdav endpoint could be manipulated to trigger http requests.

3 Bug fixes

15.22.0.28

Fixes admin-ui handling of profiles with hyphens in their IDs.

3 Bug fixes

15.22.0.27

Fixes an issue where Faceted Navigation extra searches may fail because of an index out of bounds error.

3 Bug fixes

15.22.0.27

Improves logging when extra searches take too long.

3 Bug fixes

15.22.0.26

Fixes an issue where marketing dashboard refers a non-existing URL when ui.integration_url is not configurable at the profile level.

3 Bug fixes

15.22.0.25

Fixes a NullPointerException in ViewModeBanner macro when SAML is enabled

3 Bug fixes

15.22.0.24

Fixes ip pseudonymization when Funnelback is behind a load balancer and client ip details are in the x-forwarded-for header.

3 Bug fixes

15.22.0.23

Reduces memory consumption and improves performance of the purge sessions endpoint

3 Bug fixes

15.22.0.22

Upgrades the version of RestFB library to account for recent breaking changes in the Facebook Graph API.

3 Bug fixes

15.22.0.22

Fixes an issue where Facebook collections gathered less number of documents due to a pagination issue in the Facebook Graph API.

3 Bug fixes

15.22.0.22

Facebook Graph API deprecated fields name, link, app_links and description have been removed from the default values of the facebook.page-fields configuration key.

3 Bug fixes

15.22.0.21

Fixes an XML formatting issue in Faceted Navigation click logs.

3 Bug fixes

15.22.0.20

Fixes a bug with merging under Push.

3 Bug fixes

15.22.0.19

Fixes a bug in which white space was not preserved in summaries from anchor text when the -map indexer option is enabled.

3 Bug fixes

15.22.0.18

The Push API client used in multi server push now has timeouts enabled allowing it to abandon problematic HTTP requests.

3 Bug fixes

15.22.0.17

Removes the screens for file-manager rule editing which could create security issues

3 Bug fixes

15.22.0.17

Fixes an issue where support packages could contain unintended files

3 Bug fixes

15.22.0.17

Fixes an issue where the running Funnelback jetty web server could retain permissions via supplemental groups after startup

3 Bug fixes

15.22.0.17

Limits an administration CGI script to redirect only within the Funnelback administration interface as intended

3 Bug fixes

15.22.0.17

Removes the unused administration debug.cgi script which reflected input parameters without proper escaping

3 Bug fixes

15.22.0.16

Fixes a bug where a horizontal display of columns in auto-completion dropdown doesn’t work.

3 Bug fixes

15.22.0.16

Fixes a bug where insecure operation on CSS files list was performed when CSS file was exposed via the same domain as auto-completion widget but different port.

3 Bug fixes

15.22.0.15

Fixes an issue where the push API failed to start up when using SAML authentication.

3 Bug fixes

15.22.0.14

Fixes an issue where Knowledge Graph groovy scripts are not executed when they are defined at the profile preview level.

3 Bug fixes

15.22.0.14

Fixes an issue where the Knowledge Graph class CsvImporterNeo4j does not create nodes inside Neo4J when executed externally.

3 Bug fixes

15.22.0.14

Fixes an issue where Knowledge Graph API does not work when a JDBC driver is specified for the session database.

3 Bug fixes

15.22.0.13

Fixes a bug introduced in the previous patch in which uploading configuration files in the administration dashboard stopped working.

3 Bug fixes

15.22.0.12

Prevents XSS vulnerabilities found in the classic administration dashboard.

3 Bug fixes

15.22.0.11

Move Funnelback service pid files to /var/run which is required by OS updates to systemd.

3 Bug fixes

15.22.0.11

Include some additional metadata in service template files.

3 Bug fixes

15.22.0.10

Relax permissions on creating a service. If user has access to create a profile (sec.profile.manage) they now have access to create a service. Previously they needed the more restrictive requirement of sec.administer.system.

3 Bug fixes

15.22.0.9

Fixed an issue where the crawler would follow <meta http-equiv="refresh"> redirects that appeared within html comments. Redirects inside comments are now ignored.

3 Bug fixes

15.22.0.8

Improves support for running faceted navigation on extra searches.

3 Bug fixes

15.22.0.8

Adds method 'getEffectiveExtraSearchName()' to the search transaction which gets the name of the extra search this search should be considered to be under. The result of this should be used when modifying a particular extra search. As Funnelback may create extra searches under an existing search, for example for faceted navigation, this could be used to work out if the search transaction should be modified.

3 Bug fixes

15.22.0.8

Fixes errors in the sorting of faceted navigation values, which could cause a HTTP 500 error code.

3 Bug fixes

15.22.0.7

Prevent XSS AngularJS sandbox bypassing injection in Freemarker templates escaped using output formats by inserting zero-width whitespace between consecutive open-curly-brackets.

3 Bug fixes

15.22.0.6

Empty XML elements mapped as the document URL are now ignored.

3 Bug fixes

15.22.0.6

Fixes a memory leak by disabling the conscrypt SSL provider.

3 Bug fixes

15.22.0.5

Prevent XSS AngularJS sandbox bypassing injection in Freemarker templates by inserting zero-width whitespace between consecutive open-curly-brackets.

3 Bug fixes

15.22.0.5

Please note, this patch was retracted due to an incomplete solution causing template errors when used with certain Freemarker escaping modes. The 15.22.0.7 patch, which addresses this issue, should be used instead.

3 Bug fixes

15.22.0.4

Fixes a bug in which the last seen time of an Accessibility Auditor Acknowledgment would not be updated.

3 Bug fixes

15.22.0.4

Fixes a bug in which spaces would be removed from query biased summaries which came from 'cdata' sections of XML.

3 Bug fixes

15.22.0.4

Improves the task-picker such that it can load dependencies from custom 'jar' files located '$SEARCH_HOME/lib/java/task-picker/'.

3 Bug fixes

15.22.0.4

Improves query performance when lots of curator rules are defined for any profile under a collection.

3 Bug fixes

15.22.0.4

Improves 'build_autoc' performance for profiles reducing update times.

3 Bug fixes

15.22.0.4

Reduces the time taken by the update step ContentAuditorSummary.

3 Bug fixes

15.22.0.3

Prevent XSS AngularJS sandbox bypassing injection in Freemarker templates by inserting zero-width whitespace between consecutive open-curly-brackets.

3 Bug fixes

15.22.0.3

Reduce the size of the redirector war file to reduce memory overhead and deploy time.

3 Bug fixes

15.22.0.2

Improve support for faceted navigation with queries that contain bigrams (such as CJKT characters).

3 Bug fixes

15.22.0.1

Fixes an issue where relationships labels in the knowledge graph widget have been created incorrectly.

3 Bug fixes

15.22.0.1

Fixes an issue where sorting of related entities has not been applied based on knowledge graph template configuration.

3 Bug fixes

15.22.0.1

Fixes an issue where a double request to search endpoint in the knowledge graph widget has been sent on pressing enter key when search input is focused.

3 Bug fixes

15.22.0.1

Fixes an issue where the saving of profile and server level configuration parameters have not been executed due to incorrect backup file creation.

3 Bug fixes

15.22.0.1

Improve the performance of the Accessibility Auditor interface by requesting less data.

3 Bug fixes

15.22.0.1

Fixes an issue where some of the text on the Accessibility Auditor dashboard was showing out of date information.

3 Bug fixes

15.22.0.1

Fixes an issue where the Accessibility Auditor dashboard would not generate the thumbnail screenshots for each domain.

3 Bug fixes

15.22.0.1

Tuning now uses the Perl defined in executables.cfg rather than the Perl defined on the path