Help Center

Menu

Funnelback DXP change log

The change log below provides details on changes that have been made in Funnelback since the v15.24 release. Please see what’s new in Funnelback 16 for an overview of the changes.

Use the filter to narrow down what’s in the list below, and the sort controls on each column to help locate what you are looking for.

Type Date Release version Description

4 Important changes

2024-12-09

16.30.0

Increased default value for Analytics max_heap_size.

Details

There were issues reported on Analytics report not showing anything on screen. On further investigation "Out of memory" errors were found on logs. To fix these issues, max_heap_size default value is increased to 1 GB. Also, customers can increase those values to 4 GB whenever required using this key "analytics.max_heap_size".

2024-12-09

3 Bug fixes

16.30.0

Add the double data type for the plugin key value and the UI input box.

Details

The plugin framework lacked support for floating-point numbers, limiting developers to using generic types like string for configuration keys, which could lead to issues with type safety and precision. This, in turn, affected the user interface, making it difficult for end-users to configure the correct value. A solution was implemented by allowing developers to define plugin configuration keys with floating-point number types and supporting their configuration in the plugin UI. As a result, developers were able to use more specific double types to define plugin configuration keys, ensuring greater precision and better type safety, while also providing end-users with proper feedback to configure correct values within the plugin UI.

3 Bug fixes

2024-12-09

16.30.0

Add support for Forwarded header.

Details

There was no provision to read CF-Connecting-IP coming from Clouldflare, resulting in CF-Connecting-IP being ignored while calculating access restriction IPs. However, this issue has been resolved by adding a provision to read CF-Connecting-IP from the request header, allowing these IPs to now be used for access restriction.

3 Bug fixes

2024-12-09

16.30.0

'undefined' key when saving profile keys.

Details

The JavaScript had been using incorrect data when calling the server to save data, resulting in an exception. As a result, the JavaScript did not finish running and failed to update the page. However, this issue has been resolved by modifying the code to ensure correct data is sent to the server, which now successfully updates the profile page upon saving, accompanied by a displayed success message.

2 Minor improvements

2024-12-09

16.30.0

Plugin Utils UI did not support the key with space.

Details

The user was unable to provide whitespace as part of the plugin key parameter value. As a result, the UI displayed an input error indicating that the provided value was incorrect due to containing not allowed characters. To address this issue, validation on the plugin key parameter value has been removed, allowing any characters to be entered, including whitespace.

2 Minor improvements

2024-12-09

16.30.0

Enhance field validation in admin UI.

Details

There was no flexibility with creating resource names, which allowed only alphanumeric characters and underscores. As a result, validation errors were displayed under input fields, preventing progress with the creation of resources. To address this issue, flexibility has been added to resource name creation, allowing any text to be entered on search package/data source/results page UIs. This change now allows for any text to be provided as a resource name, with the ID being generated based on the name using only lowercase letters (a-z), numbers (0-9), and underscores (_) characters.

2 Minor improvements

2024-12-09

16.30.0

Accessibility Auditor: Document results page link incorrect if selected from domain.

Details

The domain parameter was not correctly set in the navigation URL (route), leading to scoping issues in the accessibility auditor. As a result, data was not being scoped to the correct domain and all data were returned. However, this issue has been resolved by setting the domain parameter correctly in the navigation URL (route). The consequence is now that scoping data in the accessibility auditor to the domain is working as expected.

3 Bug fixes

2024-12-09

16.30.0

Change ui.modern.max_facet_extra_searches permission.

Details

The configuration item ui.modern.max_facet_extra_searches was set to 0 for all collections as it was causing excessive use of CPU. As a result, some facets such as multi-select were not displaying counts, and users were unable to reset the configuration item. To address this issue, the execution of extra searches for facets has been disabled by default, allowing staff to increase the value to a maximum of 50 when required.

3 Bug fixes

2024-12-09

16.30.0

Fix 404 tooltip for external metadata config editor.

Details

A missing documentation file used in side help was discovered. As a result there was no quick help resource for the external metadata configuration file. To address this issue, the missing file has been added. As a consequence, users can now preview definitions and expected file formats for the external metadata configuration file, and access full documentation through a link.

3 Bug fixes

2024-12-09

16.30.0

Config key editor - key sorting is not working.

Details

Alphabetical sorting in a table by column was not working, making it impossible for users to rearrange the configuration keys. However, this issue has been resolved with the implementation of correct sorting in a table by column. As a result, users can now change the order of configuration keys.

3 Bug fixes

2024-12-09

16.30.0

Facet display error when multiple facet names of the same value.

Details

It was possible for users to create multiple facets with the same name in the UI, which is not allowed. Search endpoint supports only unique facets. To address this issue, validation has been added to the form to check if a facet name already exists. With this change, users can no longer create multiple facets with the same name in the UI.

3 Bug fixes

2024-12-06

16.28.0.3.4

Fixed the cloned URL syntax in Clone documents plugin when the original URL contains a query string.

1 New and revised features

2024-12-06

16.28.0.3.3

Added the Transform Date Field plugin to convert ambiguous date formats into ISO format supported in Funnelback.

1 New and revised features

2024-12-04

16.28.0.3.2

Extended the Map URL parameters plugin to support mapping of the request headers into input parameters.

3 Bug fixes

2024-11-22

16.28.0.3.1

Extended the Date filter plugin to fix a bug where date fields that contained leading or trailing whitespace could not be parsed.

3 Bug fixes

2024-11-21

16.28.0.3

Fixed the issue where setting the lang parameter via curator wasn’t processed the same as the CGI/query parameter.

1 New and revised features

2024-11-19

16.28.0.2.5

Extended the Clone documents plugin to support the cloning of documents based on repeated values in a field.

1 New and revised features

2024-11-13

16.28.0.2.4

Extended the Split XML or HTML plugin to support inclusion of meta tags for HTML documents, namespaced XML fields and all configuration fields in a configuration file.

  • Added support to include a source document’s <meta> tags in split documents when splitting an HTML document.

  • Added the support to process an XML file with namespaced XML fields.

  • Added the support to configure Default include HTML title for all split HTML documents and Default include HTML metadata for all split HTML documents in a configuration file.

  • Upgraded the XPath processing to use native Jsoup XPath support.

1 New and revised features

2024-11-06

16.28.0.2.3

Extended the Generate start URLs plugin to support HTTP basic authentication when downloading the start URLs.

1 New and revised features

2024-11-06

16.28.0.2.2

Extended the Access restriction to search results plugin with an option to allow unrestricted access to auto-completion and click tracking endpoints.

1 New and revised features

2024-11-06

16.28.0.2.1

Extended the Generate auto-completion from search index plugin to add an option, Collapsing signature, that allows a result collapsing signature to be applied to remove duplicate items when generating the auto-completion.

3 Bug fixes

2024-10-23

16.28.0.2

Fixed the issue where a data reload would occur with previous content in the configuration admin UI.

3 Bug fixes

2024-10-23

16.28.0.2

Fixed the issue where exceptions would occur for disabled plugins in a data source/results page configuration.

3 Bug fixes

2024-10-23

16.28.0.2

Fixed the issue of incremental crawl not following links from copied files.

3 Bug fixes

2024-08-27

16.28.0.1

Increased the allowable size of the response headers

3 Bug fixes

2024-08-27

16.28.0.1

Fixed the issue where the order of results returned by the /s/suggest.json endpoint may vary.

2 Minor improvements

2024-09-10

16.28.0.0.6

Fixed the issue in Generate auto-completion from search index plugin which caused punctuation characters to be included in the trigger terms.

1 New and revised features

2024-08-28

16.28.0.0.5

Added the Generate start URLs plugin to generate a list of start URLs for a data source based on a field containing a total number of pages, which can then be used to generate page numbers passed in a URL to the API.

1 New and revised features

2024-10-11

16.28.0.0.4

Extended the Modify the search result URL plugin to support defining multiple URL match patterns and reformatting templates.

1 New and revised features

2024-10-11

16.28.0.0.3

Added the new Clean text input plugin to remove non-printable or control characters from downloaded text content.

2 Minor improvements

2024-09-10

16.28.0.0.2

Added support for XSLT 2.0 and XSLT 3.0 within the Transform XML data plugin.

2 Minor improvements

2024-08-28

16.28.0.0.1

Extended the Map URL parameters plugin to support modification of extra searches.

1 New and revised features

2024-08-12

16.28

Moved storage of session cart and session history to browser local storage instead of database. Changed implementation method for session history (search and click history) from writing complete HTML content to only customizing templates in simple.ftl.

The current tracking mechanism stores session history in the database. This mechanism will soon be deprecated, and all session storage will be shifted to browser local storage. To keep functionality intact, it is recommended that you upgrade the session history template to the latest version.

Upgrade instructions for session history can be found here: * Upgrading a template with a default sessions and history widget * Upgrading a template with a customized sessions and history widget

3 Bug fixes

2024-08-12

16.28

Fixed issue with javascript scrolling for highlighted query in search page.

1 New and revised features

2024-08-08

16.26.0.0.5

Added the Exclude by content plugin to enable the definition of indexing exclude rules based on content.

1 New and revised features

2024-08-08

16.26.0.0.4

Added the Transform XML data plugin to apply a transformation to XML data before indexing.

3 Bug fixes

2024-08-07

16.26.0.0.3

Updated the Split XML or HTML plugin to fix a bug where the plugin was not generating valid XML.

3 Bug fixes

2024-07-17

16.26.0.0.2

Updated the Query language - wildcard (truncation) support plugin to fix a bug where sometimes no search results were returned.

2 Minor improvements

2024-06-20

16.26.0.0.1

Updated the Modify the search result URL plugin to add an option to also update the displayUrl field of the data model.

2 Minor improvements

2024-06-17

16.26

Extended the Funnelback DXP developer role dxp~developer to allow editing of configuration keys crawler.ignore_robots_txt and crawler.ignore_nofollow.

2 Minor improvements

2024-06-17

16.26

Fixed an issue preventing analytics email notifications from being delivered.

2 Minor improvements

2024-06-17

16.26

Added annotation in analytics UI that all reports are displayed in UTC.

3 Bug fixes

2024-06-17

16.26

Fixed as issue where the time after midnight for the last modified in logs UI was displayed as 24 instead of 00.

2 Minor improvements

2024-06-17

16.26

Suppressed invalid collection ID warnings in update/push logs.

3 Bug fixes

2024-06-17

16.26

Fixed an issue where invalid characters were acceptable in the plugin configuration UI when setting the configuration key with parameters.

2 Minor improvements

2024-06-17

16.26

Lowered the minimum allowed value from 2 to 0 for the auto-completion.length configuration key.

3 Bug fixes

2024-06-17

16.26

Fixed an issue where selecting multiple data sources as source for a single facet tab wasn’t working.

3 Bug fixes

2024-06-17

16.26

Fixed an issue where scrolling in accessibility auditor audit document UI didn’t work in Chrome browser.

1 New and revised features

2024-04-11

16.24.0.0.1

Updated "Tabs - Set default" plugin (v.1.2.0) to allow configuring default tab sourced from a metadata field.

1 New and revised features

2024-03-04

16.24

Added a plugin to gather and index content from the Acalog academic catalog management system.

1 New and revised features

2024-03-04

16.24

Added batch polling of the push data source status in a multi-server environment.

  • This change changes the following push data source configuration keys: push.replication.master.hostname, push.replication.master.push-api.port, push.replication.master.webdav.port which are now configured by a system administrator in the server configuration options.

  • Introduced push.replication.master.push-api.call.delay to set a delay before requesting new data from the master node.

2 Minor improvements

2024-03-04

16.24

Provided an option to publish a new plugin on creation.

2 Minor improvements

2024-03-04

16.24

Improved the filter chain configuration for plugins to display descriptive labels for the filters.

2 Minor improvements

2024-03-04

16.24

Improved the search within the plugins section to search over the documentation content.

2 Minor improvements

2024-03-04

16.24

Updated the plugin listings to display a notification when a newer plugin is available.

2 Minor improvements

2024-03-04

16.24

Improved the upgrade script to automatically upgrade plugins, where possible.

2 Minor improvements

2024-03-04

16.24

Improved management of search from the DXP console by integrating with the DXP metrics collection service.

3 Bug fixes

2024-03-04

16.24

Fixed an issue where the API token generation screen did not return all token details.

3 Bug fixes

2024-03-04

16.24

Fixed an issue where configuration was left in an unrecoverable state when publishing a new plugin.

4 Important changes

2024-03-04

16.24

Upgraded Angular to v16.

4 Important changes

2024-03-04

16.24

Upgraded Jsoup to v1.17.1.

4 Important changes

2024-03-04

16.24

Upgraded the search authentication module to integrate with the Squiz DXP IdP.

3 Bug fixes

2024-03-04

16.24

Fixed an issue where analytics emails ware not being sent.

3 Bug fixes

2024-06-17

16.24

Fixed an issue where filtering results by URL in accessibility auditor all document UI wasn’t working.

3 Bug fixes

2024-06-17

16.24

Fixed an issue where restoring files from backup in the configuration file manager UI would navigate to the not found page (404 error).

1 New and revised features

2024-03-04

16.24

Added option to adjust time zone for analytics charts.

3 Bug fixes

2024-03-04

16.24

Fixed an issue where plugin key configuration was throwing error for allowed value of type different than string.

3 Bug fixes

2024-03-04

16.24

Fixed an issue where plugin UI doesn’t show the filter chain for a plugin that enables Jsoup filters.

3 Bug fixes

2024-04-29

16.24.0.1

Modified email logic to use TLS1.2+ so it can connect to AWS SES.

3 Bug fixes

2024-04-29

16.24.0.1

Fixes the issue where adding documents sourced from push data source to cart won’t work.

3 Bug fixes

2024-06-07

16.24.0.1.1

Updated the Strip HTML markup from XML plugin to fix an issue where extra whitespaces would be added before closing XML tag.

2 Minor improvements

2024-01-12

16.22.0.1.1

Extended the Fetch external metadata from URL plugin to support downloading metadata from multiple URLs.

3 Bug fixes

2023-12-19

16.22.0.1

Fixed an issue where publishing a new plugin sometimes resulted in environment specific keys being incorrectly set, resulting in plugin configuration that was available only in the live view.

1 New and revised features

2023-11-30

16.22

Added the ability to kill a document of a list of documents returned by a query using plugins.

1 New and revised features

2023-12-04

16.22

Added the ability to define the Query Independent Evidence (QIE) to a given document or a list of documents returned by a query within a data source as part of a plugin’s functionality.

1 New and revised features

2023-12-04

16.22

Added sort options for use with result collapsing, to sort the collapsed result set.

1 New and revised features

2023-12-04

16.22

Added integration with the DXP metrics service to track client license usage.

2 Minor improvements

2023-12-04

16.22

Improved the debug API to obfuscate sensitive information and added a new security permission sec.api.debug to provide access to the feature. Developers within the DXP are now able to access the Debug API, which was previously restricted to administrator level users.

4 Important changes

2023-12-04

16.22

Existing plugins that are configured on searches were automatically upgraded to the latest supported version to unlock the new plugin management interface for existing searches.

4 Important changes

2023-12-04

16.22

Removed the real-time preview feature for best bets and curator.

3 Bug fixes

2023-11-20

16.20.0.3

Fixed an issue where plugin configuration keys containing sensitive content were incorrectly encrypted via plugin PUT API.

3 Bug fixes

2023-10-27

16.20.0.2

Fixed an issue where the default values of configuration keys aren’t populated on the data source creation screen.

3 Bug fixes

2023-10-24

16.20.0.1

Fixed an issue where some newer valid top level domains did not pass the external metadata validator.

1 New and revised features

2023-10-16

16.20

Added new administration screens for the setup and configuration of plugins.

1 New and revised features

2023-10-16

16.20

Added the ability to kill documents/URLs on the basis of search queries using a new configuration file (query-kill.cfg).

1 New and revised features

2023-10-16

16.20

Added a new plugin to gather content from an SFTP server.

1 New and revised features

2023-10-16

16.20

Added the ability to define query independent evidence weightings on the basis of search queries using a new configuration file (query-qie.cfg).

1 New and revised features

2023-10-16

16.20

Added validation of external metadata configuration files with a configurable error tolerance. Added an indexer option, -externalMetaErrorThreshold to set an error threshold for the handling of external metadata.

1 New and revised features

2023-10-16

16.20

Added support to extract URL links from non-HTML files.

2 Minor improvements

2023-10-16

16.20

Extended the plugin framework (and released Funnelback shared v16.20) to support the new plugin UI, to provide a new interface for setting start URLs and to use asciidoc formatted documentation.

2 Minor improvements

2023-10-16

16.20

Upgraded existing plugins to support the new plugin management interface. This results in a minor version bump to existing plugins (the plugins do not provide any new functionality).

3 Bug fixes

2023-10-16

16.20

Restored the ability to stop and clear the locks of a stuck or crashed update.

3 Bug fixes

2023-10-16

16.20

Fixed an issue with the delete client API that resulted in an HTTP 409 error being returned.

3 Bug fixes

2023-10-16

16.20

Fixed an issue with the session cart that caused it to incorrectly display no items in your shortlist after adding an item to the shortlist.

3 Bug fixes

2023-10-16

16.20

Fixed an issue with accessibility auditor that resulted in the unchecked documents report returning an error.

3 Bug fixes

2023-10-16

16.20

Fixed an issue with the Funnelback installer that resulted in incorrect filesystem group permissions being assigned.

4 Important changes

2023-10-16

16.20

Changed the Funnelback crawler default behaviour on the handling of canonical URLs. Pages that have a canonical URL defined are now re-fetched using the canonical URL.

In certain edge cases web content that specifies canonical URLs may not be gathered after an upgrade because the canonical URL isn’t covered by the web crawler include patterns.

When this happens the page will not be stored and this will result in the page not being included in the search index. In some cases it may result in the data source failing to update because there are no or fewer documents to store.

If you are affected by this change you can amend your include patterns to capture the canonical URL, or you can set crawler.ignore_canonical_links=true in your data source configuration to disable the new canonical URL behavior within the crawler.

1 New and revised features

2023-08-16

16.18.0.4.2

Extended the XML date filter plugin to support html and released this as a new date filter plugin. The xml date filter plugin is now deprecated. We recommend upgrading any data sources that use this plugin to use the new date filter plugin.

2 Minor improvements

2023-08-07

16.18.0.4.1

Extended the Instagram gatherer plugin to gather all available attributes, which can be mapped to metadata classes for use with the search.

3 Bug Fixes

2023-07-14

16.18.0.4

Fixed the issue where contradicting messages were displayed in the SEO auditor optimization tips. Prior to the update some words were shown as frequently occurring in one section whilst another section stated the same words cannot be found.

1 New and revised features

2023-06-28

16.18.0.3.2

Added a new plugin that allows you to strip HTML tags from XML document content.

1 New and revised features

2023-06-22

16.18.0.3.1

Extended the Split XML or HTML plugin to support the sourcing of split document URLs from one of the XML or HTML fields.

2 Minor improvements

2023-06-20

16.18.0.3

Improved the search package, data source and results page management screens to identify the type of item you are managing, and to make it easier to manage linked items.

3 Bug Fixes

2023-06-20

16.18.0.3

Prevented the creation of objects within Freemarker template files to ensure that template editors can not cause external code to be executed.

3 Bug Fixes

2023-06-20

16.18.0.3

Fixed a ranking issue caused by the incorrect calculation of the main hosts score.

1 New and revised features

2023-06-20

16.18.0.3

Added new global configuration keys to configure Jetty HTTP connection.

3 Bug Fixes

2023-05-31

16.18.0.2

Fixed an issue where search results were skewed by ranking option cool.23 'host_incoming_link_score'.

1 New and revised features

2023-05-19

16.18.0.1.4

Added a new plugin that allows content to be gathered from Twitter using the Twitter V2 API.

1 New and revised features

2023-05-19

16.18.0.1.3

Added a new plugin that allows videos to be indexed from the Vimeo social media platform.

1 New and revised features

2023-05-01

16.18.0.1.2

Added a new plugin, human readable document type that adds a human friendly document type to indexed content.

1 New and revised features

2023-05-01

16.18.0.1.1

Added a new plugin, access restriction to search results that allows you to control access to search results pages using a token.

3 Bug Fixes

2023-04-03

16.18.0.1

Fixed an issue with accessibility auditor which prevented quick access to the linked content auditor report.

3 Bug Fixes

2023-04-03

16.18.0.1

Fixed an issue with the calculation of main host scores that caused inaccurate and inconsistent information to be displayed in SEO auditor.

1 New and revised features

2023-03-20

16.18

Added a new data source configuration option qie.default_weight to change the default weighting set for query independent evidence.

1 New and revised features

2023-03-20

16.18

Added a new data source configuration option qie.default_weight to change the default weighting set for query independent evidence.

1 New and revised features

2023-03-20

16.18

Added a plugin to index content from a Google Calendar.

3 Bug fixes

2023-03-20

16.18

Fixed an issue where the insights dashboard quick search did not allow searches of the preview version of a results page.

3 Bug fixes

2023-03-20

16.18

Fixed an issue where content auditor incorrectly displayed empty panels on the overview report for metadata

3 Bug fixes

2023-03-20

16.18

Fixed an issue where SEO auditor did not show the link information for a document being audited in a search package.

3 Bug fixes

2023-03-20

16.18

Fixed an issue where content auditor anchor links were not being displayed correctly.

4 Important changes

2023-03-20

16.18

Decreased the maximum size for images processed by the /s/scale endpoint to limit the potential for server outages.

4 Important changes

2023-03-20

16.18

Updated the support package permissions to allow users with sec.support-package permission to read collection.cfg backup files via WebDAV.

4 Important changes

2023-03-20

16.18

Removed the requirement for an appid and secret when creating a new Facebook data source. A valid access token is now the only requirement when creating a new Facebook data source.

1 New and revised features

2023-02-13

16.16

Added new curator triggers: the number of search results, modify extra search run, no query

This includes:

  • the number of search results trigger activates when comparison of a number of results returned by search with the expected one is true.

  • modify extra search run trigger activates when extra search is enabled with specified ID.

  • no query trigger activates if the search is submitted without providing any query terms - this includes null, empty string or a whitespace query.

1 New and revised features

2023-02-13

16.16

Added new curator actions: add URL parameters, disable extra search run, remove URL parameters, set sorting options, select facet category, deselect facet category

This includes:

  • add URL parameters action allows adding URL parameters to a request before executing search.

  • disable extra search run action allows disabling specified extra search run.

  • remove URL parameters action allows removing URL parameters from a request before executing search.

  • set sorting options action allows user to specify how the results are sorted.

  • select facet category action allows adding facet category to URL parameters.

  • deselect facet category action allows removing facet category from URL parameters.

2 Minor improvements

2023-02-13

16.16

Updated the menu presented on Funnelback administration screens to provide a more consistent DXP experience. The insights dashboard now restricts the view to results pages for the current client and provides the ability to switch between clients.

2 Minor improvements

2023-02-13

16.16

Return a 405 HTTP status code when accessing /s/search.* or /s/all-results.* endpoints using an unsupported request method. (HTTP GET is the only supported request method).

2 Minor improvements

2023-02-13

16.16

Extended list of string matching options for curator rules by adding string starts with and ends with (both case-sensitive and case-insensitive variants).

3 Bug fixes

2023-02-13

16.16

Fixed an issue affecting the annotation and spelling index build that resulted in confusing errors being written to the update logs.

1 New and revised features

2023-02-13

16.16

Added an indexer option, canon.anchor_collapse to control the behavior of anchor collapsing in URL canonicalization.

1 New and revised features

2023-02-13

16.16

Added the ability to set system network device IP address ranges to be ignored during the processing of access restriction by IP address.

1 New and revised features

2023-02-13

16.16

Added new Force HTML Mime Type filter to the set of built in filters.

2 Minor improvements

2023-02-13

16.16

Improved the accessibility auditor H44 check to skip hidden elements when looking for <label>.

1 New and revised features

2022-01-31

16.14.0.6.1

Added a plugin to maps URL parameter to alternate data model input parameters when a query runs.

3 Bug fixes

2022-12-21

16.14.0.6

Fixed an issue where the all-results endpoint would not accept system queries.

3 Bug fixes

2022-12-14

16.14.0.5

Removed unused Honeycomb tooling that consumes resources and fills up a log file with noise.

3 Bug fixes

2022-12-08

16.14.0.4

Fixed an issue where the recommender wouldn’t return results if accessed on search HTTPS or admin port.

3 Bug fixes

2022-12-08

16.14.0.4

Fixed an issue where the SEO Auditor would display an error if the search package contained data sources which had no results or had not been updated at least once.

1 New and revised features

2022-11-22

16.14.0.3.2

Added a plugin to index content from the Instagram social media platform.

1 New and revised features

2022-11-22

16.14.0.3.1

Added a plugin to split XML and HTML documents. This plugin adds XML splitting support for push data sources and other improvements to the built-in splitter.

3 Bug fixes

2022-11-15

16.14.0.3

Fixed an issue where Freemarker templates would return an error if no query string parameters were returned by the request URL.

3 Bug fixes

2022-11-15

16.14.0.3

Fixed an issue where the all-results endpoint would return a 500 error if no query parameter was provided.

3 Bug fixes

2022-11-03

16.14.0.2

Recompiled and statically linked funnelback-outliers with libstdc++ so that it can run properly on Centos 7 systems.

3 Bug fixes

2022-10-26

16.14.0.1

Fixed an issue where the Perl script to trigger analytics update didn’t set the correct update type.

3 Bug fixes

2022-10-26

16.14.0.1

Fixed an issue where the date range picker in the search analytics UI didn’t work correctly.

1 New and revised features

2022-10-24

16.14

Extend the plugin framework to allow plugins to update their persistent configuration settings. This enables plugins to maintain state between updates, and to update configuration settings such as API tokens that require a periodic refresh.

1 New and revised features

2022-10-24

16.14

Added the ability to administer results page configuration files from the administration dashboard.

1 New and revised features

2022-10-24

16.14

Improved the file manager so that it does not prevent the administration dashboard from being used while uploading a file.

2 Minor improvements

2022-10-24

16.14

Added a log file (audit.push.log) that captures audit messages for the push API.

3 Bug fixes

2022-10-24

16.14

Fixed an issue in the file manager where some backup files could not be accessed.

4 Important changes

2022-10-24

16.14

Exposed additional data source configuration files via WebDAV and the file manager.

This includes:

  • metadata_scraper.json Note: user requires edit permission for the filter.jsoup.classes or filter.classes configuration keys to edit a file and read access to keys to read the file.

  • md_normaliser.*.mapping Note: user requires edit permission for the filter.md_normaliser.keys configuration key to edit a file and read access to the key to read the file.

  • predirects.cfg: Note: user requires edit permission for the crawler.predirects_enabled configuration key to edit a file and read access to the key to read the file.

  • undesirable-text.*.cfg Note: user requires edit permission for the filter.jsoup.undesirable_text-source.* configuration key to edit a file and read access to the key to read the file.

3 Bug fixes

2022-10-10

16.12.0.5

Fixed an issue where the environment was not detected correctly in the configuration, resulting in configuration key values to revert to the default value.

3 Bug fixes

2022-10-10

16.12.0.5

Fixed an issue with client-based APIs where incorrect permissions were returned.

3 Bug fixes

2022-10-10

16.12.0.5

Fixed an issue with the IncludeUrl Freemarker macro that prevented some relative links from being converted to absolute links.

Added a new option convertRelativeRequiresSpace to the Freemarker IncludeUrl macro that expects a space between HTML attributes while converting relative URLs into absolute ones.

By default, the option is enabled. To revert to the former behavior, set the convertRelativeRequiresSpace parameter to false.

The extraction of relative links now follows the W3C standard regarding the validity of HTML pages.

3 Bug fixes

2022-08-23

16.12.0.4

Fixed an issue that prevented internal documentation within the administration dashboard from displaying in some circumstances.

3 Bug fixes

2022-08-22

16.12.0.3

Fixed an issue that caused administration dashboard labels to display intermittently.

3 Bug fixes

2022-08-15

16.12.0.2

Fixed an issue where the Freemarker template upgrader incorrectly upgraded custom variables named metaData.

3 Bug fixes

2022-08-15

16.12.0.2

Fixed an issue where trend alerts notifications were not generated.

3 Bug fixes

2022-08-08

16.12.0.1

Fixed an issue where the upgrade utility wouldn’t always set correct file access permissions depending on the user executing the script.

3 Bug fixes

2022-08-08

16.12.0.1

Fixed an issue preventing the correct export of content auditor and accessibility auditor documents in CSV format.

1 New and revised features

2022-07-22

16.12

Added an indexer option, -ignore_noindex, which allows indexing of pages, even when there’s a noindex robots meta tag.

1 New and revised features

2022-07-22

16.12

Upgrade utility wrapper script logs console output to file in addition to displaying on console.

2 Minor improvements

2022-07-22

16.12

Added support for new DXP user roles.

2 Minor improvements

2022-07-22

16.12

Replaced the extra search configuration file with corresponding configuration keys.

2 Minor improvements

2022-07-22

16.12

Extended the plugin framework to enable the use of encrypted plugin keys within the server configuration.

2 Minor improvements

2022-07-22

16.12

Added an interactive mode for the upgrade utility.

2 Minor improvements

2022-07-22

16.12

Added a log file that captures the output of the upgrade utility.

2 Minor improvements

2022-07-22

16.12

Provided basic documentation on how to provide a stemming blacklist.

2 Minor improvements

2022-07-22

16.12

Removed unused configuration file curator-advanced.json.

Note, for any implementation still using this file, its configuration should be moved to Curator.

2 Minor improvements

2022-07-22

16.12

Improved the ability to exclude data folder components when using the upgrade utility.

3 Bug fixes

2022-07-22

16.12

Fixed a bug that allowed the incorrect scheduling of unsupported types of data source updates for certain data sources.

3 Bug fixes

2022-07-22

16.12

Fixed a bug where the wrong HTTP status code was sometimes returned for pages after an administration user’s session expired.

3 Bug fixes

2022-07-22

16.12

Fixed a bug where the wrong HTTP status code was sometimes returned for invalid request parameters.

3 Bug fixes

2022-07-22

16.12

Fixed a bug in the faceted navigation editor that prevented gscope based categories from being added as tabs.

3 Bug fixes

2022-07-22

16.12

Fixed a bug where a user was sometimes redirected to a default page instead of the page they were currently on after logging back in after their login session expired.

3 Bug fixes

2022-07-22

16.12

Fixed a bug in the metadata mapping API where metadata mappings were not returned for search packages.

3 Bug fixes

2022-07-22

16.12

Improved the configuration upgrade tool (update-configs.pl) to prevent default values from being inserted into the collection configuration on upgrade.

3 Bug fixes

2022-07-22

16.12

Added the ability to restore configuration file backups on the file management screen.

4 Important changes

2022-07-22

16.12

Modified the logic that retrieves metadata for search package collections to ensure the list of metadata classes returned includes the metadata classes of the component data source collections. This changes the response from the metadata-mapping endpoint of the Admin API.

4 Important changes

2022-07-22

16.12

Exposed results page configuration files via WebDAV.

This includes:

  • reporting-stop-words.cfg

  • auto-completion.csv

  • blending.cfg

  • curator.json Note: this enables bulk uploading of curator rules. Warning: there is no validation of the file so you must ensure it is correctly formatted or it may cause an error in the curator editor, and prevent curator rules from working. The curator editor within the administration dashboard should be used under normal circumstances.

  • faceted_navigation.cfg Note: this enables bulk uploading of faceted navigation configuration. Warning: there is no validation of the file so you must ensure it is correctly formatted or it may cause an error in the faceted navigation editor, and prevent faceted navigation from working. The faceted navigation editor within the administration dashboard should be used under normal circumstances.

  • synonyms.cfg Note: this enables bulk uploading of synonyms configuration. Warning: there is no validation of the file so you must ensure it is correctly formatted or it may cause an error in the synonyms editor, and prevent synonyms from working. The synonyms editor within the administration dashboard should be used under normal circumstances.

  • template.xsl

  • ui.*.cfg

4 Important changes

2022-07-22

16.12

Introduced new permission sec.auto-completion allowing modify results page configuration file auto-completion.csv via WebDAV.

Note, permission was granted to default implementer, support and administrator roles available within the product.

4 Important changes

2022-07-22

16.12

Upgrade utility wrapper did not have executable permissions. This has been fixed.

The upgrade utility wrapper upgrade-v15-to-v16 should be able to be run and execute the upgrade utility.

3 Bug fixes

2022-07-11

16.10.0.4

Fixed an issue where adding a new category to a tab or custom facet via administration dashboard was prevented in some conditions.

3 Bug fixes

2022-06-17

16.10.0.3

Fixed an issue where an invalid profile value provided in the search request returned a 500 server error.

3 Bug fixes

2022-06-17

16.10.0.3

Fixed an issue where unknown product configuration key values are not expanded correctly for the $COLLECTION_NAME variable.

3 Bug fixes

2022-05-26

16.10.0.2

Fixed an issue where fetching Facebook comments would cause an infinite loop due to changes within the Facebook endpoints.

3 Bug fixes

2022-05-13

16.10.0.1

Fixed a security vulnerability where com.google.oauth-client hasn’t implemented PKCE support in accordance with the RFC for OAuth 2.0 for Native Apps.

1 New and revised features

2022-05-04

16.10

Added option to upgrade utility to support importing resources in bulk by defining an affix.

1 New and revised features

2022-05-04

16.10

Provided a simple command line alias for running the upgrade utility.

1 New and revised features

2022-05-04

16.10

Provided a confirmation step before executing the upgrade utility command.

1 New and revised features

2022-05-04

16.10

Improved the upgrade utility so that configuration files that don’t apply to the current collection type (e.g. data sources, search packages) are now removed.

1 New and revised features

2022-05-04

16.10

Improved logic for the upgrading of collection IDs in existing configuration.

This includes:

  • upgrading meta collection index.sdinfo into configuration keys meta.components.*.weight with correct collection IDs

  • upgrading collection IDs in the start URLs list

  • detecting usage of collection references in templates and scripts

1 New and revised features

2022-05-04

16.10

Improved the upgrade utility by adding detection of using deprecated functionality.

This includes:

  • detecting usage of scripts (hook scripts, workflow scripts, custom filters, custom gatherers etc.)

  • detecting usage of pre-v15.12 legacy facets and removed data model features in templates

  • detecting usage of collection references in templates and scripts

1 New and revised features

2022-05-04

16.10

Fixed the upgrade utility handling of invalid component collections of a meta collection so that invalid collections are removed from the upgraded search package.

2 Minor improvements

2022-05-04

16.10

Updated the default metadata mappings to detect descriptions added to Microsoft Word metadata.

2 Minor improvements

2022-05-04

16.10

Improved the plugin documentation template that is created when generating a new plugin using the Maven archetype.

3 Bug fixes

2022-05-04

16.10

Fixed a bug where scheduled analytics updates were not correctly set for collections upgraded from v15.

3 Bug fixes

2022-05-04

16.10

Exposed reporting stop words configuration file via WebDav API for search packages and results pages.

3 Bug fixes

2022-05-04

16.10

Fixed a bug in the administration dashboard where the redirects.txt log file was not accessible from the log viewer.

4 Important changes

2022-05-04

16.10

The default version of Handlebars used in the default template and knowledge graph widget preview has been upgraded to 4.7.7.

4 Important changes

2022-05-04

16.10

The default version of jQuery used with auto-completion and cache highlights widgets has been upgraded to 3.6.0.

4 Important changes

2022-05-04

16.10

Upgraded the Angular version to 12.2.15.

1 New and revised features

2021-12-16

16.8.3.3

The following plugins have been added:

1 New and revised features

2021-12-16

16.8.3.2

The following plugins have been added:

1 New and revised features

2021-12-16

16.8.3.1

The following plugins have been added:

3 Bug fixes

2021-12-16

16.8.3

Removed broken administration dashboard used to configure reporting email as since v16 those settings are configured via results page configuration UI.

3 Bug fixes

2021-12-16

16.8.3

Upgrades log4j2 to version 2.17.1 to fix the security vulnerability where Apache Log4j2 Thread Context Lookup Pattern is vulnerable to remote code execution in certain non-default configurations.

3 Bug fixes

2021-12-16

16.8.1

Upgrades log4j2 to version 2.15 to fix the security vulnerability where log4j2 JNDI features do not protect against attacker-controlled LDAP and other JNDI related endpoints.

1 New and revised features

2021-12-07

16.8

The following plugins have been added:

1 New and revised features

2021-12-07

16.8

Extended the metadata scraper filter to support a number of content checking rules.

1 New and revised features

2021-12-07

16.8

Extended the undesirable text filter to support separated or combined undesirable text sources and individual values provided via configuration keys.

1 New and revised features

2021-12-07

16.8

Extended the metadata normalizer filter to support XML documents.

1 New and revised features

2021-12-07

16.8

Added CGI parameters that enable a user to search for results containing a specific metadata class (set to any value), or results that are missing a specified metadata class.

3 Bug fixes

2021-12-07

16.8

Fixed the positioning of the live chat widget so that it does not obscure other functionality.

3 Bug fixes

2021-12-07

16.8

Fixed a bug in the upgrade utility where it did not correctly handle environment specific configuration keys.

3 Bug fixes

2021-12-07

16.8

Fixed an issue where the collection tool would return an error for the index presence check.

3 Bug fixes

2021-12-07

16.8

Restored access to data reports from the administration dashboard.

3 Bug fixes

2021-12-07

16.8

Fixed an issue which resulted in some URLs being incorrectly handled in SEO auditor due to problems with URL encoding.

3 Bug fixes

2021-12-07

16.8

Fixed an issue that caused counts for metadata on the metadata mapping screen to be incorrect or marked as not found.

4 Important changes

2021-12-07

16.8

The following plugins will automatically be upgraded due to upgrades to underlying dependencies.

  • Auto-completion 1.0.0-2.0.2 will be upgraded to 2.0.3

  • Combine metadata 1.0.0 will be upgraded to 1.0.1

  • Query - wildcard support 1.0.0 will be upgraded to 1.0.1

4 Important changes

2021-12-07

16.8

The default behaviour of the metadata scraper filter has changed between versions when using regex matching. An option includeFirstCaptureGroup can be set to true to restore the former behavior.

4 Important changes

2021-12-07

16.8

Upgraded Freemarker to 2.3.31.

3 Bug fixes

2021-09-07

16.6.2

Fixed a bug in the auto-completion plugin where summaries were truncated to 1 character. This issue is fixed in the auto-completion plugin v2.0.2

1 New and revised features

2021-09-07

16.6.2

The following plugins have been added:

1 New and revised features

2021-09-07

16.6.1

The following plugins have been added:

1 New and revised features

2021-09-07

16.6

The following plugins have been added:

1 New and revised features

2021-09-07

16.6

Extended the plugin framework to support:

1 New and revised features

2021-09-07

16.6

Release of the program finder and local government stencils.

1 New and revised features

2021-09-07

16.6

Extended the metadata scraper filter to support XML.

1 New and revised features

2021-09-07

16.6

Added support within the Squiz DXP for the management and editing of configuration, template and web resources using a WebDAV client.

1 New and revised features

2021-09-07

16.6

Added support for basic HTTP authentication with an API token when using the Funnelback APIs.

1 New and revised features

2021-09-07

16.6

Added support for result page thumbnails to be set from the administration and marketing dashboard.

1 New and revised features

2021-09-07

16.6

Improved WebDAV support for Windows Explorer and MacOS Finder.

2 Minor improvements

2021-09-07

16.6

Improved the task scheduler listing screen to enable additional tasks to be scheduled.

2 Minor improvements

2021-09-07

16.6

Added support for a custom HTTP header x-funnelback-request-detailed-logging which can be supplied with a HTTP request to increase the modern UI log level for the duration of the request.

2 Minor improvements

2021-09-07

16.6

Added support for auto-completion to be called on preview versions of results pages.

3 Bug fixes

2021-09-07

16.6

Fixed a bug in the configuration/template files UI where in incorrectly reported that there were unpublished files.

3 Bug fixes

2021-09-07

16.6

Fixed a bug in the raw configuration editor that prevented some configuration keys from being saved.

3 Bug fixes

2021-09-07

16.6

Fixed a bug in the YouTube gatherer that was triggered by an empty playlist ID.

3 Bug fixes

2021-09-07

16.6

Fixed a timeout issue that prevented data sources and search packages from being created correctly under certain conditions.

3 Bug fixes

2021-09-07

16.6

Restored global domain redirects (redirects.cfg) functionality.

3 Bug fixes

2021-09-07

16.6

Fixed a bug to enable search package and data source management API functions to work correctly with query processors which run in read-only mode.

3 Bug fixes

2021-09-07

16.6

Fixed a bug preventing the publication of configuration files when using the perl file manager.

4 Important changes

2021-09-07

16.6

The following features have been deprecated:

4 Important changes

2021-09-07

16.6

Added the following new permissions:

  • sec.administer.read-only.override to support search package and data source management in multi-server environments.

4 Important changes

2021-09-07

16.6

Added or updated the following API calls

Updated the following API calls to v2. These APIs now return additional username and password fields that can be used to authenticate (via HTTP basic authentication) with the Funnelback APIs. The corresponding v1 calls are now deprecated. v1 tokens will continue to work but require regeneration to use HTTP Basic Authentication.

  • /application-tokens/v2/application-token

  • /application-tokens/v2/clients/<CLIENT-ID>/application-token/<TOKEN-NAME>

2 Minor improvements

2021-06-16

16.4

Improved the security of the download support package feature and enabled users of the Squiz DXP to access this function.

1 New and revised features

2021-06-16

16.4

Added administration screens for scheduling of data source and analytics updates.

1 New and revised features

2021-06-16

16.4

Added WCAG 2.1 support to accessibility auditor.

The following checks have been added:

  • Technique G208

  • Technique G211

  • Technique G216

  • Technique F107

2 Minor improvements

2021-06-16

16.4

The new workflow.publish_hook.batch allows for batching up configuration changes before publishing to query processors to reduce network load.

2 Minor improvements

2021-06-16

16.4

Added an option to view the details of configuration keys when the system is running in read only mode.

2 Minor improvements

2021-06-16

16.4

Increased the summary text limit (indexer option RSTXT) to 10000000 bytes.

2 Minor improvements

2021-06-16

16.4

Revised the top level structure of the documentation to make it easier for users to find relevant information.

2 Minor improvements

2021-06-16

16.4

Added a new API for getting the licence usage of a client.

3 Bug fixes

2021-06-16

16.4

Fixed a bug in the web crawler where base href tags were not processed correctly.

3 Bug fixes

2021-06-16

16.4

Fixed a bug where instant updates would always include the start URLs.

3 Bug fixes

2021-06-16

16.4

Fixed a bug where YouTube data sources could not be updated without a channel ID.

3 Bug fixes

2021-06-16

16.4

Fixed an issue with the document title fixer filter that caused it to insert bad titles sourced from h1 and h2 tags.

3 Bug fixes

2021-06-16

16.4

Fixed an issue where the faceted navigation editor screen became unresponsive when there were too many templates to check for legacy facets.

4 Important changes

2021-06-16

16.4

The push API no longer returns a HTTP 302 status code when SAML is enabled. A HTTP 401 status code will be returned to unauthenticated users. Users should authenticate using API tokens rather than directly when using a web browser. When using a web browser to access push API endpoints you should first authenticate using the admin API.

1 New and revised features

2021-03-31

16.2

Added a new upgrade utility to support the migration and upgrade of collections from Funnelback v15.24 to v16.

Upgrading from Funnelback 15 and earlier into Funnelback 16 is complicated due to a number of architectural changes within Funnelback.

The upgrade utility:

  • automates many of the tasks required to perform the upgrade.

  • can transfer the search from a v15 server. This includes configuration, data and users.

1 New and revised features

2021-03-31

16.2

Added a new configuration file, v15collectionremap.cfg, to support the mapping of v15.24 and earlier style collection IDs to v16 search package/data source IDs for public search and push API endpoints.

1 New and revised features

2021-03-31

16.2

Added various API calls to support the upgrade tool.

4 Important changes

2021-03-31

16.2

Collection scheduled updates times are not preserved when upgrading from 15.24 and earlier.

These must be recreated for all the data sources by applying configuration keys to each data source.

See: scheduling data source updates. In addition, if running and in-place upgrade (running the v16 installer in upgrade mode on an older Funnelback server) any pre-existing cron jobs for collection updates will need to be manually removed by editing the search user’s crontab.

4 Important changes

2021-03-31

16.2

Collection ID format change when upgrading from v15.24

There is a breaking change for collection IDs when upgrading from v15.24 and earlier. This will break existing integrations with Funnelback.

System administrators can configure collection remapping using the v15collectionremap.cfg file to partially mitigate these issues.

4 Important changes

2021-03-31

16.2

v15 default roles are assigned to a default client on upgrade from Funnelback 15.

Default roles will be prefixed with the default_roles internal client ID when upgrading from v15.

e.g. the v15 role default-administrator is upgraded to default_roles~default-administrator in v16.

4 Important changes

2021-03-31

16.2

The Funnelback installer now automatically rebuilds indexes after an upgrade.

In Funnelback 15, this was a step that had to be manually triggered when an administrator logged into Funnelback after an upgrade.

As a result search indexes will be unavailable until the reindexing process is complete.

4 Important changes

2021-03-31

16.2

Upgrading to v16 may affect ranking on any collections which were not of the web or meta types due to the automatic wrapping of such collections in a meta collection/search-package. Explicitly setting -SSS=0 as a query processor option on any affected search package will restore the previous ranking behaviour for non-web, non-meta collections.

1 New and revised features

2021-03-15

16.1.2108

Added support for Funnelback for higher education in the Squiz DXP.

The v16 release of Funnelback for higher education includes a tool for performing an initial setup of the package.

The setup tool is currently only available for use by Squiz implementation teams.

1 New and revised features

2021-03-15

16.1.2108

Added an auto-completion plugin which generates structured auto-completion to be generated from an existing Funnelback index.

The auto-completion plugin enables structured auto-completion to be generated by:

  • Setting a number of configuration options that define the trigger and action

  • Configuring the set of metadata fields to include in the auto-completion JSON response

1 New and revised features

2021-03-15

16.1.2108

Added a new enhanced update task scheduler.

The new update task scheduler enables updates to be configured by a search administrator with schedules based on time between updates, or scheduled at a fixed time.

The scheduler replaces the OS scheduler (cron) and existing scheduled updates must be manually set up in the new scheduler.

1 New and revised features

2021-03-15

16.1.2108

Added the ability to log in to the administration dashboard using a simple user ID that omits the client ID. This enables login with an email addresses as the ID if it has been used for the username when setting up the account.

1 New and revised features

2021-03-15

16.1.2108

Added support for a global plugin to validate all content updates added to the files via the admin-api.

An example use case for this is to provide virus scanning of content that is submitted via the administration interface (or via the API).

1 New and revised features

2021-03-15

16.1.2108

Added a new screen to the administration dashboard for the management of client application tokens.

A new user permission, sec.application-token.non-expiring.create, is required for access to this screen and the underlying API.

1 New and revised features

2021-03-15

16.1.2108

Added a new API (services/v1/services/{service}/thumbnail) for the management of thumbnails used for results pages.

1 New and revised features

2021-03-15

16.1.2108

Added a new API (/plugins/v1/plugins/version/{version}/debug/run-on-index) that enables a user to debug a plugin to see the result of running the plugin on an index.

1 New and revised features

2021-03-15

16.1.2108

Added a new API (/config/v2/server/keys/intercom.id) and configuration option (intercom.id) for configuring the Intercom ID. Users require the sec.server.config and edit.key.intercom.id permissions to modify the value.

1 New and revised features

2021-03-15

16.1.2108

Added a new API (/clients/v1/clients) for creating clients. A new user permission sec.clients.create has also been added to control access to this API.

1 New and revised features

2021-03-15

16.1.2108

Added a new API (/internal/clients/{client-id}/collections/{collection-id}) for adding a collection (search package or data source) to a client.

2 Minor improvements

2021-03-15

16.1.2108

Updated the push API (/v1/collections/{collection}/restore/from-remote) to support v15 remote servers.

2 Minor improvements

2021-03-15

16.1.2108

Search packages analytics updates are now automatically scheduled when they are created.

2 Minor improvements

2021-03-15

16.1.2108

Updated the client~primary administration role so that administration users with this role are able to grant the client~primary role to other administration users or tokens.

2 Minor improvements

2021-03-15

16.1.2108

Added support for system administrators to define range restrictions on configuration keys with a duration type value.

3 Bug fixes

2021-03-15

16.1.2108

Fixed an issue with accessibility auditor that caused the user interface to break when the user’s web browser language was not set to English.

3 Bug fixes

2021-03-15

16.1.2108

Fixed an issue with the web crawler that caused it to ignore the http_source_host configuration option.

3 Bug fixes

2021-03-15

16.1.2108

Fixed an issue with best bets where it failed to remove a search result that had the same URL as the best bet URL when the removing matching search result option was selected.

3 Bug fixes

2021-03-15

16.1.2108

Fixed an issue with the web crawler form interaction feature that caused it to incorrectly handle form parameters with empty values.

3 Bug fixes

2021-03-15

16.1.2108

Fixed an issue that caused invalid XML to be returned when the query contained a vertical tab character.

3 Bug fixes

2021-03-15

16.1.2108

Fixed an XXE vulnerability in cached documents that allowed files to be read from disk.

4 Important changes

2021-03-15

16.1.2108

Updated the default configuration to enable accessibility auditor by default. Accessibility auditor can be disabled on a data source by setting the accessibility-auditor.check configuration option to false.

4 Important changes

2021-03-15

16.1.2108

The outliers (trend alerts) cron job is no longer able to be scheduled via the administration dashboard. A system administrator can update or delete this scheduled update by editing the search user’s crontab.

4 Important changes

2021-03-15

16.1.2108

Legacy best bets (from v14.2 and earlier) have been removed.

This should not be confused with, and does not affect, the v15.0 and newer implementation of best bets (which is managed via the marketing dashboard).

The removal of legacy best bets has the following implications:

  • best_bets.cfg is no longer supported. Existing best_bets.cfg files will now be ignored and should be removed.

  • The response.resultPacket.bestBets element has been removed from the search data model

  • The s.BestBets macro has been removed from the funnelback_classic Freemarker library.

  • The best bets auto-completion source supported by build_autoc has been removed.

  • Best bets related options have been removed from padre.

Any legacy best bets should be manually re-entered via the best bets editor located within the marketing dashboard.

4 Important changes

2021-03-15

16.1.2108

Funnelback no longer supports additional administration service providers or single logout (SLO) when using SAML authentication.

Existing SAML configurations and the Groovy permission mapper may need to be updated.

1 New and revised features

2021-01-04

16.1.2041

Added support for client API access tokens, which enabled applications and scripts to access the admin API using a token that is independent of specific administration users.

2 Minor improvements

2021-01-04

16.1.2041

Added warnings to the administration dashboard when license limits are exceeded.

2 Minor improvements

2021-01-04

16.1.2041

Improved the admin API’s JSON representation of timezone and other date/time related config settings.

2 Minor improvements

2021-01-04

16.1.2041

Introduced support for higher-performance zstd compression in push document storage.

2 Minor improvements

2021-01-04

16.1.2041

Improved presentation of errors from SAML user mapper scripts.

2 Minor improvements

2021-01-04

16.1.2041

Improved performance of administration dashboard pages showing large numbers of data sources.

2 Minor improvements

2021-01-04

16.1.2041

Improved administration dashboard messaging when Funnelback server is in read only mode.

2 Minor improvements

2021-01-04

16.1.2041

Eliminated the use of jsessionid cookies when using SAML authentication.

2 Minor improvements

2021-01-04

16.1.2041

Product documentation links now refer to the hosted documentation which can be updated when errors are identified.

2 Minor improvements

2021-01-04

16.1.2041

Added user interface for enabling plugins.

2 Minor improvements

2021-01-04

16.1.2041

Improved administration dashboard breadcrumbs in a number of areas.

2 Minor improvements

2021-01-04

16.1.2041

Added client application tokens, an improvement for application tokens, suitable for use on the Squiz DXP

2 Minor improvements

2021-01-04

16.1.2041

Added plugin servlet filter hooks, a replacement for custom servlet filter hooks, suitable for use on the Squiz DXP

3 Bug fixes

2021-01-04

16.1.2041

Fixed handling of missing originalQuery entries in search query history.

3 Bug fixes

2021-01-04

16.1.2041

Fixed preservation of URL parameters through the SAML login flow.

3 Bug fixes

2021-01-04

16.1.2041

Fixed broken funnelback_documentation search result template.

3 Bug fixes

2021-01-04

16.1.2041

Fixed support for access_restriction settings based on hostnames when Funnelback is deployed behind a load balancer.

3 Bug fixes

2021-01-04

16.1.2041

Fixed accessibility auditor to account for the removal of the legacy facet data model.

3 Bug fixes

2021-01-04

16.1.2041

Fixed display of default web collection file type restrictions.

3 Bug fixes

2021-01-04

16.1.2041

Fixed return key handling when adding web data source URLs.

3 Bug fixes

2021-01-04

16.1.2041

Fixed a number of cases of old 'collection' terminology usage within administration dashboard.

4 Important changes

2021-01-04

16.1.2041

The legacy metaData section of the result data model has been removed. listMetadata should now be used instead.

1 New and revised features

2020-08-31

16.1.2034

Introduced screens for managing plugins and enabling/disabling them on search packages and result pages.

2 Minor improvements

2020-08-31

16.1.2034

Added instant-update and push logs to data source log viewing and WebDAV.

2 Minor improvements

2020-08-31

16.1.2034

Analytics email settings are now configured in result page config instead of a dedicated file.

2 Minor improvements

2020-08-31

16.1.2034

Improved the performance of the tuning dataset administration dashboard when many entries are added.

2 Minor improvements

2020-08-31

16.1.2034

Added a message to the administration dashboard when the server is set to read-only mode.

2 Minor improvements

2020-08-31

16.1.2034

Simplified the search transaction data model by removing some deprecated sections.

3 Bug fixes

2020-08-31

16.1.2034

Fixed handling of profiles containing hyphens in the legacy perl CGI administration pages.

3 Bug fixes

2020-08-31

16.1.2034

Fixed handling of permissions within some areas of the administration dashboard to disable inaccessible options.

3 Bug fixes

2020-08-31

16.1.2034

Accounted for the new client ID requirement within LDAP authentication.

4 Important changes

2020-08-31

16.1.2034

The removal of some data model elements to simplify its usage may cause some freemarker templates, plugins and hook scripts to fail when they reference the removed elements.

These implementation components must be updated to work with this and future versions of Funnelback. Specific data model elements removed include:

  • The legacy facet/categories.

  • The duplicate resultsWithTierBars section.

  • The rawInputParameters and inputParameterMap sections - inputParameters should now be used instead.

See: Data model changes in Funnelback v16

1 New and revised features

2020-08-11

16.1.2032

New administration dashboard, terminology and client separation

Funnelback’s administration dashboard has been overhauled to make search implementation easier.

As part of this improvement, new terminology of search packages, data sources and results pages has been introduced. Broadly, search packages are equivalent to meta collections in earlier versions of Funnelback, and a search package is now required to bundle up the data sources and results pages used to deliver a search. Data sources are equivalent to non-meta collections in earlier versions (but without the ability to serve search results), while result pages are the new equivalent of profiles and are used to define the search results page functionality and formatting.

The new administration dashboard also formalises the separation of implementations in a multi-tenant Funnelback environment with the concept of a client, which groups together all components of an implementation.

1 New and revised features

2020-08-11

16.1.2032

Plugins for reusable custom code

A new bundled format for custom code to be run on the Funnelback server, known as a plugin, has been introduced to the product.

Plugins are intended to separate customisations from individual data sources and results pages so that they can be more easily reused and can go through a separate approval process to ensure quality in multi-tenant environments.

Plugins provide similar capabilities to custom Groovy scripts in earlier versions and add the ability to supply some types of data, such as external metadata, dynamically during a data sources update to avoid the need to assume the internal file system layout.

The features superceded by plugins - Groovy document and Jsoup filters, hook scripts, custom workflow and custom gatherers are no available and any existing implementations that are updated to Funnelback 16 must rewrite this custom code using the plugin framework.

1 New and revised features

2020-08-11

16.1.2032

The support for performing the following tasks via REST APIs has been introduced:

  • Creating and deleting results pages.

  • Reading log files (via WebDAV).

  • Reading/editing of gscopes.cfg and external_metadata.cfg (via WebDAV).

  • Determining the update progress of a data source

  • Determining when the last successful update of a data source occurred.

  • Determining whether a running task has been asked to stop.

  • Deleting data sources and search packages."

2 Minor improvements

2020-08-11

16.1.2032

The query_processor_options configuration setting is now supported on results pages, overriding whatever is set on the search package configuration.

2 Minor improvements

2020-08-11

16.1.2032

The individual data source components and associated relative weightings of data sources in search packages can now be set via the configuration APIs with the meta.components and meta.components.[component].weight settings. When modified, these settings will be automatically applied to the search package.

2 Minor improvements

2020-08-11

16.1.2032

Streamlined knowledge graph administration experience.

2 Minor improvements

2020-08-11

16.1.2032

Tuning runs are now subject to the same task queueing system as data source and analytics updates.

2 Minor improvements

2020-08-11

16.1.2032

Updates to the default template to support results page level configuration settings where possible.

2 Minor improvements

2020-08-11

16.1.2032

Added support for knowledge graph scripts at the results page level.

2 Minor improvements

2020-08-11

16.1.2032

The data source components of a search package can now be set within the configuration editing screens.

2 Minor improvements

2020-08-11

16.1.2032

A number of improvements to SAML authentication support, in particular reduction in the number of SAML service providers required for administration setups and to support integration with Auth0.

2 Minor improvements

2020-08-11

16.1.2032

Added a Content-Type response header to the push API endpoint

2 Minor improvements

2020-08-11

16.1.2032

The 'Intercom' support tool has been integrated into the Funnelback administration dashboard.

3 Bug fixes

2020-08-11

16.1.2032

Prevented WebDAV clients which take long-timeout locks and do not reliably release them from locking out other clients.

3 Bug fixes

2020-08-11

16.1.2032

Fixed recommender operation on filecopy data sources.

3 Bug fixes

2020-08-11

16.1.2032

Fixed the all-results.json endpoint to handle when search sessions are enabled.

3 Bug fixes

2020-08-11

16.1.2032

Fixed presentation of sparklines within trend alerts reports.

3 Bug fixes

2020-08-11

16.1.2032

Fixed possible configuration setting loss when encrypting configuration values for the first time after installation.

3 Bug fixes

2020-08-11

16.1.2032

Prevented creation of users with service user prefixes.

3 Bug fixes

2020-08-11

16.1.2032

Fixed the web crawler to handle responses without a Content-Type header.

3 Bug fixes

2020-08-11

16.1.2032

Improved Padre handling of invalid XML characters.

3 Bug fixes

2020-08-11

16.1.2032

Fixed handling of ui.modern.pseudonymise_client_ips when Funnelback is used behind a proxy or load balancer.

3 Bug fixes

2020-08-11

16.1.2032

Fixed publication of web resources files containing spaces in their filenames.

3 Bug fixes

2020-08-11

16.1.2032

Fixed isAdminUI Freemarker macro when search and administration ports are the same.

3 Bug fixes

2020-08-11

16.1.2032

Fixed consistency of status codes resulted by the update history API.

4 Important changes

2020-08-11

16.1.2032

Tuning is now a task under the task queue, this means when it runs can be controlled by the task picker. Tuning can no longer be started by post /search-quality/v1/tuning/collections/{collection}/profiles/{profile}/runs?action=START_TUNING, instead it can only be started by using the task queue API: post /task/v1/queue/TUNING.

4 Important changes

2020-08-11

16.1.2032

The task queue now allows running tasks to be added to the queue and allows multiple tasks which use the same resources (for example, the same data source), to be in the queue at the same time.

4 Important changes

2020-08-11

16.1.2032

query_processor_options now supports configuration environments.

4 Important changes

2020-08-11

16.1.2032

The meta.cfg config file no longer exists, being replaced by the meta.components search package configuration setting.

4 Important changes

2020-08-11

16.1.2032

The groovy script specified by the auth.admin.saml.groovy-permission-mapper setting now supports defining roles the user is always permitted to edit.

4 Important changes

2020-08-11

16.1.2032

The administration dashboard’s edit file-manager rules pages are no longer available. Any remaining cases where custom file manager rules are required must be set directly in the relevant .ini files.

4 Important changes

2020-08-11

16.1.2032

Naming of log files has been made more consistent between data source types. For example, crawl.log is now named gather.log to be consistent across the product.

Renamed log files include:

  • crawler.central.log → crawler.log

  • crawl.log → gather.log

  • crawl.log.<N> → crawl.<N>.log

  • dbgather.log → gather.log

  • directory_gather.log → gather.log

  • social_media.log → gather.log

  • filecopier.log → gather.log

4 Important changes

2020-08-11

16.1.2032

Local data sources are not supported in this version, reflecting the restriction on direct filesystem access in the AWS SaaS environment. Existing local collections should be converted to either a web or custom data sources depending on the logic implemented in the local collection.

4 Important changes

2020-08-11

16.1.2032

Search package and data source IDs are now limited to 192 characters

4 Important changes

2020-08-11

16.1.2032

Freemarker templates are now only able to import/include files which end in .ftl

4 Important changes

2020-08-11

16.1.2032

Removed the deprecated collection_root and data_root configuration keys.

4 Important changes

2020-08-11

16.1.2032a

The question.collection.configuration data model exposed to Freemarker no longer contains server (global) configuration settings.

4 Important changes

2020-08-11

16.1.2032

The padre_opts.cfg profile configuration file has been removed. Query processor options specified within this file should now be set in the results page configuration, query_processor_options. The behavior has also changed slightly - the query processor options now override any options specified in the search package or default server configuration. This behavior differs from padre_opts.cfg in Funnelback 15.24 and earlier which merged the options.

4 Important changes

2020-08-11

16.1.2032

SAML authentication now creates user .ini files representing logged in users to support cross-component authentication.

4 Important changes

2020-08-11

16.1.2032

The _default profile is now hidden in the administration dashboard, but can still be accessed via WebDAV.

4 Important changes

2020-08-11

16.1.2032

Added a post_collection_create_script and pre_collection_delete_script server configuration options that run a defined script when a collection is created or deleted. The pre_collection_delete_script supersedes the pre_collection_delete_command from previous versions.

4 Important changes

2020-08-11

16.1.2032

Removed several Mediator API calls.

The following Mediator API calls have been removed and can no longer be accessed via the Mediator REST endpoint or via mediator.pl:

  • GetConfiguration

  • GetIndexTime

  • RecordStatistics

  • StartFilecopySlave

  • ValidateExternalMetadata

4 Important changes

2020-08-11

16.1.2032

Removed support for the angularJS sessions widget (funnelback-sessions-1.0.0.js or funnelback-sessions.js).

4 Important changes

2020-08-11

16.1.2032

Removed support for the search.classic and padre-sw.cgi legacy search endpoints.

4 Important changes

2020-08-11

16.1.2032

Removed support for the slack collection type.

4 Important changes

2020-08-11

16.1.2032

Removed support for multi-server filecopy gathering.

4 Important changes

2020-08-11

16.1.2032

Knowledge graph

  • Previous versions of Funnelback used a reserved id metadata class to hold the knowledge graph identifier which caused compatibility issues if id was required for another purpose. This has been replaced with a FUNkgLiveUrl metadata class. Any systems using the knowledge graph nodes endpoint directly will need to account for this change as part of upgrading to this and future versions.

  • All knowledge graphs must be manually updated after upgrading to support the metadata class change noted above.