- Before you start
- Administration users and roles
- Using the upgrade utility
- Defining the set of users to import
- Copying analytics log files
|If you are running a version of Funnelback earlier than 15.24 it is recommended that you first upgrade to 15.24 before attempting to upgrade to Funnelback 16.|
The Funnelback upgrade utility automates a number of tasks with the aim of minimizing the work required to import a search from v15. The upgrade tool:
Transforms v15 collections and profiles into v16 search packages, data sources and results pages:
Converts meta collections to search packages
Converts non-meta collections to a meta collection plus results pages (for any pre-existing service-enabled profiles) plus a data source.
Updates collection ID references within templates and other configuration files
Updates some deprecated data model references in templates.
Can pull in v15 collections from a remote Funnelback 15.24 instance (including configuration, data, logs and associated analytics, and related users and roles).
Put together a list of all collections that should be upgraded as a set.
Group the collections into clients. This will normally be a single client that maps back to a customer (or organization). An upgrade should only focus on a single client at a time.
Determine if the client exists in the v16 server or if you need to create a new client.
For new clients a license is required.
For existing clients make a note of the client ID.
Put together a list of all client roles (and the role IDs) used by the client in v15 (ignoring default roles).
Put together a list of all client users (and the user IDs) in v15.
For each collection (non-meta and non-push)) note down the current update schedule as these will need to be set up again after the upgrade.
|Upgrading all collections, users and roles that belong to a client in a single upgrade operation is recommended.|
It is recommended that you perform a review and cleanup of your v15 collections to:
Ensure all profiles you want to be converted to results pages are set as services (in v15 select the relevant profile from the profile switcher, then click the enable service button). This includes any profile that is used for system-specific purposes, such as generating auto-completion or for search-powered content.
Identify any push collections and put these onto a separate list (push collections should be upgraded separately).
Identify any profiles used for generating auto-completion (see the auto-completion upgrade guide for detailed instructions on migrating your auto-completion configuration to use the auto-completion plugin).
The concept of a client has been added in v16. This extends the v15 concept of collection grouping to collect everything that relates to a particular organisation/client/customer under a single identity.
A client can have the following items associated with it:
users and roles
search packages and data sources (and results pages)
The client concept introduces a client ID which is prepended to search package and data source IDs and also usernames and roles.
Collections, users and roles that are migrated into v16 must be attached to a v16 client.
When the upgrade tool is run the client must be specified. If the client exists in v16 then the collections/users/roles will be attached to this client. If the client doesn’t exist then it will be created by the upgrade utility.
v15 collections and profiles are now replaced with v16 search packages, data sources and results pages.
The mapping of these concepts between v16 and v15 is as follows:
|Search package||Meta collection|
|Data source||Non-meta collection|
|Results page||Service-enabled profile|
Key points to note:
In v16 results pages are only associated with search packages (data sources cannot have search packages set up on them).
v16 search packages and results pages must be associated with a client, and the IDs will change to have the client ID prepended to the pre-existing collection ID.
Upgrading non-meta collections into v16 will result in a search package and data source being created in place of the old collection. Any service-enabled profiles associated with the non-meta collection will be attached to the search package.
The ID of each collection migrated into v16 will change to be prefixed with the v16 client ID.
e.g. Migration two v15 collections,
example-collection2 into v16, attaching to the example-client customer:
|The v16 search package or data source ID is a concatenation of the v16 client ID, a tilde and the v15 collection ID.|
v15 collection ID references within the following configuration will be upgraded to v16 IDs:
faceted navigation configuration
users and roles
The upgrade of collection IDs contained within users and roles has some limitations and upgraded users and roles should be manually checked after the upgrade.
v15 collection ID references in other configuration are not upgraded to v16 IDs and will require manual upgrading. This includes:
Auto-completion widget configuration
Filters, workflow scripts, hook scripts
Session and history templates
Each v16 imported search package and data source ID will be assigned to the client (via the
example-client~resources role). Users that are granted this role will have access to the search packages and data sources.
The v15 collection configuration will be upgraded with v16 changes when migrated to v16 search packages and results page. This includes:
Update of Freemarker templates and hook scripts to account for changes in the search data model
reporting-email.cfgsettings into results page configuration options.
In addition to the collection ID changes mentioned above the upgrade tool will attempt to auto-upgrade templates and hook scripts to account for a number of data model changes.
This includes upgrading references of:
However, the tool cannot guarantee all syntax variations used in templates and hook scripts will be upgraded correctly and the upgraded templates should also be manually checked for errors. Errors may result in the template breaking and returning a 500 error, or error messages being logged to the modernui logs for the results page.
|If the content of any listed above property was assigned to separate variable it may not be upgraded.|
e.g. the usage of
inputParameterMap won’t be upgraded properly to
inputParameters in the groovy code snippet below:
def questionParam = inputQuestion.inputParameterMap questionParam["f.Residency|3"] = "local"
expected in v16
def questionParam = inputQuestion.inputParameters questionParam.get("f.Residency|3") = "local"
upgraded with tool
def questionParam = inputQuestion.inputParameters questionParam["f.Residency|3"] = "local"
The ID of each administration user and role migrated into v16 will change to be prefixed with the v16 client ID.
e.g. Migration a v15 administration user,
example-user and role,
example-role into v16, attaching to the
Each user/role ID will be assigned to the client (by inheriting
/opt/funnelback/linbin/java/bin/java -cp "/opt/funnelback/lib/java/all/*" com.funnelback.upgrade.wrapper.CLI <parameters>
This the source (v15) and destination (v16) locations for the upgrade utility
The servers can communicate via ssh or WebDAV.
A URI specifying the source for example:
file:///tmp/funnelbackwill set the source as the search home
/tmp/funnelback/local to the server; For Windows the form is
webdav://fb15-admin.server.comwill set the source as a remote WebDAV server assumed to be hosted by Funnelback. By default port 8076 will be used but that can be configured to an alternate port if required. e.g.
webdav://fb15-admin.server.com:1234. This will use a system user and expects the server secret of the remote Funnelback to be set in the environment
SOURCE_SERVER_SECRET. If the WebDAV connection is slow, set the java option to increase the number of threads:
-- java -Djava.util.concurrent.ForkJoinPool.common.parallelism=16
ssh://firstname.lastname@example.org/opt/funnelback/will set the source to be via a ssh server using the user
searchand expects the remote Funnelback search home to be the path in this case
/opt/funnelback/. If the running machine has many cores ( e.g. 32 or more) you may need to limit the number of threads with the java option:
-- java -Djava.util.concurrent.ForkJoinPool.common.parallelism=1
The configuration migrated from v15 must be assigned to a client by providing the
If the client defined by
clientId doesn’t exist, a new client will be created:
--client-human-friendly-name=<clientHumanFriendlyname>is provided, it will be used to set the readable name of the client
A license must be specified when creating a new client using
--license-id=<licenseId>. The specified license will assigned to the newly created client
--remote-collection parameter is used to specify the list of collections to import from v15 configuration. The v15 collection ID must be supplied as the parameter value.
e.g. to import collections
By default all parts of the collection will be imported. Specific things can be excluded by setting the
--exclude to the following values:
CONF: The collection configuration files, sourced from v15
REPORTS: The collection analytics and report databases, sourced from v15
DATA_REPORTS: The collection data reports, sourced from v15
ARCHIVE: The collection archive log files, sourced from v15
DATABASES: The collection database files, sourced from v15
LOG: The collection log files, sourced from v15
VIEW_LIVE: The collection live data files, sourced from v15
VIEW_OFFLINE: The collection offline data files, sourced from v15
e.g. exclude databases and data reports:
--remote-user parameter is used to specify the set of users to import from v15 configuration. The v15 user ID must be supplied as the parameter value.
e.g. to import users
--remote-role parameter is used to specify the set of roles to import from v15 configuration. The v15 role ID must be supplied as the parameter value.
e.g. to import users
|Default Funnelback roles should not be imported as v16 already includes updated default roles.|
This option can be used to define additional mappings for collections, users and roles that belong to the client but are not being imported with the defined set of collections, users and roles.
This enables the upgrade tool to correct references within the imported configuration to these additional items.
These options are useful for cases where it is not possible to move all collections/users/roles in one go. e.g. one customer may have few different projects/implementations, or one user/role is managing collections/users/roles from different projects or for a few different customers
These options provide information about already upgraded collections/users/roles and their IDs so that references to old IDs are upgraded properly when the additional collections, users or roles are imported. For example, if collection
foo was already upgraded but is referred to in a subsequent run of the upgrade utility, provide the mapping to the new ID using
--other-collection-mapping=<String=String>defines an additional collection mapping to apply when upgrading collections. This is used to configure the tool for any collections that are part of the set of collections that have already been migrated previously. The CLI tool,
com.funnelback.common.upgradeconfig.UpgradeConfUtilityCLIcan be used to correct unresolved references (to additional collections) for previously migrated collections.
--other-role-mapping=<String=String>defines an additional user mapping to apply when upgrading collections. This is used to configure the tool with mappings for any roles that have already been imported that are referenced from the new set of collections.
--other-user-mapping=<String=String>defines an additional user mapping to apply when upgrading collections. This is used to configure the tool with mappings for any users that have already been imported that are referenced from the new set of collections.
--new-collection-mapping=<String=String>can be used to define mappings for collections that were previously imported. This is used when importing additional roles and users to ensure that any references to previously imported collections are corrected mapped.
a full set of command line options will be displayed if you run the following command
The upgrade utility will transfer all the analytics log files when the upgrade command is run. However it is often desirable to re-sync the log files when the upgrade is ready to be released so that all the search logs are transferred to the server.
The logs can be re-synced by running the following command:
$SEARCH_HOME/linbin/java/bin/java -classpath "$SEARCH_HOME/lib/java/all/*" com.funnelback.mediator.core.collectionpuller.CLI <SOURCE> <SOURCE-COLLECTION-ID> $SEARCH_HOME --target-collection=<TARGET-COLLECTION-ID> --sync-archive-logs
<SOURCE>: is a URL specifying the source server and search home (see: Defining the source and target locations above).
<SOURCE-COLLECTION-ID>: is the collection ID of the collection containing the search logs (this is the collection ID that is supplied as the collection parameter when you run a search) on the v15 server.
<TARGET-COLLECTION-ID>: is the v16 collection ID of the search package that was created from the v15 collection when it was imported.