Help Center

Menu

Upgrading generated auto-completion to use the auto-completion plugin

This guide covers the steps required to upgrade a v15 Funnelback collection that generates structured (CSV) auto-completion for a search to use the Funnelback auto-completion plugin.

Generated auto-completion must be re-implemented for v16. There have been many (non-standard) approaches to generating the auto-completion CSV over the years and this guide, with the help of the plugin, will standardize the way this is generated which will enable smoother upgrades and enhancements to auto-completion in future.

The auto-completion plugin:

  • Uses the Funnelback index to generate the CSV used for structured auto-completion.

  • Uses a dedicated results page to scope the search index (if required) and define what metadata fields, triggers and actions to include in the CSV.

  • Generates CSV with JSON results type (legacy HTML and text types are redundant).

    If your existing auto-completion CSV uses the HTML (H) or text (T) display types these will probably need to be updated to return a JSON (J) packet. The handlebars template that is used to display the auto-completion can be defined in the Javascript that configures the auto-completion.

High level migration process

Funnelback 15 and earlier provided support for a static CSV data source for auto-completion. The auto-completion.csv file could be manually edited via the search dashboard.

Implementers made use of Funnelback’s advanced workflow system to generate this CSV file using the Funnelback index and a custom template. There were a couple of commonly used implementations made available via GitHub and as part of stencils that were reused for implementation, however there was no formal supported way of generating auto-completion and as a result there are many variants of CSV generation in use.

Due to changes with how Funnelback 16 works it is no longer possible to use build_autoc to generate auto-completion indexes for Funnelback from update workflow commands - this breaks the v15 process for generating auto-completion.

This means that all generated auto-completion will need to be re-implemented for v16 using the auto-completion plugin. In v15 custom workflow commands were used to generate the auto-completion.

To convert an existing auto-completion generation process requires analysis of the existing implementation of each auto-completion source to determine:

  • the collection/profile combination used to generate the auto-completion and the template that was used to format the CSV data.

  • The field(s) used to set the triggers for the generated auto-completion

  • The field(s) returned as data for the auto-completion suggestion

  • The type of auto-completion response data (JSON vs HTML vs text)

  • The type of action used for the auto-completion (URL vs query)

Once these have been determined, a new results page can be created on the v16 search package with the purpose of generating a single auto-completion source using the auto-completion plugin.

The new results page should scope the search package to restrict the results to match the old profile (this will normally require adding an additional scope to restrict results to only those from the relevant data source).

If the old generated auto-completion produced HTML or text suggestions the Javascript configuration will need to be amended to provide a handlebars template to format the JSON returned by the new auto-completion.

Migrating auto-completion configuration that uses the GitHub helper code or Stencils

Many Funnelback implementations utilize standard code and configuration as defined in the funnelback-concierge GitHub project or use a variant of this that is part of stencils.

Check if your existing auto-completion is generated using this by looking at your collection.cfg files - for your meta collection and also the component collections.

Look for the following configuration keys:

  • auto-completion.DATASET-NAME.triggers

  • auto-completion.DATASET-NAME.action-mode

  • stencils.auto-completion.DATASET-NAME.triggers

  • stencils.auto-completion.DATASET-NAME.action-mode

If your collection includes either if these keys you are probably using this standard method of generating the auto-completion CSV.

Converting the GitHub/Stencils auto-completion generation to the plugin

Follow the generating structured auto-completion from the search index to configure the auto-completion plugin in v16.

The conversion will need to:

  1. Set up a results page on your search package to generate the auto-completion. In v15 auto-completion was usually generated on the data source, in v16 this must be moved to the search package.

  2. Enable the auto-completion plugin on the (auto-completion) results page.

  3. Configure the plugin auto-completion triggers by translating the auto-completion.DATASET-NAME.triggers. e.g. s.result.metaData["name"] would translate to /listMetadata/name.

  4. If the action-mode is set to Q in the v15 configuration set plugin.auto-completion-plugin.config.action-type=Q. For stencils all listed fields are metadata. e.g. stencils.auto-completion.triggers=name,title would translate to /listMetadata/name,/listMetadata/title.

  5. If the v15 configuration was on a data source collection set plugin.auto-completion-plugin.config.data-sources to the data source ID containing the source data. This is used to ensure auto-completions are restricted to items from the specified data source.

  6. Inspect the padre_opts.cfg on the v15 profile and note the SF value and use the set of metadata fields to specify the plugin.auto-completion-plugin.config.display-fields configuration setting.

  7. Other items set within padre_opts.cfg should be set up as query_processor_options in the results page configuration. e.g. if padre_opts.cfg had the following options set -gscope1=3 -SM=off -log=false -num_ranks=1000 -vsimple=true the set query_processor_options=-gscope1=3 -SM=off -log=false -num_ranks=1000 -vsimple=true in the results page configuration.

  8. Update the Javascript auto-completion configuration in the search results template with the new profile (results page ID) and collection (search package ID).

  9. If you’ve imported existing v15 collections into v16 you should remove the old auto-completion related configuration settings, workflow and any auto-completion related profiles on data source collections.

Manual migration a v15 auto-completion configuration

It is not possible to provide specific details for other implementations of auto-completion generation beyond the two outlined above as these are generally unique for each implementation.

This section provides generic advice that assists in determining the information required so that auto-completion can be re-implemented using the auto-completion plugin.

Determine if you are using the concierge Javascript or the older JQuery-UI implementation

Inspect your search results html and look at the Javascript imports. Most implementation will be using the typeahead implementation of auto-completion with Javascript imports similar to:

<script src="${GlobalResourcesPrefix}thirdparty/typeahead-0.11.1/typeahead.bundle.min.js"></script>
<script src="${GlobalResourcesPrefix}thirdparty/handlebars-4.7/handlebars.min.js"></script>
<script src="${GlobalResourcesPrefix}js/funnelback.autocompletion-2.6.0.js"></script>

Inspect the Javascript block that configures the auto-completion. Your Javascript should define one or more 'dataset' elements (which translate to columns in the auto-completion widget), each of which define an auto-completion data set. Each dataset will define a collection and profile item (or inherit this from a parent element). Make a note of each of these datasets as you will need to create a new auto-completion source with the plugin for each of these.

If there are no dataset elements in use you might be using the older JQuery-UI version of auto-completion. If you are using the older jquery implementation you will have imports similar to:

<script src="${GlobalResourcesPrefix}js/jquery/jquery-1.10.2.min.js"></script>
<script src="${GlobalResourcesPrefix}js/jquery/jquery-ui-1.10.3.custom.min.js"></script>
<script src="${GlobalResourcesPrefix}thirdparty/bootstrap-3.0.0/js/bootstrap.min.js"></script>
<script src="${GlobalResourcesPrefix}js/jquery/jquery.tmpl.min.js"></script>
<script src="${GlobalResourcesPrefix}js/jquery.funnelback-completion.js"></script>

The JQuery-UI version of auto-completion there only supports a single collection and profile for auto-completion but there may be several CSV files that are used to build the auto-completion.

Upgrading a concierge Javascript implementation

Inspect the configuration of each collection/profile combination that was identified when looking at the Javascript.

For each of these determine if the auto-completion is a CSV based auto-completion.

For each CSV source:

  • Inspect the workflow commands/scripts used to generate the auto-completion. The workflow will normally consist of two steps - a step that accesses a Funnelback search URL via http (usually using curl, wget or groovy code) and a step that runs build_autoc. The Funnelback search URL will specify a collection/profile/template that runs a search and returns the search results as a CSV file (in auto-completion CSV format). The build_autoc command converts the CSV file into an auto-completion index. In v16 the workflow commands will need to be removed and replaced with auto-completion plugin configuration.

  • Inspect the template that is used to generate the CSV. The template will populate one or more rows of CSV for each search result. Make a note of what is being used to populate each of the CSV columns. If the display data is returned as HTML you will need to note down what fields are used within the HTML snippet, and also make a note of the html template used as this will need to be recreated as a handlebars template in v16. If the display data is returned as plain text this will need to be upgraded to JSON unless the trigger term is displayed as the suggestion.

  • Inspect the configuration of the profile and look at the padre_opts.cfg. Pay special attention to any query processor options that scope the result set as these will need to be applied to the results page configuration in v16. Other query processor options used for auto-completion profiles mostly optimize the query so that auto-completion generation is fast. These should also be copied across to the v16 results page configuration.

  • Once you have determined the source of the CSV fields you can follow the generating structured auto-completion from the search index guide to configure the auto-completion plugin in v16.

  • If you’ve imported existing v15 collections into v16 you should remove the old auto-completion related configuration settings, workflow and any auto-completion related profiles on data source collections.

Upgrading a JQuery-UI Javascript implementation

Inspect the configuration for the collection/profile combination that was identified when looking at the Javascript.

  • Inspect the collection.cfg and check for auto-completion.source - this should be set to csv or csv,internal. If not set then there is probably no CSV based auto-completion being used, or custom workflow is used to generate then build the auto-completion.

  • Inspect the collection.cfg and check for auto-completion.source.extra - this will outline any additional CSV sources that are used when generating auto-completion.

  • Inspect the collection.cfg and check for workflow commands that generate auto-completion. These will usually be a post_gather_command or pre_index_command. The commands will need to be analysed to determine if/how auto-completion is being generated. Make a note of any CSV auto-completion files that are being generated and what is being used to populate each field in the CSV.

  • Once you have determined the source of the CSV fields you can follow the generating structured auto-completion from the search index guide to configure the auto-completion plugin in v16.

  • If you’ve imported existing v15 collections into v16 you should remove the old auto-completion related configuration settings, workflow and any auto-completion related profiles on data source collections.

When upgrading a JQuery-UI Javascript implementation it is highly recommended to generate each of the CSV sources as separate auto-completion sources then upgrade the Javascript to use the concierge Javascript code which allows each source to be configured separately. The upgrading to concierge guide provides guidance on how to upgrade your Javascript to use the newer concierge Javascript.