Plugin: Generate auto-completion from search index

Purpose

Use this plugin to generate structured auto-completion from your search index.

This plugin generates a structured auto-completion source from the search index.

Unlike most other results page plugins you will need to run an update (such as advanced update > re-index the live view) after saving your plugin to ensure the auto-completion indexes are generated.

Configuring the trigger(s)

The main part of configuring this plugin is to set one or more appropriate triggers to compare to and defining what will be returned as the suggestion. The trigger is a word (or set of words) that are used to generate a suggestion, if it starts with whatever the user has typed into the search box. Punctuation is ignored when generating the trigger terms.

By default, the trigger will only match from the start of a multiple word trigger. If you wish each word to be compared to what the user has typed you need to set the match each word in a multi-word trigger option (see below).

Terminology

Partial query: what the user has typed in to a search input box so far. e.g. Stev.
Suggestion: the query completion suggested to the user as a possible auto-completion of their partial query. e.g. Steven.
Trigger: The name of the field in which to look for suggestions e.g. metadata.firstName.

Valid values

This plugin supports triggers specified using JsonPath expressions, which are compared to an individual results taken from the search result data model:

triggers Description

triggers	Description
`/title`	(Default) Suggestions will be sourced from the document title
`/listMetadata/<field>`	Suggestions will be sourced from the `field` metadata property.
`/listMetadata/<field1>,/listMetadata/<field2>`	Suggestions will be sourced from the `field1` and `field2` metadata properties.

/title

(Default) Suggestions will be sourced from the document title

/listMetadata/<field>

Suggestions will be sourced from the field metadata property.

/listMetadata/<field1>,/listMetadata/<field2>

Suggestions will be sourced from the field1 and field2 metadata properties.

If there’s no value specified for the trigger, or if the plugin can’t find the provided metadata then the plugin will take the title of the result object as the trigger.

Usage

Enable the plugin

Select Plugins from the side navigation pane and click on the Generate auto-completion from search index tile.
From the Location section, select the results page to which you would like to enable this plugin from the Select a results page select list.

The plugin will take effect as soon as you finish running through the plugin setup steps.

Configuration settings

The configuration settings section is where you do most of the configuration for your plugin. The settings enable you to control how the plugin behaves.

The configuration key names below are only used if you are configuring this plugin manually. The configuration keys are set in the results page configuration to configure the plugin. When setting the keys manually you need to type in (or copy and paste) the key name and value.

Trigger sources

Configuration key

plugin.auto-completion-plugin.config.triggers

Data type

array

Default value

[/title]

Required

This setting is required

Comma-separated list of fields in JsonPath format from where auto-completion triggers are sourced.

The values contained within these fields will be used as the trigger values.

The JsonPaths are applied to the search result element (/title corresponds to the result.title element from the search data model for an individual search result).

Action type

Configuration key

plugin.auto-completion-plugin.config.action-type

Data type

string

Default value

Q

Allowed values

Q,U

Required

This setting is optional

Action to perform when the suggestion is selected. U: open the URL which is sourced from the result.liveUrl field in the search result data model. Q: execute the suggestion as a query (default).

Data sources to include

Configuration key

plugin.auto-completion-plugin.config.data-sources

Data type

array

Required

This setting is optional

Comma-separated list of data source IDs from where suggestions are sourced.

Acceptable values are a subset of these data source IDs.

By default, results will be sourced from all data sources included in the results page.

Display fields

Configuration key

plugin.auto-completion-plugin.config.display-fields

Data type

array

Default value

[/title]

Required

This setting is required

Comma-separated list of result fields (expressed using JsonPaths) to return in the display object.

The values contained within these fields will be returned by the auto-completion web service in the response for use in your auto-completion template.

The JsonPaths are applied to the search result element (/title corresponds to the result.title element from the search data model for an individual search result).

Suggestion weight

Configuration key

plugin.auto-completion-plugin.config.weight

Data type

string

Default value

1

Value format

Allowed values must match the regular expression:

\d|\d\d|\d\d\d

Required

This setting is optional

Sets the weight applied to plugin suggestions. This usually only needs to be set if you are returning auto-completion suggestions in a single JSON response/auto-completion source. Setting a high value (e.g. 900) is recommended if you always want your plugin-generated suggestions to appear before any spelling generated suggestions. Default value: 1, Range: 0-999.

This only applies when you set the auto-completion.source to include multiple auto-completion sources (e.g. auto-completion.source=internal,csv.

Metadata class from which to source weight

Configuration key

plugin.auto-completion-plugin.config.weight_meta

Data type

string

Required

This setting is optional

Defines a Funnelback metadata class from where the item weight will be sourced. The weighting must be an integer value in the range 0-999. Items that do not have this metadata class, or have an invalid value will receive the default weighting.

Category field

Configuration key

plugin.auto-completion-plugin.config.category

Data type

string

Required

This setting is optional

JsonPath to a metadata field from where a category of the suggestion (to enable grouping of suggestions) will be sourced.

Category can be only sourced from a metadata field and only the first value will be used as the category value.

The JsonPaths are applied to the search result element (e.g. /listMetadata/category corresponds to the listMetadata["category"][0] element from the search data model for an individual search result).

Match each word in a multi-word trigger

Configuration key

plugin.auto-completion-plugin.config.trigger-expansion

Data type

boolean

Default value

false

Required

This setting is optional

This controls if the partial query is matched against each word in a multi-word trigger. e.g. 'smi' will match a 'john smith' trigger.

Possible values are:

true: generate extra triggers that will allow a user’s query to match the start of a word inside a multi-word trigger. This provides similar behaviour to the -partials mode for simple auto-completion. e.g. For a trigger value, John Proctor Leopold Smith trigger, generate extra triggers Proctor Leopold Smith, Leopold Smith and Smith.
false: (default) Only generate a single trigger value. e.g. for a value of John Proctor Leopold Smith only trigger the suggestion when someone starts typing joh.

Triggers starting with a stop word (such as and or the) will be skipped.

Collapsing signature

Configuration key

plugin.auto-completion-plugin.config.collapsing-sig

Data type

string

Required

This setting is optional

Collapses the results set using the supplied collapsing signature, e.g. '[title,code]'.

Result collapsing must be configured on the underlying data sources and include the defined signature in the 'indexing.collapse_fields'.

Result collapsing can be used to remove duplicates from your suggestions.

Auto-completion source

Configuration key

auto-completion.source

Data type

string

Default value

internal

auto-completion.source is a standard results page option that controls the sources of auto-completion when the auto-completion indexes are generated. By default, auto-completion generated on a result-page contains organic spelling suggestions.

In many cases you will want your generated auto-completion to only contain your structured suggestions. This setting will prevent the spelling-based suggestions from being added.

To configure this setting you will need to access the results page configuration screen (edit results page configuration) after you have finished setting up your plugin as set the following configuration key to suppress the spelling-based suggestions:

Plugin setting Value

Plugin setting	Value
`auto-completion.source`	(set this to an empty value)

auto-completion.source

(set this to an empty value)

Configuring trigger expansion

The plugin is able to expand a single trigger into multiple partial triggers by stripping words from the beginning of the trigger.

For example:

Funnelback Search Engine becomes Funnelback Search Engine, Search Engine, Engine

When the trigger contains very common english words (aka stop words) these will be skipped rather than being used to begin a trigger, to, for example, avoid the issue of thousands of triggers beginning with 'and'.

This mode is not enabled by default. To enable this mode, set the following config setting in the results page containing the plugin:

Plugin setting Value

Plugin setting	Value
Match each word in a multi-word trigger	`true`

Match each word in a multi-word trigger

true

Configuring the contents of the `disp` property (the display fields)

By default, the suggestion endpoint disp property contains a plain text string that matches the trigger value.

To get a JSON object with more details about the suggestion match, specify a list of JsonPath expressions in the display-fields configuration setting. Search result values that match the JsonPath values will be copied across to the disp object, with expressions that don’t match silently dropped.

For example, to return the title, url, and listMetadata.lastName values, you would specify:

Plugin setting Value

Plugin setting	Value
Display fields	`/title,/url,/listMetadata/lastName`

Display fields

/title,/url,/listMetadata/lastName

For the first search result from the example above, this will result in the following disp object:

{
    "title": "Steven Smith | Senior Lecturer",
    "url": "http://funnelback-university/staff/123",
    "listMetadata": {
        "lastName": ["Smith"]
    }
}json

If fields that don’t exist are specified, they are not returned in the disp result. e.g.

Plugin setting Value

Plugin setting	Value
Display fields	`/title, /foo, /listMetadata/lastName, /listMetadata/foobar`

Display fields

/title, /foo, /listMetadata/lastName, /listMetadata/foobar

results in:

{
    "title": "Steven Smith | Senior Lecturer",
    "listMetadata": {
        "lastName": ["Smith"]
    }
}json

Examples

Examples for setting different keys

Example: use the `title` and `summary` fields as triggers.

Plugin setting Value

Plugin setting	Value
Trigger sources	`/title,/summary`

Trigger sources

/title,/summary

Example: go to a URL when the suggestion is clicked.

Plugin setting Value

Plugin setting	Value
Action to perform when the suggestion is selected	`U`

Action to perform when the suggestion is selected

U

Example: data sources

Plugin setting Value

Plugin setting	Value
Data sources to include	`client~ds-staff, client~ds-customers`

Data sources to include

client~ds-staff, client~ds-customers

Example: include `title` and metadata `firstName` elements:

Plugin setting Value

Plugin setting	Value
Display fields	`/title,/listMetadata/firstName`

Display fields

/title,/listMetadata/firstName

Example: source the category from the `category` metadata field.

Plugin setting Value

Plugin setting	Value
Category field	`/listMetadata/category`

Category field

/listMetadata/category

Example: suggestion weighting

Plugin setting Value

Plugin setting	Value
Suggestion weighting	`900`

Suggestion weighting

900

Example: source suggestion weighting from metadata

If the 'importance' metadata class is set then the weight will be set to this value if it is a valid weighting, otherwise it will receive the default weighting of 900.

Plugin setting Value

Plugin setting	Value
Metadata class from which to source weight	`/listMetadata/importance`
Suggestion weighting	`900`

Metadata class from which to source weight

/listMetadata/importance

Suggestion weighting

900

The value can be sourced from any metadata class - this can be mapped in your metadata mappings, or provided via external metadata.

Example: Generate auto-completion from a staff directory

[{
    ...
    "title": "Steven Smith | Senior Lecturer",
    "url": "http://funnelback-university/staff/123",
    "listMetadata": {
        "firstName": ["Steven"],
        "lastName": ["Smith"]
    },
    ...
}, {
    ...
    "title": "Steve Wonder | Fellow",
    "url": "http://funnelback-university/staff/456",
    "listMetadata": {
        "firstName": ["Steve"],
        "lastName": ["Wonder"]
    },
    ...
}, {
    ...
    "title": "Jane Doe | Dean",
    "url": "http://funnelback-university/staff/789",
    "listMetadata": {
        "firstName": ["Jane"],
        "lastName": ["Doe"]
    },
    ...
}]json

If the plugin was configured with:

Plugin setting Value

Plugin setting	Value
Trigger sources	`/listMetadata/firstName`

Trigger sources

/listMetadata/firstName

When a user starts searching for Stev, this plugin will compare Stev against ["Steve", "Jane", "Steven"] and offer Steve and Steven as auto-completion suggestions.

If the plugin was configured with:

Plugin setting Value

Plugin setting	Value
Trigger sources	`/listMetadata/firstName,/listMetadata/lastName`

Trigger sources

/listMetadata/firstName,/listMetadata/lastName

this plugin will compare Stev against ["Steven", "Smith", "Steve", "Wonder", "Jane", "Doe" ].

If the plugin was configured with:

Plugin setting Value

Plugin setting	Value
Trigger sources	`/title`

Trigger sources

/title

The suggestions for Stev would be: Steven Smith Senior Lecturer and Steve Wonder Fellow

Change log

[2.5.0]

Added

Added an option, Metadata class from which to source weight, that allows the weight for auto-completion suggestions generated by the plugin to be sourced from a mapped metadata class.

[2.4.0]

Added

Added an option, Suggestion weight, that allows the default weight for auto-completion suggestions generated by the plugin to be defined.

[2.3.1]

Changed

Upgraded jackson-core, jackson-databind and jackson-datatype-guava dependency from v2.12.5 to 2.18.2.
Upgraded lombok dependency from v1.18.16 to 1.18.24.

[2.3.0]

Added

Added an option, Collapsing signature, that allows a result collapsing signature to be applied to remove duplicate items when generating the auto-completion.

[2.2.0]

Fixed

Fixed bug in trigger generation which caused punctuation characters to be included in the trigger terms.
Updated to the latest version plugin framework (Funnelback shared v16.24) to enable integration with the new plugin management dashboard.

[2.1.0]

Changed

Updated to the latest version plugin framework (Funnelback shared v16.20) to enable integration with the new plugin management dashboard.

[2.0.3]

Changed

Upgraded jackson-datatype-guava dependency from v2.10.3 to 2.12.5
Use Java native isBlank method as in Jsoup v1.12.1 class org.jsoup.helper.StringUtil was moved and marked for internal use only.

[2.0.2]

Added

Summary field will show full data

[2.0.1]

Added

plugin.auto-completion-plugin.config.category configuration key to set category field.

[2.0.0]

Added

plugin.auto-completion-plugin.config.data-sources configuration key to scope suggestions from particular data sources.

Changed

Configuration key to set triggers (see upgrade notes).

[1.0.0]

Added

plugin.auto-completion-plugin.config.trigger configuration key to set suggestion trigger.
plugin.auto-completion-plugin.config.action-type configuration key to define action to execute on selecting suggestion.
plugin.auto-completion-plugin.config.display-fields configuration key to define list of fields to return with suggestion to customize the suggestion display.

Plugin: Generate auto-completion from search index

Purpose

Configuring the trigger(s)

Terminology

Valid values

Usage

Enable the plugin

Configuration settings

Trigger sources

Action type

Data sources to include

Display fields

Suggestion weight

Metadata class from which to source weight

Category field

Match each word in a multi-word trigger

Collapsing signature

Auto-completion source

Configuring trigger expansion

Configuring the contents of the disp property (the display fields)

Examples

Examples for setting different keys

Example: use the title and summary fields as triggers.

Example: go to a URL when the suggestion is clicked.

Example: data sources

Example: include title and metadata firstName elements:

Example: source the category from the category metadata field.

Example: suggestion weighting

Example: source suggestion weighting from metadata

Example: Generate auto-completion from a staff directory

Change log

[2.5.0]

Added

[2.4.0]

Added

[2.3.1]

Changed

[2.3.0]

Added

[2.2.0]

Fixed

[2.1.0]

Changed

[2.0.3]

Changed

[2.0.2]

Added

[2.0.1]

Added

[2.0.0]

Added

Changed

[1.0.0]

Added

See also

Configuring the contents of the `disp` property (the display fields)

Example: use the `title` and `summary` fields as triggers.

Example: include `title` and metadata `firstName` elements:

Example: source the category from the `category` metadata field.