Search-based auto-completion

Table of Contents

Background

Funnelback has a native auto-completion system (suggest.json) that suggests completions as the user types. This auto-completion system uses a JSON REST API to provide suggestions based on the user’s partially entered query using a lightweight search algorithm. This ensures that suggestions are returned quickly and with minimal load on the server.

In some cases however, there may be a need to use the full text search algorithm to provide suggestions to the user. In doing this however, we face a couple of challenges:

  • The user has not entered the full query they are looking for. We may only have queries like "con" or "dav" not giving a lot of context to what the user is searching for yet.

  • The time it takes to run a full query using the standard search algorithm is slower than using the native autocompletion system

To overcome these challenges we can take the following steps:

  • Utilise the top suggestions from the native autocomplete and perform a full text search based on these suggestions

  • Limit the data returned by the result packet

  • Turn off unnecessary search features or settings

Occasionally Funnelback’s auto-completion is too basic to service the needs of a user and it becomes necessary to use a standard Funnelback collection and padre-sw query to supply the auto-completion.

The following process can be used to deploy collection based auto-completion.

The use of search-based auto-completion is not recommended and should be avoided unless other auto-completion options have been exhausted. Funnelback’s built-in auto-completion system is designed for speed (and minimal system overhead when the request is processed). Using a standard query to produce query completion can result in significant load being placed on the server as a query can be fired for each user keystroke.

Setup process

  1. Create a results page that will be used to deliver auto-completion suggestions

  2. Add the following to the query processor options for the results page to ensure the query is optimized.

    -stem=0 -SM=off -bb=false -rmcf=[dummyvalue] -spelling=off -contextual_navigation=false -QL=0 -log=false -countgbits=63  -collapsing=false -show_qsyntax_tree=off
  3. Add the following custom Freemarker template to the results page:

    qc.ftl
    <#ftl encoding="utf-8" />
    <#import "/web/templates/modernui/funnelback_classic.ftl" as s/>
    <#import "/web/templates/modernui/funnelback.ftl" as fb/>
    <#escape x as x?jsonstring>
    <#if question.inputParameterMap["callback"]?exists>${question.inputParameterMap["callback"]}(</#if>[
    <@s.AfterSearchOnly>
    <#if response.resultPacket.resultsSummary.totalMatching != 0>
    <@s.Results>
    <#if s.result.class.simpleName != "TierBar">
    {
    "key" : "<#if question.inputParameterMap["partial_query"]?exists>${question.inputParameterMap["partial_query"]?json_string}</#if>",
    "disp" : "${s.result.title}",
    "disp_t" : "T",
    "wt" : "${s.result.score}",
    "cat" : "",
    "cat_t" : "1",
    "action" : "${s.result.clickTrackingUrl}",
    "action_t" : "U"
    }<#if (s.result.rank < response.resultPacket.resultsSummary.currEnd)>,</#if>
    </#if>
    </@s.Results>
    </#if>
    </@s.AfterSearchOnly>]
    <#if question.inputParameterMap["callback"]?exists>)</#if>
    </#escape>
  4. Enable the query language - wildcard (truncation) support plugin.

  5. Add the following options to the results page configuration:

    auto-completion.program

    ../s/search.html

    ui.modern.form.qc.content_type

    application/javascript

    partial_query_expansion_index

    8 1

    Note: set 1 as per the plugin setup instructions.