Help Center

Menu

Upgrading hook scripts

Hook scripts allowed search developers to implement custom groovy code that manipulated Funnelback’s search data model.

This is not permitted in the DXP as it is a security risk, and it also prevents automatic upgrades of the search.

This guide outlines the process you need to follow when upgrading the following configuration files:

  • hook_pre_process.groovy

  • hook_pre_datafetch.groovy

  • hook_post_datafetch.groovy

  • hook_post_process.groovy

  • hook_extra_searches.groovy

High level process

The key to successfully upgrading hook scripts is to break down the different unique tasks that the hook scripts are performing. When doing this you often need to look at the configuration holistically - because hook script logic is often dependent on both other collection configuration, and also on how the tasks span different hook scripts.

e.g. you might conditionally modify an input parameter in your pre-process hook script that is then used in the post datafetch hook script to make some other changes.

The identified tasks should be performing discrete operations that can then be replaced with other product functionality.

Replacing hook script functionality

Once you have broken down all the hook script functionality into a set of discrete tasks, you then need to figure out how this can be done in the DXP without any custom coding.

The high-level options to replace hook scripts tasks are:

  • Existing plugins: these implement commonly occurring tasks that were previous implemented as hook scripts - e.g. cleaning result titles or modifying the search URL. Become familiar with the plugins that are available and the functions they perform.

  • Curator rules: conditional logic in hook scripts can often be replaced with curator rules as these follow the trigger/action model. You need to ensure that there are appropriate triggers and actions for what the hook script is implementing as not all logic can be converted.

  • Other built-in functionality: hook scripts often replicate behavior that is available through built-in functionality - e.g. sorting of faceted navigation categories has been available as built-in functionality since Funnelback 15.12, but you still commonly find implementations where the categories are sorted within a post datafetch hook script.

  • Move the custom processing into your template/presentation layer. Highly custom modifications to the data model can often be replaced with conditional logic or Javascript within your search template.

Common patterns and their replacements

Modifications to the result title

Typical source: post-process or post-datafetch hook script

Example
transaction?.response?.resultPacket?.results.each() {
    // Remove ' - ACME International' from the end of titles
    it.title = (it.title =~ / - ACME International/).replaceAll("");
}

Remediation: replace this with the clean title plugin.

Modifications to the result URL

Typical source: post-datafetch hook script

Example
transaction.response.resultPacket.results.each()
{
    //Replace the URL with the value stored in the custom URL metadata field.
    it.displayUrl = it.displayUrl = it.listMetadata["url"][0];
    it.liveUrl = it.liveUrl = it.listMetadata["url"][0];
}

Remediation: replace this with the modify the search URL plugin

Setting a default query, supporting empty queries

Typical source: pre-process or pre-datafetch hook script

Example
// Enable empty query search
if (transaction.question.query == '' || transaction.question.query == null) {
transaction.question.query = '!padrenull'
}

Remediation: replace this the enable empty queries plugin

Supporting for wildcard queries

Typical source: pre-process or pre-datafetch hook script

Example
  // convert a partial query into a set of query terms
  // maximum number of query terms to expand partial query to - read from collection.cfg partial_query_expansion_index parameter
  // eg. partial_query=com might expand to query=[commerce commercial common computing]
  def partial_query_expansion_index = 5
  if ((q.collection.configuration.value(["partial_query_expansion_index"]) != null) && (q.collection.configuration.value(["partial_query_expansion_index"]).isInteger())) {
    partial_query_expansion_index = q.collection.configuration.value(["partial_query_expansion_index"])
  }

  if (q.inputParameterMap["partial_query"] != null) {
    File searchHome = new File("/opt/funnelback")
    File indexStem = new File(q.collection.configuration.value(["collection_root"]) + File.separator + "live" + File.separator + "idx","index")

    // NOTE: CONSTRUCTOR HAS CHANGED post v14.2 and requires searchHome as the first param
    List<Suggestion> suggestions = new PadreConnector(searchHome,indexStem)
    .suggest(q.inputParameterMap["partial_query"])
    .suggestionCount(partial_query_expansion_index)
    .fetch();

    def expanded_query = ""

    suggestions.each {
      expanded_query += '"'+it.key+'" '
    }

    // set the query to the expanded set of query terms ORed together
    if (expanded_query != "") {
      q.query = "["+expanded_query+"]"
    }
  }

Remediation: replace this the Query language - wildcard (truncation) support plugin

Sets a default sort (including for specific tabs)

Typical source: pre-process or pre-datafetch hook script

Example
// Automatically sort by the custom metadata rank, then normal score
if (transaction.question.inputParameterMap["sort"] == "" || transaction.question.inputParameterMap["sort"] == null) {
    // Only apply sort if null query has been run
    if (transaction.question.query == '!padrenull'){
        transaction.question.dynamicQueryProcessorOptions << "-sort=metarank"
    }
}

Remediation: replace this the curator rules

Sets input parameters

Typical source: pre-process or pre-datafetch hook script

Example
// If no audience facet is selected, select Public by default
if (transaction.question.profile == '_default' || transaction.question.profile == '_default_preview') {
    if (transaction.question.inputParameterMap['form'] != null && transaction.question.inputParameterMap['form'] == "sitemap" && transaction.question.inputParameterMap['gscope1'] == null) {
        transaction.question.inputParameterMap['gscope1'] = 'discontinued!'
        transaction.question.additionalParameters['gscope1'] = ['discontinued!']
    }
}

Remediation: replace this the curator rules

Normalizes query input, adds stop words

Typical source: post-process hook script

Example
// Library imports required for the normalisation
import java.text.Normalizer;
import java.text.Normalizer.Form;

if (transaction.question.form == "auto-completion") {
transaction?.response?.resultPacket?.results.each() {
// Do this for each result item

        // Create a normalised version of the name metadatafield that removes diacritics
        // This calls a Java function to normalize the name metadata field and write the normalised
        // version of the name into a new metadata field called nameNormalized
            if (it.metaData["name"] != null) {
                it.metaData["nameNormalized"] = Normalizer.normalize(it.metaData["name"], Form.NFD).replaceAll("\\p{InCombiningDiacriticalMarks}+", "");
            }
        }

        // read stop words into data model customData element
        //using $SEARCH_HOME for the path of the file
        //def stopFile = "${transaction.question.collection.configuration.searchHomeDir}\\share\\lang\\en_stopwords"
        // Linux
        def stop_file = "/opt/funnelback/share/lang/en_stopwords";
        // Windows
        // def stop_file = "c:\\funnelback\\share\\lang\\en_stopwords";
        def stop = new File(stop_file).readLines();
        transaction.response.customData["stopwords"] = stop
}

Remediation: remove this hook script. The normalization functionality is part of the old way of generating auto-completion and this becomes redundant when you set up the auto-completion plugin for generating the auto-completion.

Sorting faceted navigation

Typical source: post-process or post-datafetch hook script

Example
//**** SORT FACETS ****
if ( transaction.response != null && transaction.response.facets != null
        && transaction.response.facets.size() > 0 ) {
        transaction.response.facets.each() {
            if (it.name != "Residency") {
                it.categories.each() { sortCategory(it) }
            } else {
                it.categories.each() { sortCategoryZA(it) }
            }
        }
}
// Recursively sort A-Z categories listing by alphabetical order
def sortCategory(category) {
        category.values.sort {a, b -> a.label.compareTo(b.label)}
        category.categories.each() { sortCategory(it) }
}
def sortCategoryZA(category) {
        category.values.sort {a, b -> b.label.compareTo(a.label)}
        category.categories.each() { sortCategory(it) }
}

// Recursively sort A-Z categories listing by Special MMM YYYY order - note input labels expected in MMMM YYYY (eg. February 2000)
def sortCategoryDate(category) {
        category.values.sort {a, b -> sortCompare(b, a)} //Note to sort reverse date swap to use sortCompare(a, b) instead of (b, a)
        category.categories.each() { sortCategoryDate(it) }
}
def sortCompare(a, b) {
//convert labels into a sortable representation of the date YYYYMM  eg Mar 2013 -> 201303
    def labela = a.label;
    def labelb = b.label;

    def datea = new Date().parse("MMMM yyyy", labela);
    def dateb = new Date().parse("MMMM yyyy", labelb);

    labela = String.format('%tY%<tm',datea);
    labelb = String.format('%tY%<tm',dateb);
//logger.info("[["+labela+"|"+labelb+"]]")

    labelb.compareTo(labela);
}


// Recursively sort categories belonging to a Gscope based facet by custom order.  The categories are each of the gscope category lines defined for the gscope facet.
def sortGscopeCategory(facet) {
        facet.categories.sort {a, b -> sortGscopeCategoryCompare(a, b)}
}
def sortGscopeCategoryCompare(a, b) {
    def labelOrderMap = [ // Gscope facet label 1 - 5 correspond to the label values for the gscope categories that sit under the gscope facet - sort in order 1-5
                         "Gscope facet label 1":"900012",
                         "Gscope facet label 2":"900011",
                         "Gscope facet label 3":"900010",
                         "Gscope facet label 4":"900009",
                         "Gscope facet label 5":"900008"
                         ];

    def labela = a.values[0].label;
    def labelb = b.values[0].label;

    if (labelOrderMap.containsKey(labela)) {
        labela = labelOrderMap[labela];
    }
    if (labelOrderMap.containsKey(labelb)) {
        labelb = labelOrderMap[labelb];
    }

    labelb.compareTo(labela);
}

Remediation: Replace by configuring the sort type within the configuration for the facet. If custom sort is used then you will also need to enable and configure the faceted navigation categories - custom sort order plugin.