Writing search lifecycle manipulation code to manipulate the data model

Modifications to the search query and response can be implemented by writing a search lifecycle plugin which contains code that can read and write to the data model.

General concepts

Search lifecycle plugins provide methods which interact with the data model at different points in the search lifecycle.

Pre-post phase step Plugin - SearchLifeCyclePlugin interface method

pre-process

void preProcess(SearchTransaction transaction)

pre-datafetch

void preDatafetch(SearchTransaction transaction)

post-datafetch

void postDatafetch(SearchTransaction transaction)

post-process

void postProcess(SearchTransaction transaction)

If you are familiar with writing hook scripts, the contents of these methods match the contents of the equivalent hook script. If you wish to convert a hook script into a plugin you can transfer the code from your hook script into your plugin and then convert it to Java.

Search types

By default, search lifecycle code will run on all search requests including those run by content auditor, accessibility auditor and also extra searches.

Each of these searches has a particular search question type that indicates the type of search that is running.

Conditional code based on the search question type can be used to write code that only affects the data model question and response for the specific types of searches.

Obtaining the search question type

The following Java method provides a function call to return the current search question type which can be called from the hook script.

// Get the list of defined question types
import com.funnelback.publicui.search.model.transaction.SearchQuestion.SearchQuestionType;

...

// Get the question type of the current search
SearchQuestionType questionType = transaction.getQuestion().getQuestionType()

Possible values for the question type are:

Question type Description

SearchQuestionType.SEARCH

A search query submitted to the HTML, XML, JSON endpoints (e.g. search.html, search.json)

SearchQuestionType.SEARCH_GET_ALL_RESULTS

A search query submitted to the all-results endpoint all-results.json or all-results.csv.

SearchQuestionType.EXTRA_SEARCH

An extra search configured on a search package.

SearchQuestionType.CONTENT_AUDITOR

A content auditor query.

SearchQuestionType.CONTENT_AUDITOR_DUPLICATES

A content auditor duplicates query.

SearchQuestionType.ACCESSIBILITY_AUDITOR

An accessibility auditor query.

SearchQuestionType.ACCESSIBILITY_AUDITOR_ACKNOWLEDGEMENT_COUNTS

Accessibility auditor query to determine acknowledgement counts.

SearchQuestionType.ACCESSIBILITY_AUDITOR_GET_ALL_RESULTS

Accessibility auditor all-results query.

SearchQuestionType.FACETED_NAVIGATION_EXTRA_SEARCH

Built-in extra search used to obtain faceted navigation information.

Restricting search lifecycle code to specific search types

This allows the plugin code to target a specific question type using a conditional statement. e.g. only run for the CONTENT_AUDITOR search:

// Only run this for content auditor queries
if (questionType.equals(SearchQuestionType.CONTENT_AUDITOR)) {
    // Whatever you want to do...
}

Testing

When writing unit tests for the SearchTransaction use the following helper classes to generate the mock data sent to your tests:

  • TestableSearchTransaction

  • SearchQuestionTestHelper

Logging

Log messages will appear in the search package’s user interface logs.

Examples

Example: title-prefix-plugin

The developing your first plugin guide runs through the process of creating a search lifecycle plugin. This plugin replaces a pattern at the start of a title in matching search results.

  • NOTE: This plugin is explained in detail as part of the developing your first plugin tutorial - please refer to this for detailed notes on various parts of the code.

TitlePrefixPluginSearchLifeCyclePlugin.java
package com.example.plugin.titleprefixplugin;

import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;

import com.funnelback.plugin.SearchLifeCyclePlugin;
import com.funnelback.publicui.search.model.transaction.SearchTransaction;

/**
 * Note: This plugin is explained in detail as part of the 'Developing your first plugin' guide in the Funnelback product documentation.
 */
public class TitlePrefixPluginSearchLifeCyclePlugin implements SearchLifeCyclePlugin {

    private static final Logger log = LogManager.getLogger(TitlePrefixPluginSearchLifeCyclePlugin.class);
    @Override public void postProcess(SearchTransaction transaction) {
        String prefixToMatch = transaction.getQuestion().getCurrentProfileConfig().get(PluginUtils.KEY_PREFIX + "pattern");
        String givenReplaceWith = transaction.getQuestion().getCurrentProfileConfig().get(PluginUtils.KEY_PREFIX + "replaceWith");

        if (prefixToMatch == null || prefixToMatch.isBlank()) {
            // There is no prefix configured, we can't do anything. Lets log a warning and abort.
            log.warn("Could not find a pattern specified at " + PluginUtils.KEY_PREFIX +  "pattern. The TitlePrefixPlugin will do nothing.");             return;
        }
        // We'll mark replaceWith as an optional key, if someone doesnt supply it, then we'll just remove the pattern configured.
        String replaceWithToUse = givenReplaceWith != null ? givenReplaceWith: "";

        transaction.getResponse().getResultPacket().getResults().stream().filter(result -> {
            return result.getTitle().startsWith(prefixToMatch);
        }).forEach(matchingResult -> {
            String modifiedTitle = matchingResult.getTitle().replace(prefixToMatch, replaceWithToUse);
            matchingResult.setTitle(modifiedTitle);
        });
    }
}
TitlePrefixPluginSearchLifeCyclePluginTest.java
package com.example.plugin.titleprefixplugin;

import com.funnelback.publicui.search.model.padre.Result;
import com.funnelback.publicui.search.model.transaction.testutils.TestableSearchTransaction;
import org.junit.Assert;
import org.junit.Before;
import org.junit.Test;

import java.util.List;
import java.util.Set;
import java.util.stream.Collectors;

/**
 * Note: This plugin is explained in detail as part of the 'Developing your first plugin' guide in the Funnelback product documentation.
 */
public class TitlePrefixPluginSearchLifeCyclePluginTest {

    TitlePrefixPluginSearchLifeCyclePlugin pluginUnderTest = new TitlePrefixPluginSearchLifeCyclePlugin();
    TestableSearchTransaction testInput;

    @Before
    public void setup() {
        // Create some test data.
        testInput = new TestableSearchTransaction()
            .withResult(Result.builder().title("ExampleCorp - Fake search data to take home").build())
            .withResult(Result.builder().title("ExampleCorp - Another result with the same prefix").build())
            .withResult(Result.builder().title("This result does not match the pattern").build());
    }

    @Test
    public void testPostProcess_with_replacement_pattern() {
        testInput
            .withProfileSetting("plugin.title-prefix-plugin.pattern", "ExampleCorp -")
            .withProfileSetting("plugin.title-prefix-plugin.replaceWith", "EC |");
        pluginUnderTest.postProcess(testInput);
        Set<String> expectedResultTitles = Set.of(
            "EC | Fake search data to take home",
            "EC | Another result with the same prefix",
            "This result does not match the pattern");
        List<Result> mockResults = testInput.getResponse().getResultPacket().getResults();
        Set<String> actualResultTitles = mockResults.stream().map(finalResult -> finalResult.getTitle()).collect(Collectors.toSet());
        Assert.assertEquals("the pattern should have been replaced with the new one", expectedResultTitles,actualResultTitles);
    }

    @Test
    public void testPostProcess_without_replacement_pattern(){
        testInput.withProfileSetting("plugin.title-prefix-plugin.pattern", "ExampleCorp - ");
        pluginUnderTest.postProcess(testInput);
        Set<String> expectedResultTitles = Set.of(
            "Fake search data to take home",
            "Another result with the same prefix",
            "This result does not match the pattern");
        List<Result> mockResults = testInput.getResponse().getResultPacket().getResults();
        Set<String> actualResultTitles = mockResults.stream().map(finalResult -> finalResult.getTitle()).collect(Collectors.toSet());
        Assert.assertEquals("the pattern should have been removed", expectedResultTitles, actualResultTitles);
    }
}

Example: clean-title-plugin

Removes a pattern from a title.

CleanTitleSearchLifeCyclePlugin.java
package com.example.cleantitle;

import com.funnelback.plugin.SearchLifeCyclePlugin;
import com.funnelback.publicui.search.model.transaction.SearchTransaction;
import com.funnelback.publicui.search.model.padre.Result;

/**
 * This example explains how we can use the `SearchLifeCycle` plugin to modify the title of a document
 * during the `postDataFetch` phase. The title of the documents will be modified after gathering finished.
 */
public class CleanTitleSearchLifeCyclePlugin implements SearchLifeCyclePlugin {

    @Override
    public void postDatafetch(SearchTransaction transaction) {
        String regex = transaction.getQuestion().getCurrentProfileConfig().get(PluginUtils.KEY_PREFIX + "regex-pattern");
        if (transaction.hasResponse()) {
            if (transaction.getResponse().hasResultPacket()) {
                transaction.getResponse().getResultPacket().getResults().forEach(r -> cleanTitle(r, regex));
            }
        }
    }

    protected Result cleanTitle(Result result, String regex) {
        String title = result.getTitle();
        title = title.replaceAll(regex, "");
        result.setTitle(title);
        return result;
    }

}
CleanTitleSearchLifeCyclePluginTest.java
package com.example.cleantitle;

import org.junit.Assert;
import org.junit.Test;
import com.funnelback.publicui.search.model.padre.Result;

public class CleanTitleSearchLifeCyclePluginTest {

    @Test
    public void postDatafetchTest(){
        CleanTitleSearchLifeCyclePlugin underTest = new CleanTitleSearchLifeCyclePlugin();
        String regex = "(- Funnelback Documentation - Version )?\\d+\\.\\d+\\.\\d+(-SNAPSHOT)?";
        Result result = new Result();
        result.setTitle("Funnelback documentation - Funnelback Documentation - Version 15.25.2007");
        Result cleanResult = underTest.cleanTitle(result, regex);

        Assert.assertEquals("Title of the document should be \"Funnelback documentation\"","Funnelback documentation ", cleanResult.getTitle());
    }
}