Troubleshooting techniques

Isolating Funnelback

It is not uncommon for Funnelback search results to be called indirectly by another system. This commonly occurs if the search results are fetched as JSON or XML or as a HTML fragment that is nested into a page within a CMS.

You can tell that the search results are being nested if the URL that you are accessing when viewing the search results doesn’t access one of Funnelback’s public search endpoints (/s/search.html, /s/search.xml, /s/search.json).

If you have access to the system that is wrapping the search (e.g. a Squiz Matrix REST asset) then check the URL that is being requested. It will be something similar to https://FUNNELBACK-SERVER/s/search.html or https://FUNNELBACK-SERVER/s/search.json with some parameters.

Determine exactly what URL is being called by the system and then take this URL and access it from your web browser. Once you have this response you can inspect it and confirm if there is an error in the response. If Funnelback’s response looks correct then the error is in the code that processes the Funnelback response and needs to be addressed by the site owner.

If the problem is reflected in Funnelback’s response then you can continue your investigations.

View the Funnelback data model

Viewing the underlying data model for a query is a vital technique for successfully debugging query time-issues.

The data model is accessed by viewing either the search.json or search.xml endpoints. The JSON endpoint is more useful because it also indicates the type of each field.

Viewing the data model is useful because it shows the raw values available for each element. If there is a problem present in the data model then it eliminates the template as being the problem. Note that hook scripts can modify the data model, so it is often a good idea to disable them when testing (see below).

The data model endpoints are worth checking where any type of template is involved (including any Freemarker ftl templates that return custom JSON/XML) and also when the JSON or XML is consumed directly by another system (which may break something when consuming the data).

If you’re using Freemarker templates, try using the default template

Attempting to view the search results using the default template is a good way of determining if there is an issue in a custom Freemarker template.

The default template (simple.ftl) is usually customized as part of an implementation. In order to test with the default template you’ll need to create an additional template for your results page and copy in the default template code. e.g. create a new template called default.ftl and then copy in the default template code. To get the default template code you’ll need to create an additional results page and the copy the template code from there. You can then delete that new results page.

Once you’ve setup a template (e.g. default.ftl) containing the default template code you can then test the same query but using the default template by setting &form=default in the query.

View the stored source document

If you are experiencing a content related issue viewing the source code of the stored document can be very useful in assisting to pinpoint an issue.

The simplest way to view the source document is to view the cached version of it, either by directly selecting view cached version from the search results (if this is exposed in the template), or by extracting the cache URL from the data model and then accessing this URL.

This will show you the content of the source document - which is what Funnelback scanned when creating the index.

This will sometimes be different from what you see when you access the live url. Reasons for this could be:

  • The content has changed since Funnelback gathered the document.

  • A filter that processes the document has modified it (and there may be an error in the filter).

  • The content may be returned to Funnelback in a different format to what a connected user sees (for example because of Javascript in the page that alters the page DOM in memory or the content that it includes, because Funnelback is crawling with a different user agent, because Funnelback is crawling as a specific user).

Understand what to investigate

When troubleshooting a search-related problem it is important to be able to figure out whether you need to inspect the search package, or one of the results pages or data sources.

If you are investigating a data source update failure then it will be obvious where you need to look. However, if the issue is showing up in the search results then things become a bit more complicated.

If the problem relates to search result formatting, synonyms, best bets, curator rules, search sessions or any other front-end related feature then investigate the search package and results page that is indicated by the collection and profile parameters in the search query.

If the issue relates to search result content (such as missing metadata) then you will need to check the data source from where the search result has been indexed. If you’re not sure what this is you can view the collection property of the individual search result in the search data model. This will tell you which data source the result belongs to.

Understand which view to investigate

Understanding the correct view to inspect is also very important when troubleshooting problems.

If the issue is relating to search results seen by users or an update that was successful then inspect the live view of a data source.

If the issue relates to an update that failed, or you want to find out information about an update that is currently running, or you are interested in the previously successful update then you need to inspect the offline view.

the state of the offline view will depend on if an update of the data source is currently running.

If the offline view contains the previously successful update then you can revert the search by swapping the live and offline views. This is achieved by running an advanced update on the affected data source and selecting the swap live and offline views option.

This can occasionally solve a problem where a change caused an update to omit required content. Swapping the views makes the previously live indexes live again and allows you to start off another update.

Funnelback support packages

Funnelback includes an option within the search dashboard to export a support package.

The generated support package bundles up the global configuration, specific configuration and various logs into a file that can be supplied along with a support request.

Using the support package

Support packages are supplied as .zip files. When decompressed these contain the same folder structure as a standard Funnelback instance but only including the files relevant to the exported support package.

MacOS Finder sometimes has problems decompressing the support package zip file. If you have this problem you can use the unzip command from a MacOS terminal to decompress the file.

This can be decompressed and inspected to diagnose issues and in many cases can be transferred onto a local Funnelback installation allowing the search to be mirrored locally and tested.

Caveats:

  • Always make sure any testing is performed on the same version of Funnelback as the installed version (unless you are attempting to test if a newer version solves an issue).

  • It may not be possible to create your own mirror of the installed search if the content included in the data sources is private or the configuration accesses private content.