Custom data sources

Custom data sources are a generic data source type that allow the indexing of content from arbitrary sources such as the gathering of content via API, or programatically via Java libraries or an SDK. The gathering logic for a custom collection uses a plugin or a custom gather script that must be supplied before the collection is usable.

Creating a custom data source

To create a new custom collection follow the data source creation guide selecting custom as the data source type.

Configuring a custom data source

A custom data source is basically a container collection that contains no gathering logic.

The data source’s gathering logic is implemented in one of the following ways:

  • Using a custom gatherer included as part of a Funnelback plugin.

  • Using a custom_gather.groovy configuration file that directly implements the custom gather logic in Groovy.

The custom gatherer used by a custom data source must be configured before the custom collection can be used.

Using a custom gatherer sourced from a plugin

  1. Enable the plugin containing the custom gatherer on the custom data source

  2. Follow any additional configuration steps as outlined in the plugin’s readme file.

If enabling more than one plugin containing a custom gatherer ensure that the plugin.force-gatherer is used to define which custom gatherer to use.

Using a custom gatherer source from custom_gather.groovy

custom_gather.groovy is deprecated and is not available when using Funnelback SXC.

Configuring filters

The custom gatherer can then feed the output through Funnelback’s filter framework which can be used to convert JSON/CSV to XML as detailed above.

Filters can be sourced from:

© 2015- Squiz Pty Ltd