Custom data sources

Custom data sources are a generic data source type that allow the indexing of content from arbitrary sources such as the gathering of content via API, or programmatically via Java libraries or an SDK. The gathering logic for a custom data source uses a plugin or a custom gather script that must be supplied before the data source is usable.

Creating a custom data source

To create a new custom data source follow the data source creation guide selecting custom as the data source type.

Configuring a custom data source

A custom data source uses a plugin gatherer which implements the logic required to connect to and gather content from the custom repository.

Available custom gatherer plugins:

Using a custom gatherer sourced from a plugin

  1. Enable the plugin containing the custom gatherer on the custom data source

  2. Follow any additional configuration steps as outlined in the plugin’s readme file.

If a custom data source imports multiple plugins that implement custom gathering then you will need to specify which gatherer to use by setting the data source configuration option, plugin.gather-with, to the ID of the plugin.

Configuring filters

The custom gatherer can then feed the output through Funnelback’s filter framework which can be used to analyze or transform the content. e.g. convert JSON/CSV to XML.

Filters can be sourced from: