Types of plugins

Plugins allow you to package up functionality that includes:

  • logic that will run at the same time as executing a search package’s hook scripts

  • filters that can be added to a data source filter chain

  • custom gatherers

  • logic to determine additional start URLs

Some plugins will combine a few of these together.

When do plugins run?

A plugin will run at different times depending on the type of plugin. These can include:

  • At the start of a custom gather, to provide documents to be indexed

  • During indexing, to provide metadata or do other processing.

  • When a query is executed by a user.

Push data sources

When working with a push data source there are a few caveats:

  • If you are using a plugin that requires a filter to be run must, then you must set the filter classes when calling the API - if you don’t set the filter classes then the plugins you have enabled won’t work. This includes plugins that use Jsoup filters, which require the built-in JsoupProcessingFilterProvider to be called.

  • Push data sources will run all plugins at a regular interval that is determined by the time it takes to run them all. They will also all be re-run when executing a re-index. Push data sources will apply the data from the last successful plugin execution to any new documents committed.