Types of plugins
Plugins allow you to package up functionality that includes:
-
logic that will run at the same time as executing a search package’s hook scripts
-
filters that can be added to a data source filter chain
-
custom gatherers
-
logic to determine additional start URLs
Some plugins will combine a few of these together.
When do plugins run?
A plugin will run at different times depending on the type of plugin. These can include:
-
At the start of a custom gather, to provide documents to be indexed
-
During indexing, to provide metadata or do other processing.
-
When a query is executed by a user.
Push data sources
When working with a push data source there are a few caveats:
-
If you are using a plugin that requires a filter to be run must, then you must set the filter classes when calling the API - if you don’t set the filter classes then the plugins you have enabled won’t work. This includes plugins that use Jsoup filters, which require the built-in
JsoupProcessingFilterProvider
to be called. -
Push data sources will run all plugins at a regular interval that is determined by the time it takes to run them all. They will also all be re-run when executing a re-index. Push data sources will apply the data from the last successful plugin execution to any new documents committed.