Plugin: XML date filter
|Other versions of this plugin may exist. Please ensure you are viewing the documentation for the version that you are currently using. If you are not running the latest version of your plugin we recommend upgrading. See: list of all available versions of this plugin.|
Excludes documents in an XML data source based on a date/time contained in the XML data.
The most common use case for this filter plugin is to exclude social media posts older than a given date from being included in the search index.
Enable the plugin
Enable the xml-date-filter plugin on your data source from the Extensions screen in the administration dashboard or add the following data source configuration to enable the plugin.
|This plugin requires a full update of the data source to take effect.|
Plugin configuration settings
XmlDateFilter filter must be added to the filter chain for the plugin to work correctly:
Add the filter to the
filter.classes in the data source configuration.
|The filter should be placed at an appropriate position in the filter chain. In most circumstances this should be located towards the end of the filter chain.|
The following option must be set in the data source configuration to configure the plugin:
plugin.xml-date-filter.config.unit: Specifies the unit of time used to calculate if the document should be filtered and is required. Valid values are 'YEARS', 'MONTHS', 'DAYS', 'HOURS', 'MINUTES'.
plugin.xml-date-filter.config.amount: Specifies the amount of units above used to calculate if the document should be filtered and is required.
The following option must be set if you are not using the built-in 'Facebook', 'YouTube' or 'Twitter' data source types:
plugin.xml-date-filter.config.record_type: Specifies the type of custom XML data source. Valid values are 'instagram' (when using the Stencils Instagram gatherer) or 'custom'.
The following options must be set if
plugin.xml-date-filter.config.record_type is set to
plugin.xml-date-filter.config.date_element: Specifies the XML element name (located at the root of the record’s XML) that contains the date/time value to be used for filtering.
plugin.xml-date-filter.config.date_format: Specifies the format of the date/time value in the XML element that contains the date/time value. Value must be a valid Java date format string.
Example - exclude items older than 30 days
For a custom XML record:
<item> <title><![CDATA[Example record]]></title> <timestamp>2000-12-24T04:35:21+1100</timestamp> <description><![CDATA[Example description]]></description> </item>
Exclude records older than 30 days from a custom XML data source using the 'timestamp' element:
plugin.xml-date-filter.config.unit=DAYS plugin.xml-date-filter.config.amount=30 plugin.xml-date-filter.config.record_type=custom plugin.xml-date-filter.config.date_element=timestamp plugin.xml-date-filter.config.date_format=yyyy-MM-dd'T'HH:mm:ssZ