Manage a data source
The data source management screen of the search dashboard provides access to all the configuration and management tools for a data source.
This screen is accessed by clicking on a data source name or selecting the configuration menu option from a search package or manage data source listing.
The data source management screen is composed of several regions, which perform different functions. Some of these are highlighted below:
-
Data source information: This panel provides information such as the data source ID, number of indexed documents and information relating to updates of the data source.
-
Data source settings: Provides access to all the configuration for the data source.
-
Data source tools: Maintenance tools and log files for the data source.
-
On-demand updates: On-demand updates to the data source index.
-
Scheduled tasks: Scheduled updates to the data source index.
-
Delete data source: Deletes the data source. Use with caution.
-
Data source plugins: Lists the plugins that are enabled on this data source and enables you to edit the configuration of those plugins.
-
Search packages: Lists the search packages that this data source is attached to.
-
Data source indicator: Indicates that you are currently managing a results page.
Data source information
The data source information panel provides information about the data source:
- Data source type
-
The type of data source.
- Data source ID
-
The data source ID used to identify the data source.
- Status
-
Provides the current update status of the data source. If an update is running the status will provide information about what the update is currently doing, otherwise it will indicate if the update was successful, failed (or has never been updated). The search dashboard shows various statuses for a data source, with the main ones being:
-
Update complete: Indicates that the last update completed successfully. This is what you want to see when viewing your statuses.
-
Update required: Indicates that the data source has been set up, but it has not yet been updated. The last successful update column will also display a value of never. To start an update select one of the options from the update menu.
-
Update failed: Indicates that something went wrong during the update. Errors were logged and the search index was not updated. Errors can sometimes clear without any intervention (e.g. a website you were crawling might have been unavailable when the update ran) or there may be other problems that require intervention before a successful update will run. See: Search dashboard shows an update failed status for a data source for guidance on troubleshooting a failed update.
-
N/A: (not applicable) Displayed for push data sources as these don’t follow a normal update process.
-
Other status messages are displayed during a running update.
-
- Number of documents
-
This is the number of documents included within the data source index. This number counts towards the document count of your license.
- Last successful update
-
This is the time when the last successful update of the data source was started.
- Next scheduled update
-
This is the time when the next scheduled update is expected to commence.
Data source settings
The data source settings panel provides access to all the different configuration screens for the data source. The list of available options will vary depending on what type of data source you are managing.
- Edit data source configuration
-
Manage the advanced configuration settings for this data source. This is where you set the data source’s individual configuration settings.
- Browse data source files
-
Manage common data source configuration files. The specific configuration files will vary depending on what type of data source you are managing.
- Configure metadata mappings
-
Manage the metadata field mappings for this data source. This is where you configure what metadata is extracted from your data source.
- Configure XML processing
-
Manage the XML processing settings for this data source. This is where you configure how any XML documents in your data source are handled. This includes indicating XML paths used to split the XML file into individual records and an XML field that indicates a URL for the record.
- Configure site profiles
-
(web data source only) Configures a set of site/domain-specific settings for a web data source. This includes settings for (basic HTTP) username and password authentication, limiting the number of stored URLs, and the number of URLs on a domain you can request in parallel.
Data source tools
- Collection tools
-
Access the monitor graphs (web, file system and TRIM data sources only) and lookup information about a URL in the index.
- Browse log files
-
Access all the log files relating to the data source.
- Download support package
-
Download a compressed file that includes all the configuration and logs for the data source.
On-demand updates
Run an on-demand update of the data source. The on-demand update will be added to the server’s task queue and will run as soon as resources become available.
- Update this data source
-
Clicking this will start a standard update of the data source.
- Start advanced update
-
Provides a set of advanced updates that can be run for the data source. This includes restarting a failed update from a specific point, instant updates and special types of index updates to apply gscopes or kill documents.
- Stop update
-
If an update of the data source is running this can be clicked to stop the update.
Scheduled tasks
- Schedule an update
-
This is used to schedule an automatic update of the data source. Updates can be scheduled on a periodic cycle or to be added to the task queue at specific fixed times.
- View scheduled tasks
-
Lists the scheduled tasks for the data source.