Scheduling collection updates
Scheduling updates interface
Funnelback provides two scheduling interfaces, one oriented towards scheduling collection updates for a Linux platform, and one oriented towards scheduling collection updates for a Windows system:
Scheduling updates on Linux
The 'Schedule updates' page allows you to specify regular times at which collections should be updated. The scheduling system will be familiar to users of the Unix cron tool (which may be used directly if additional flexibility is required). Note that care is required when scheduling updates to ensure that simultaneous updates to several collections do not cause excessive load on the search server.
In the scheduling interface you can specify a number of parameters to control when updates occur. This can include the hours of the day, days of the week and days of the month on which updates should occur for each collection.
- In each of the input boxes you can enter:
- single values: such as 17, Fri or Any.
- range values: such as 9-17 or Mon-Fri
- several values: a comma separated list of single items, such as
- Take care when specifying both days of the week and the days of the month for a collection. Normally one of these columns should be specified as
Any. If an update is specified with both a day of the week and a day of the month (e.g. Tuesday and the 14th day of the month), that update will occur only when both constraints are satisfied (e.g. on Tuesdays which are also the 14th of the month).
- Do not overlap large collection updates.
- Allow sufficient time for update completion. If an update normally takes two days, there is no point in trying to re-update every hour or even every day.
- Note that if you specify 29, 30 or 31 in the day of the month column no action will be taken in months which do not contain those days.
To update every Weekday at 9am and 5pm:
- Hour of the Day = 9,17
- Day of the Month = Any
- Day of the Week = Mon-Fri
To update every fortnight:
- Hour of the Day = 3
- Day of the Month = 1,14
- Day of the Week = Any
To update every hour of a weekend:
- Hour of the Day = 0-23
- Day of the Month = Any
- Day of the Week = Sat,Sun
Scheduling on Windows platforms
The schedule updates page allows administrators to specify regular update times for collections. The interface should be familiar to users of the built-in Windows schedule tasks tool, and tasks created using the schedule updates page will also be available through the standard Windows scheduled tasks interface. Note that care is required when scheduling updates to ensure that simultaneous updates to several collections do not cause excessive load on the search server.
When scheduling an update, an administrator enters:
- A collection to update.
- A valid Windows username and password (required by Windows). This needs to be an account who can schedule tasks on the server, so in practice it's often a local machine account rather than a domain account.
- A schedule type: either daily, weekly or monthly.
- Daily schedule specifies that the update should run every day, and requires no additional parameters.
- Weekly schedule specifies that the update should run on a weekly basis, and also requires one or more days to be specified.
- Monthly schedule specifies that the update should run on a monthly basis, and also requires a day of the month to be specified. Care should be taken when specifying a day of the month: days such as "31" do not occur in all months, and scheduled updates will not run for months which do not contain the day specified.
- A time to update the collection at, either AM or PM (AM by default). Time is of the format HH:MM.
The user must have the certain access rights on the server in order to be able to schedule tasks:
The following policy Network access: Do not allow storage of passwords and credentials for network authentication must be disabled in order for the user to be able to schedule a task. If the user does not have this right the following error is displayed when attempting to schedule an update:
ERROR: A specified logon session does not exist. It may already have been terminated.
The user must be added to the list of users that have the Log on as a batch job privilege. If a user doesn't have this right the following error is displayed when attempting to schedule an update:
WARNING: The task is registered, but may fail to start. Batch logon privilege needs to be enabled for the task principal.
Ensure the user that runs the task has a non-expiring password otherwise the task will need to be updated every time the password is changed.
Manually scheduling updates on Windows
To manually schedule a collection to automatically update under Windows, use any of the built in Windows schedulers (
Task Scheduler) to run the following command:
C:\Perl\bin\perl.exe %SEARCH_HOME%\bin\update-win.pl %SEARCH_HOME%\conf\collection_name\collection.cfg
%SEARCH_HOME% is the path to the Funnelback files (eg.
C:\Funnelback\search) and collection_name is the internal name of the collection you wish to update.
Create a scheduled task
Use the Windows shell to create a new scheduled task, using the built-in
Scheduled Tasks program.
Give the new scheduled task a name (e.g. "Update My Collection") and open the properties sheet for the new task.
Then perform the following:
- Enter the command given above in the 'Run' field.
- Enter %SEARCH_HOME%\bin in the 'Start In' field.
- Make sure the 'Run As' field is set to the 'funadmin' user (or another user with permission over Funnelback files)
- Click 'Set password...' and enter the 'Run As' users password.
- Use the 'Schedule' tab to set the appropriate time for the update to occur.
- Click 'Apply' and then 'OK'.
Upgrading scheduled updates
When Funnelback is upgraded, paths such as the
SEARCH_HOME may change. When this happens, old scheduled tasks that depended on the old
SEARCH_HOME need to be updated.
Funnelback attempts to automatically update Funnelback specific scheduled tasks during an upgrade, on Unix and Windows. If this process fails, a warning message will be displayed.
Administrators may run the scheduled tasks update script to fix this problem automatically. The script is located in
$SEARCH_HOME/bin/setup/. On Linux it requires the arguments
new_sh (which should point to the new installation of Funnelback, eg:
old_sh (which should point to the old installation of Funnelback, eg:
/usr/local/fb). On Windows, the
old_sh arguments are required, as are
pass. The username and password provided will be used to schedule updates, and scheduled updates will run as the given user.
Alternatively, administrators may update scheduled tasks by hand. If they are sure that all updates are correct, the file
$SEARCH_HOME/log/fix_tasks_failure.txt can be deleted to disable the warning message.