Command line administration

This feature is not available to users of the Squiz Experience Cloud version of Funnelback.

The recommended method to administer a Funnelback installation is through the web based administration interface. This provides an easy-to-use frontend for administrators. However, it is also possible to administer Funnelback from the command line. This is useful if other systems need to be integrated with Funnelback.

Locations

We assume in the following instructions that the SEARCH_HOME environment variable is defined. This should point to your installation directory. By default, this is /opt/funnelback/ on Linux and C:\funnelback\ on Windows.

$SEARCH_HOME/bin/

Contains all administration scripts.

$SEARCH_HOME/conf/

Contains various global configuration files, as well as search package and data source specific configuration files, under $SEARCH_HOME/conf/<SEARCH-PACKAGE-or-DATA-SOURCE-ID>.

$SEARCH_HOME/log/

Contains global log files, such as the create.log file, which records creation of collections and the delete.log file, which records deletion of collections.

$SEARCH_HOME/web/

Contains files relating to the admin console and public search interface (such as the cgi files). Web server configuration files are stored in $SEARCH_HOME/web/conf/.

$SEARCH_HOME/data/

Contains collection specific data, such as gathered documents, indexes and log files.The data area has the following structure:

  • Each search package and data source will have a sub-directory under data, containing live, offline, log and archive directories.

  • The archive directory contains compressed query and click log files for the collection.

  • The log directory contains logs that don’t fit into any other category, such as update logs and reporting logs.

  • The live and offline directories contain gathered and filtered documents (in "data"), indexes (in "idx") and collection specific logs (in "log").

Command line scripts reference

Detailed internal documentation may be gained for many scripts through the standard Perl "perldoc" command.

Creates a collection, including its collection.cfg file.

new-collection.pl <search-package-or-data-source-id> <collection type> [start url]

Creates a collection from an existing collection.cfg file.

create-collection.pl <collection config>

Deletes a collection, including its gathered documents, indexes, configuration, scheduled updates and logs. It also removes references to the now non-existent collection from user configuration files.

delete-collection.pl <collection config>

Update.pl is a wrapper around the entire update process, and calls the appropriate update subscripts.

update.pl <collection config> [update type: -incremental, -reindex, ...]

Gathers documents from web collections.

crawl.pl <collection config> [update type: -check, -incremental, -instant-update]

Gathers documents from filecopy collections.

filecopy.pl <collection config> [other options]

Gathers documents from database collections.

dbgather.pl <collection config> [--full] [other options]

Calls Padre to index a collections documents.

index.pl <collection config> [-reindex] [-instant-update]

Processes a collections data files, producing reports on their contents.

make_report.pl [--collection "collection config"] [--log] ...

Updates the trend alerts reports for a collection (or all collections if none is specified).

outliers-log-processing.pl [--collection "collection name"]

Swaps the live and offline views of a collection after a successful update, placing the newly gathered and indexed data in live for querying, and safely storing the older gathered and indexed data in offline.

swap-views.pl <collection config> [-force]

Archives a collections queries.log and clicks.log log files to the collection’s archive directory.

archive-log.pl <collection config> [view]

Reads a collections log files and stores a binary database for reporting purposes. The admin UI report frontend will read this database for displaying reports.

reports-load-queries-log.pl [--collection "collection-id"]

Sends email query reports to users who have requested them for the specified collection (or for all if none was specified).

reports-send-email.pl [--collection "collection-id"]

Updates the location of the perl interpreter for all .cgi and .pl scripts.

modify_perl_hashbang_line.pl

Trigger local or remote administrative tasks.

mediator.pl --help

Changes a user’s password.

$SEARCH_HOME/web/bin/change_password.sh <user> <password>

Managing encrypted passwords

Funnelback encrypts passwords within collection.cfg in an encrypted form to minimise the risk of exposing the passwords. Administrators may, however, need to be able to encrypt and decrypt passwords manually, and this can be achieved by using the following commands.

Within these examples Funnelback is assumed to be installed at /opt/funnelback and the example password used is secret - Please adjust the commands to suit your specific circumstance.

Please be aware that:

  • Funnelback will generate a unique encryption key on installation, so your encrypted passwords will not match this example.

  • The commands themselves may change between Funnelback versions and hence they may not be appropriate for use in long-lived scripts.

Encrypting

> GROOVY_TURN_OFF_JAVA_WARNINGS=true JAVA_HOME=/opt/funnelback/linbin/java/ /opt/funnelback/tools/groovy/bin/groovy -cp "/opt/funnelback/lib/java/all/*" -e "println new com.funnelback.config.security.ConfigPasswordEncryptionService(new File('/opt/funnelback')).encryptPassword(new com.funnelback.config.types.ConfigPasswordUnencrypted('secret')).getEncryptedPassword();"

ENCRYPTED:AU4zJO2wUnoORBxsutBDZr1tnbUFqt7WAKd8vmzIDJm3hxnmbgHi

Decrypting

> GROOVY_TURN_OFF_JAVA_WARNINGS=true JAVA_HOME=/opt/funnelback/linbin/java/ /opt/funnelback/tools/groovy/bin/groovy -cp "/opt/funnelback/lib/java/all/*" -e "println new com.funnelback.config.security.ConfigPasswordEncryptionService(new File('/opt/funnelback')).decryptPassword(new com.funnelback.config.types.ConfigPasswordEncrypted('ENCRYPTED:AU4zJO2wUnoORBxsutBDZr1tnbUFqt7WAKd8vmzIDJm3hxnmbgHi')).getCleartextPassword();"

secret

© 2015- Squiz Pty Ltd