Help Center

Menu

Query independent evidence (QIE)

Introduction

Query independent evidence (QIE) is used to assign rank weightings to documents, without consideration of the user’s query. For example, documents from a particular website may be up-weighted, while documents of a particular filetype may be down-weighted. QIE is configured through the qie.cfg file.

QIE can be used for a variety of purposes. For example:

  1. Externally computed page popularities, such as PageRanks can be used to promote popular pages.

  2. Weights can be associated with particular document formats — e.g. 1.0 for HTML, 0.5 for PDF and 0 for PPT. This can be used to discourage (but not prevent) the return of certain types of document.

  3. Weights associated with particular websites or part of sites can be used to bias results towards an important main site.

  4. Spam scores (0 for spam and 1 for non-spam) can be used to bias against (but not prevent) the display of spam results.

QIE configuration files

QIE configuration files are typically placed in:

$SEARCH_HOME/conf/<DATA-SOURCE-ID>/qie.cfg

The file-manager can be used to create and edit this file. Lines in the configuration file have the following format:

# comment line
qie_weight url_pattern

Where:

qie_weight is a floating point number (assumed normalised to the range 0-1, specifying the qie score to be applied). A QIE weight of 0.5 is the default. A larger value indicates an upweight, a smaller value indicates a downweight.

url_pattern is a perl5 syntax regular expression to be matched against the document’s indexed URL.

comment-line is an ignored line starting with a hash.

For example:

# down-weight pages from all states except Western Australia
0.25  ^(https://)?[^/]*nsw.gov.au/
1.0   ^(https://)?[^/]*wa.gov.au/
0.25  ^(https://)?[^/]*sa.gov.au/
0.25  ^(https://)?[^/]*nt.gov.au/

Each indexed URL is matched against every URL pattern, stopping at the first match. If none match, the default score passed to padre-qi is used.

PADRE strips "http:// " from URLs which start with that. Consequently, don’t include "http:// " in your URL patterns.

At indexing time if a data source has a qie.cfg file a supplemental index file will be produced that contains information on URLs to upweight and downweight.

QIE configuration is not automatically applied to all generations in a push data source. QIE configurations are applied to newly committed generations as well as merged generations. To re-apply QIE configurations to an entire Push collection you will need to trigger a Vacuum.

Configuring Funnelback to use QIE

Funnelback must be configured to use the QIE by configuring the influence of QIE on the ranking algorithm. This is set by assigning a weighting to the cool.4 ranking option.

See also