Crawler Classes Urlstore (collection.cfg)


The crawler stores downloaded documents on a local file system for them to be indexed. This option identifies the Java class to be used to store the documents.

The main store classes are:

Store cached documents in a single compressed WARC file.

Store cached documents using a mirror of their URL directory structure.

Default value

See also