warc.compression

Control how content is compressed in a WARC file.

Key: warc.compression
Type: WarcCompressionTypes allowed values: contentonly or none
Can be set in: collection.cfg

Description

This option controls how content will be compressed in a WARC file. Allowed values are none and contentonly (the default).

WARC files are used to store gathered content for indexing.

warc.compression=contentonly

The default is to compress content only but not the WARC headers.

To turn off all compression (e.g. for debugging purposes):

warc.compression=none