Restricts the file extensions the web crawler should crawl.

Key: crawler.accept_files
Type: List<String>
Can be set in: collection.cfg


This is a comma-separated list of file extensions that will be downloaded by the crawler. It is normally left empty, so that the crawler will accept all valid content regardless of the suffix.

Default Value

(Empty) - This means there are no restrictions on what files will be downloaded.



In this example a specific list of filetypes (based on suffix) is listed - only files of these types will be downloaded.