Net error: read timed out

This error occurs when a request for a URL times out.

Error message

Displayed in the url_errors.log file

E http://example.com/path/example.html [Net Error: Read timed out] [2014:07:31:09:10:51]

Cause

This indicates that the web crawler received a timeout when attempting to fetch the URL.

Resolution

  • Ensure that the Funnelback server can access the page (e.g. use the DEBUG admin API call to test the failing URL). If the page cannot be accessed in this way there could be a network connection issue or Funnelback could be being blocked by a firewall or similar.

  • Increase the crawler.request_timeout setting in the data source configuration (default is 15000 ms).

  • Enable timeout retires by setting crawler.max_timeout_retries in the data source configuration.