Social media collections

Introduction

Funnelback provides support for gathering content from a range of social media sources for inclusion within search results. Currently supported sources are:

Setting up a social media collection

Social media collections within Funnelback are created using Custom collections, allowing the social media gathering code to be customised where required. For each of the social media sites noted above, Funnelback includes custom gathering templates and default metadata mappings, providing a starting point for configuring the custom collection appropriately.

General gathering script format

The following code outlines the general structure of the social media gathering template scripts. Broadly:

The call to the execute method on the fetcher arranges to gather the results for each of the given queries, to convert each result into XML appropriate for Funnelback, and then to store this XML ready for indexing.

import com.funnelback.common.config.*;
import com.funnelback.common.io.store.xml.*;
import com.funnelback.socialmedia.fetcher.*;
import com.funnelback.socialmedia.processing.*;
import com.funnelback.socialmedia.SOCIAL_MEDIA_TYPE.*;
import com.funnelback.socialmedia.utils.*;
import java.net.URL;

// Any SOCIAL_MEDIA_TYPE specific login

def config = ConfigFactory.createNoOptionsConfigWithLogging(new File(args[0]), args[1]);

def processor = new CollectionStoreRecordProcessor(new XmlStoreFactory(config));

List<SOCIAL_MEDIA_TYPEQuery> queries = [
  // SOCIAL_MEDIA_TYPE specific Query objects
];

new SOCIAL_MEDIA_TYPEFetcher(processor, config).execute(queries);

Specific social media sites

The following sections provide additional detail on the gathering templates for each social media source, including the different types of queries which are supported for each.

See also