Help Center

Menu

YouTube data source

The YouTube data source is used to index content sourced from the YouTube video platform.

Please note that your usage of Funnelback to gather content from YouTube must comply with YouTube’s terms of service.

Getting an API key

To be able to gather content from YouTube you must first use your Google account to create an API key. Google provides instructions on creating an API key which will provide the most up-to-date instructions for creating the key.

Configuration options

YouTube data sources support the following configuration options:

Standard options

Option Description

Advanced options

Option Description

Filter options

Option Description

Discarding old items

Items older than a certain date can be discarded by enabling the date filter plugin and configuring it to discard the older items.

User mentions and hash-tags

User mentions and hash-tags within YouTube content can be made searchable by enabling the social tags plugin.

Metadata mappings

YouTube data sources includes a number of default YouTube specific metadata mappings:

Class ID Type Behaviour Explanation Metadata fields included

author

text

content

/com.funnelback.socialmedia.youtube.v3.YouTubeXmlRecord/channelTitle

authorUrl

text

display

/com.funnelback.socialmedia.youtube.v3.YouTubeXmlRecord/channelUrl

c

text

content

Description

/com.funnelback.socialmedia.youtube.v3.YouTubeXmlRecord/description

category

text

content

/com.funnelback.socialmedia.youtube.v3.YouTubeXmlRecord/category

commentCount

number

N/A

/com.funnelback.socialmedia.youtube.v3.YouTubeXmlRecord/commentCount

d

date

date

Date

/com.funnelback.socialmedia.youtube.v3.YouTubeXmlRecord/publishedDate

dislikeCount

number

N/A

/com.funnelback.socialmedia.youtube.v3.YouTubeXmlRecord/dislikes

duration

text

display

/com.funnelback.socialmedia.youtube.v3.YouTubeXmlRecord/durationPretty

durationInSeconds

number

N/A

/com.funnelback.socialmedia.youtube.v3.YouTubeXmlRecord/durationInSeconds

favoriteCount

number

N/A

/com.funnelback.socialmedia.youtube.v3.YouTubeXmlRecord/favoriteCount

identifier

text

display

/com.funnelback.socialmedia.youtube.v3.YouTubeXmlRecord/videoId

imageSmall

text

display

Image URL - small

/com.funnelback.socialmedia.youtube.v3.YouTubeXmlRecord/thumbnails/veryDefault/thumbNailUrl

imageMedium

text

display

Image URL - medium

/com.funnelback.socialmedia.youtube.v3.YouTubeXmlRecord/thumbnails/medium/thumbNailUrl

imageLarge

text

display

Image URL - large

/com.funnelback.socialmedia.youtube.v3.YouTubeXmlRecord/thumbnails/high/thumbNailUrl

latLong

geospatial x/y co-ordinate

N/A

/com.funnelback.socialmedia.youtube.v3.YouTubeXmlRecord/latLong

likeCount

number

N/A

/com.funnelback.socialmedia.youtube.v3.YouTubeXmlRecord/likes

t

text

content

Title

/com.funnelback.socialmedia.youtube.v3.YouTubeXmlRecord/title

viewCount

number

N/A

/com.funnelback.socialmedia.youtube.v3.YouTubeXmlRecord/viewCount

Use the -SF query processor option to access these metadata fields on the search response and in the templates (i.e. -SF=[author,viewCount]).

Limits

Please note that YouTube applies limits to the volume of content which can be retrieved from their APIs, and so in the case of large channels Funnelback may be unable to gather all historical content.

Working with the fetched data

Funnelback will crawl YouTube and convert responses into XML. You can use the metadata customization tool to map elements to metadata classes.

To preview the gathered records please enable debug mode by setting the youtube.debug=true data source configuration option. This will dump the raw XML data into the logs.

The XML that Funnelback generates for a YouTube data source is as follows:

<com.funnelback.socialmedia.youtube.v3.YouTubeXmlRecord>
  <url>http://www.youtube.com/watch?v=video_id&amp;feature=youtube_gdata_player</url>
  <title>some title</title>
  <description>some description</description>
  <videoId>video_id</videoId>
  <viewCount>1</viewCount>
  <commentCount>0</commentCount>
  <thumbnails>
    <veryDefault>
      <thumbNailUrl>https://i.ytimg.com/vi/video_id/default.jpg</thumbNailUrl>
      <thumbNailUrlWidth>120</thumbNailUrlWidth>
      <thumbNailUrlHeight>90</thumbNailUrlHeight>
      <thumbNailUrlExtras class="linked-list"/>
    </veryDefault>
    <medium>
      <thumbNailUrl>https://i.ytimg.com/vi/video_id/mqdefault.jpg</thumbNailUrl>
      <thumbNailUrlWidth>320</thumbNailUrlWidth>
      <thumbNailUrlHeight>180</thumbNailUrlHeight>
      <thumbNailUrlExtras class="linked-list"/>
    </medium>
    <high>
      <thumbNailUrl>https://i.ytimg.com/vi/video_id/hqdefault.jpg</thumbNailUrl>
      <thumbNailUrlWidth>480</thumbNailUrlWidth>
      <thumbNailUrlHeight>360</thumbNailUrlHeight>
      <thumbNailUrlExtras class="linked-list"/>
    </high>
    <stndard>
      <thumbNailUrl>https://i.ytimg.com/vi/video_id/sddefault.jpg</thumbNailUrl>
      <thumbNailUrlWidth>640</thumbNailUrlWidth>
      <thumbNailUrlHeight>480</thumbNailUrlHeight>
      <thumbNailUrlExtras class="linked-list"/>
    </stndard>
    <maxres>
      <thumbNailUrl>https://i.ytimg.com/vi/video_id/maxresdefault.jpg</thumbNailUrl>
      <thumbNailUrlWidth>1280</thumbNailUrlWidth>
      <thumbNailUrlHeight>720</thumbNailUrlHeight>
      <thumbNailUrlExtras class="linked-list"/>
    </maxres>
  </thumbnails>
  <publishedDate>2016-08-16T23:14:04.000Z</publishedDate>
  <category/>
  <durationInSeconds>378</durationInSeconds>
  <durationPretty>00:06:18</durationPretty>
  <likes>1</likes>
  <dislikes>0</dislikes>
  <favoriteCount>0</favoriteCount>
  <channelTitle>channel_name</channelTitle>
  <channeld>channel_id</channeld>
  <channelUrl>https://www.youtube.com/channel/channel_id</channelUrl>
  <embedHtml>&lt;iframe width=&quot;480&quot; height=&quot;270&quot; src=&quot;//www.youtube.com/embed/video_id&quot; frameborder=&quot;0&quot; allow=&quot;autoplay; encrypted-media&quot; allowfullscreen&gt;&lt;/iframe&gt;</embedHtml>
  <extras class="linked-list"/>
</com.funnelback.socialmedia.youtube.v3.YouTubeXmlRecord>

See also