Plugin: Microdata

Purpose

Use this plugin to index microdata and JSON-LD as metadata.

This plugin will scan HTML pages for microdata and JSON-LD metadata that is embedded in the page.

It will then extract this metadata and convert it to basic html meta tags that are injected into the page header.

This plugin adds additional metadata fields to your HTML document. To use this metadata in your search you need to map these to metadata classes in your metadata mapping configuration.

Limitations

This plugin only supports HTML documents.
Metadata that is extracted from microdata and JSON-LD will be flattened when added to the document, which may results in the loss of some of the meaning.
The plugin only supports JSON-LD metadata that is embedded in the page. JSON-LD that is sourced from a remote location is not supported.
The plugin only provides a basic level of support for microdata and JSON-LD when parsing into metadata.

Microdata conversion

<div itemscope itemtype="https://schema.org/Person">
<span itemprop="name">Jane Doe</span>
<img src="janedoe.jpg" itemprop="image" alt="Photo of Jane Doe"/>

  <span itemprop="jobTitle">Professor</span>
  <div itemprop="address" itemscope itemtype="https://schema.org/PostalAddress">
    <span itemprop="streetAddress">
      20341 Whitworth Institute
      405 N. Whitworth
    </span>
    <span itemprop="addressLocality">Seattle</span>,
    <span itemprop="addressRegion">WA</span>
    <span itemprop="postalCode">98052</span>
  </div>
  <span itemprop="telephone">(425) 123-4567</span>
  <a href="mailto:jane-doe@xyz.edu" itemprop="email">
    jane-doe@xyz.edu</a>

  Jane's home page:
  <a href="http://www.janedoe.com" itemprop="url">janedoe.com</a>

  Graduate students:
  <a href="http://www.xyz.edu/students/alicejones.html" itemprop="colleague">
    Alice Jones</a>
  <a href="http://www.xyz.edu/students/bobsmith.html" itemprop="colleague">
    Bob Smith</a>
</div>

This will produce a set of metadata tags prefixed with microdata.schema-org.Person. and microdata.schema-org.PostalAddress.

Metadata is produced for each set of items that are nested within a field containing both an itemscope and itemtype property.

<meta name="microdata.schema-org.Person.name" content="Jane Doe">
<meta name="microdata.schema-org.Person.image.src" content="janedoe.jpg">
<meta name="microdata.schema-org.Person.image.alt" content="Photo of Jane Doe">
<meta name="microdata.schema-org.Person.jobTitle" content="Professor">
<meta name="microdata.schema-org.Person.address.streetAddress" content="20341 Whitworth Institute 405 N. Whitworth">
<meta name="microdata.schema-org.Person.address.addressLocality" content="Seattle">
<meta name="microdata.schema-org.Person.address.addressRegion" content="WA">
<meta name="microdata.schema-org.Person.address.postalCode" content="98052">
<meta name="microdata.schema-org.Person.telephone" content="(425) 123-4567">
<meta name="microdata.schema-org.Person.email.href" content="mailto:jane-doe@xyz.edu">
<meta name="microdata.schema-org.Person.email" content="jane-doe@xyz.edu">
<meta name="microdata.schema-org.Person.url.href" content="http://www.janedoe.com">
<meta name="microdata.schema-org.Person.url" content="janedoe.com">
<meta name="microdata.schema-org.Person.colleague.href" content="http://www.xyz.edu/students/alicejones.html">
<meta name="microdata.schema-org.Person.colleague" content="Alice Jones">
<meta name="microdata.schema-org.Person.colleague.href" content="http://www.xyz.edu/students/bobsmith.html">
<meta name="microdata.schema-org.Person.colleague" content="Bob Smith">
<meta name="microdata.schema-org.PostalAddress.address.streetAddress" content="20341 Whitworth Institute 405 N. Whitworth">
<meta name="microdata.schema-org.PostalAddress.address.addressLocality" content="Seattle">
<meta name="microdata.schema-org.PostalAddress.address.addressRegion" content="WA">
<meta name="microdata.schema-org.PostalAddress.address.postalCode" content="98052">

JSON-LD conversion

This plugin will look for embedded schema.org LD+JSON within script tags in your HTML.

When processing JSON-LD, the top-level @content and @type will be used for generating the metadata tag names.

e.g.

<script type="application/ld+json">
{
  "@context": "https://schema.org",
  "@type": "Person",
  "address": {
    "@type": "PostalAddress",
    "addressLocality": "Seattle",
    "addressRegion": "WA",
    "postalCode": "98052",
    "streetAddress": "20341 Whitworth Institute 405 N. Whitworth"
  },
  "colleague": [
    "http://www.xyz.edu/students/alicejones.html",
    "http://www.xyz.edu/students/bobsmith.html"
  ],
  "email": "mailto:jane-doe@xyz.edu",
  "image": "janedoe.jpg",
  "jobTitle": "Professor",
  "name": "Jane Doe",
  "telephone": "(425) 123-4567",
  "url": "http://www.janedoe.com"
}
</script>

This will produce a set of metadata tags prefixed with jsonld.schema-org.Person.

<meta name="jsonld.schema-org.Person.address.addressLocality" content="Seattle">
<meta name="jsonld.schema-org.Person.address.addressRegion" content="WA">
<meta name="jsonld.schema-org.Person.address.postalCode" content="98052">
<meta name="jsonld.schema-org.Person.address.streetAddress" content="20341 Whitworth Institute 405 N. Whitworth">
<meta name="jsonld.schema-org.Person.colleague" content="http://www.xyz.edu/students/alicejones.html">
<meta name="jsonld.schema-org.Person.colleague" content="http://www.xyz.edu/students/bobsmith.html">
<meta name="jsonld.schema-org.Person.email" content="mailto:jane-doe@xyz.edu">
<meta name="jsonld.schema-org.Person.image" content="janedoe.jpg">
<meta name="jsonld.schema-org.Person.jobTitle" content="Professor">
<meta name="jsonld.schema-org.Person.name" content="Jane Doe">
<meta name="jsonld.schema-org.Person.telephone" content="(425) 123-4567">
<meta name="jsonld.schema-org.Person.url" content="http://www.janedoe.com">

Usage

Enable the plugin

Select Plugins from the side navigation pane and click on the Microdata tile.
From the Location section, select the data source to which you would like to enable this plugin from the Select a data source select list.

The plugin will take effect after setup steps and an advanced > full update of the data source has completed.

Additional configuration settings

After configuring this plugin you will need to edit the metadata mappings for your data source and add additional metadata mappings for any metadata you wish to use in your search.

Filter chain configuration

This plugin uses filters which are used to apply transformations to the gathered content.

The filters run in sequence and need be set in an order that makes sense. The plugin supplied filter(s) (as indicated in the listing) should be re-ordered to an appropriate point in the sequence.

Changes to the filter order affects the way the data source processes gathered documents. See: document filters documentation.

Jsoup filter classes

This plugin supplies a filter that needs to run in the HTML document (Jsoup) filter chain:`com.funnelback.plugin.microdata.MicrodataJsoupFilter`

Drag the com.funnelback.plugin.microdata.MicrodataJsoupFilter plugin filter to where you wish it to run in the filter chain sequence.

Examples

Example: JSON-LD

Consider the following HTML document containing embedded JSON-LD.

<html>
<head>
<title>Party Coffee Cake</title>
<script type="application/ld+json">
{
"@context": "https://schema.org/",
"@type": "Recipe",
"name": "Party Coffee Cake",
"author": {
"@type": "Person",
"name": "Mary Stone"
},
"datePublished": "2018-03-10",
"description": "This coffee cake is awesome and perfect for parties.",
"prepTime": "PT20M"
}
</script>
</head>
<body>
<h2>Party coffee cake recipe</h2>
<p>
<i>by Mary Stone, 2018-03-10</i>
</p>
<p>
This coffee cake is awesome and perfect for parties.
</p>
<p>
Preparation time: 20 minutes
</p>
</body>
</html>

When the plugin is enabled the JSON-LD will be extracted and converted to meta tags.

The HTML document will be modified to the following:

<html>
<head>
<title>Party Coffee Cake</title>
<script type="application/ld+json">
{
"@context": "https://schema.org/",
"@type": "Recipe",
"name": "Party Coffee Cake",
"author": {
"@type": "Person",
"name": "Mary Stone"
},
"datePublished": "2018-03-10",
"description": "This coffee cake is awesome and perfect for parties.",
"prepTime": "PT20M"
}
</script>

<meta name="jsonld.schema-org.Recipe.author.name" content="Mary Stone" />
<meta name="jsonld.schema-org.Recipe.datePublished" content="2018-03-10" />
<meta name="jsonld.schema-org.Recipe.prepTime" content="PT20M" />
<meta name="jsonld.schema-org.Recipe.description" content="This coffee cake is awesome and perfect for parties." />
<meta name="jsonld.schema-org.Recipe.name" content="Party Coffee Cake" />

</head>
<body>
<h2>Party coffee cake recipe</h2>
<p>
<i>by Mary Stone, 2018-03-10</i>
</p>
<p>
This coffee cake is awesome and perfect for parties.
</p>
<p>
Preparation time: 20 minutes
</p>
</body>
</html>

Example: Microdata

Consider the following HTML document containing embedded microdata.

<html>
  <head>
    <title>Some Movies</title>
  </head>
  <body>
    <div itemscope itemtype ="https://schema.org/Movie">
      <h1 itemprop="name">Avatar</h1>
      <span>Director: <span itemprop="director">James Cameron</span> (born August 16, 1954)</span>
      <span itemprop="genre">Science fiction</span>
      <a href="../movies/avatar-theatrical-trailer.html" itemprop="trailer">Trailer</a>
    </div>
  </body>
</html>

When the plugin is enabled the microdata will be extracted and converted to meta tags.