SEARCH 101 - Using search

Introduction

This course is aimed at communications and marketing professionals and takes you through the features available in the Funnelback search interface and the reporting and tools provided by the Funnelback insights dashboard.

Each section contains:

  • A summary: A brief overview of what you will accomplish and learn throughout the exercise.

  • Exercise requirements: A list of requirements, such as files, that are needed to complete the exercise.

    The exercises can be read as tutorials or completed interactively. In order to interactively complete the exercises you will need to set up the training searches, or use a training VM.
  • Detailed step-by-step instructions: detailed step-by-step instructions to guide you through completing the exercise.

  • Some extended exercises are also provided. These exercises can be attempted if the standard exercises are completed early, or as some review exercises that can be attempted in your own time.

How does search work?

A search engine’s job is to compile a unified index of content which can come from many sources.

This index is similar to the index at the back of a book - it provides a way of quickly finding the information that is relevant to a series of keywords provided to it when a user runs a search.

How does the index get built?

In order to create the index a search engine like Funnelback has to first scan through all the content that needs to be indexed. The process of gathering this content will depend on the type of content being indexed - for web content a tool called a web crawler is used.

A web crawler gathers content by loading a start page (or set of start pages) and following all the links that it finds. Each link is downloaded and the process repeated until all the content is gathered. Rules about what links can be followed are set when configuring the search (a rule might be something like only gather pages from www.mysite.com).

This means that a page must be linked in order for it to be discovered by a search engine.

Other types might just supply a list of items to gather and index.

A copy of each item is stored as they are discovered. Once all the content is gathered the search engine analyses all the content to build the index. A lot of information about each item is recorded, from counts of each keyword in a document to the number of items that link to the document. This information all forms part of the index and is used to evaluate how relevant an item might be when a search is run.

What happens when a search is run?

Modern search engines have to be able to return results that a relevant to the end user very quickly, often with very little information from the user making the search.

The first thing a search engine does it to take the words entered by the user and simplify them - this includes removing punctuation and words such as and, or, or of.

It will then take this shorter list of words and find pages that match each of the words, compiling these into a list of search results.

The order of the search results is determined by many factors and these will vary from search engine to search engine, and Funnelback allows you to change things that are important in determining how relevant something is.

Some examples of things considered include:

  • How many times each word appears in the content of an item

  • How many times the item is linked to by other items

  • Do the word(s) appear in the item’s title?

  • When was the item last modified?

What is Funnelback?

Funnelback is a search platform that not only produces search results, but also includes a suite of tools that can be used to transform, audit and analyse site content.

As a search engine Funnelback’s role is similar to other search engines - to find relevant documents based on very basic keywords. These keywords are matched against the text inside all of the documents that it finds and retrieves.

Funnelback excels at searching unstructured content with simple keywords. This differs from a database SQL search that operates on a very structured (or fielded) repository and uses complex queries to provide very specific matches within the data.

The Funnelback platform also provides a number of other tools to assist you in understanding and auditing your content such as:

  • Content auditor

  • Accessibility auditor

  • SEO auditor

  • Search analytics

The search results page is the means by which a user refines and interacts with their search results. It provides direct access to the results but also provides summary information for each result and tools for refining the search. A search results page and how it looks is infinitely variable, and can be defined by an administrator.

Simple search form

simple search 01

The search box is the means by which most users interact with a search engine.

A question is asked by typing a few words into the search box and clicking the search button or pressing enter.

This question is then processed by the search engine and a list of results returned to the user.

Search boxes have become simpler over the years now mostly being a single input field and submit button. However, the level of functionality provided by the search box has increased despite the search box becoming simpler.

A lot of things happen behind the scenes when interacting with a modern search box - this can include things like expanding a user’s query with synonyms, or using information that is known about the user’s current location.

This has led to users having high expectations of the quality of what is returned by a search engine, even when minimal information is provided. Most users input single word queries, yet search engines are still able to deliver results that are relevant to the user.

Advanced search form

Advanced search forms were very common in the past - they provide users with lots of fields that allow for very specific searches to be defined by a user. However, they have disappeared from searches as simple search boxes have become smarter to the point where they are quite rare nowadays.

Funnelback has support for advanced search forms but the practical use for an advanced search form has diminished to the point where they are only suitable for very specific audiences such as librarians or legal professionals who have a long history of using advanced search.

It is a common misconception that advanced search will provide the user with better results. In fact studies have found that in most cases users of advanced search yield poorer results because the complexity of the search form means the user doesn’t really understand how to use the form, thus specifying queries that are suboptimal, or queries that don’t make sense.

The advanced search form can be fully customized.

The location of an advanced search feature using Funnelback will vary with each implementation, but will often look similar to the following:

advanced search 01

Auto-completion

Auto-completion provides the user with real-time suggestions as search terms are typed into the search box. Funnelback supports several forms of auto-completion.

The different types of auto-completion can be combined so that suggestions are returned from multiple auto-completion sources

Simple auto-completion

As the user begins to type a search keyword or phrase, the search box presents the user a list of suggestions that predict what the user might want. For example:

simple auto completion 01

The suggestions presented by simple auto-completion are based off words and phrases found within the content that is being searched.

Clicking on the suggested term runs a search for the suggestion.

Structured auto-completion

This provides advanced auto-completions based on structured data provided by the search administrator.

The suggestions can be grouped by an optional category value and provide rich content (such as thumbnail images and other data) to the users.

Clicking on a suggestion can do a number of things, depending on how the suggestion has been configured. It might:

  • Run a search (similar to simple auto-completion)

  • Take you directly to a page (like clicking on a search result)

  • Add some keywords to the search box

  • Display some results that you can’t click on (like information about a person in directory)

structured auto completion 01

Tutorial: Auto-completion

  1. Log in to the search dashboard where you are doing your training.

    See: Training - search dashboard access information if you’re not sure how to access the training. Ignore this step if you’re treating this as a non-interactive tutorial.
  2. Inside the search box start to type the word chicken. As you begin to type, a list of suggestions will begin to display after you’ve typed a minimum of three characters. This list of suggestions updates as you enter each additional character:

    exercise auto completion 01
  3. Select chickpeas from the list of suggestions to generate the search results page for the search query chickpeas.

    exercise auto completion 02
  4. Start to type a search for the word baking. Observe that structured suggestions for cooking techniques are returned.

    exercise auto completion 03
  5. Select the baking cookies suggestion and observe that you are taken directly to a page about baking cookies. This contrasts to the simple suggestion that ran a search for the suggestion. What happens when you click on a structured suggestion depends on what is in the CSV file used to create the suggestions.

    Note: the CSV can be configured to run a search as was the case for the simple auto-completion suggestion.

    exercise auto completion 04

Using the search results page

Result set summaries

Result set summary

Once a search has been conducted via the search box, a results page for that search will be produced containing a number of search results.

A short summary of the set of results is returned along with the search results. This summary provides information about the number of results returned, which page of results is currently displayed and what was searched for.

result set summary 01

Result item summary

Each search result includes information about the result. This information will vary depending on what data is available and how the search has been configured. By default, the following is returned by Funnelback:

result summary 01
  • Linked title of the result item

  • URL of the item

  • Access to a cached version of the result

  • The date recorded for the result

  • A short summary of the result. This summary is usually automatically generated and centred around occurrences of the search terms within the context.

  • Indexed metadata can be used to enhance your search results - in this example the thumbnail URL is indexed as metadata and used to display the thumbnail image in the result template.

The keywords used by the user will always be highlighted in both the title, URL and result.

Tutorial: Results summary

  1. Run a search for chickpeas.

  2. Observe the result set summary message:

    exercise results summary 01
  3. Run a query for roasted chickpea and observe how the result summary message changes:

    exercise results summary 02
  4. The results summary message informs us that the search has returned with 90 results. 3 of those results are fully matching documents or pages that contain the keywords roasted and chickpea. The remaining 87 results are partial matches. A partial match is when the result does not contain all the search terms - in this case either roasted or chickpea but not both. Fully matching results will always appear before partially matching results when sorting by relevance.

Faceted navigation

Faceted navigation provides search users with a set of filters that can be applied to narrow down a set of search results.

What is a facet?

A facet is best thought of as a way of filtering or segmenting the content within a website based on some property of the item. Each facet will include one or more facet categories which are the values of this item property.

For example a facet for color might be defined, with facet category values corresponding to individual colors (red, blue and so on).

faceted navigation 01

Configuration of faceted navigation relies on each item being tagged with the appropriate values that indicate the categories that apply for the item.

The number of the results for each category within a facet are estimates only.

Types of facets

Funnelback supports five types of facets. Each type allows a user to filter the result set.

The types differ by proving choice in how the filtering is applied - allowing filtering on single or multiple categories and providing options on how the filters are combined and how they apply to the result set.

Filter on a single category

Allows a user to filter the result set using a single category value chosen from the facet. Upon selection the facet displays only the selected category value.

Example: Searching for engineer jobs

filter on a single category 01

Selecting the permanent category value from the type facet results in the search results being filtered to only include permanent jobs.

filter on a single category 02

Further category selection from the type facet is not possible, however additional filters can be applied from other facets (location and industry).

Filter on a single category, with drill down

This is sometimes referred to as a hierarchical facet.

Allows a user to filter the result set using a single category value chosen from the facet. After the filter is applied sub-categories may be displayed for the facet.

Example: Searching for engineer jobs

filter on a single category with drill down 01

Selecting the VIC category value from the location facet results in the search results being filtered to only include jobs located in Victoria. This is the same behaviour as for a filter on single category facet.

However further sub-category selection from the type facet for cities is now possible.

filter on a single category with drill down 02

Selecting Bendigo from the sub-categories on the location facet further restricts the results to now only include results from Bendigo. Further refinement is possible if additional sub-category metadata is defined (for this example the type facet has been refined as much as possible).

filter on a single category with drill down 03

Further refinement on other facets is possible.

Filter on multiple categories

This is sometimes referred to multi-select or checkbox faceting.

Allows a user to filter the result set using multiple categories from the same facet. The categories can be combined using either AND or OR logic.

Example: Searching for engineer jobs

In this example the industry facet is defined as a multi-select facet using the default OR combine logic.

filter on multiple categories 01

Note: the location facet has been collapsed in the examples so that the industry facet is visible in the screenshots.

Selecting the engineering category value from the industry facet results in the search results being filtered to only include jobs categorised as engineering. This is the same behaviour as for a filter on single category facet.

filter on multiple categories 02

The display updates to show that the engineering category filter is applied, but allows additional categories to be added. Selecting the sales category adds this filter so that the result set is now filtered to return results that are categorised as engineering OR sales.

For most facets an OR combine logic will give the desired behaviour, however this can be chosen when defining the facet and changed so that the categories are combined with an AND.

filter on multiple categories 03
Toggle between categories

Sometimes also referred to as a radio facet due to the facet operating like a radio button list.

Allows a user to filter the result set on a single category. The set of categories remain present after the filter is applied allowing a user to toggle through the different category values.

Example: Searching for engineer jobs

In this example the industry facet is defined as a toggle between categories facet.

toggle between categories 01

Selecting advertising & media from the industry facet filters the result set to only results categorised as advertising & media. Observe that the other categories remain available and the counts are unchanged.

toggle between categories 02

Selecting banking & financial services switches (or toggles) to filter the result set to only include results categorised as banking & financial services. Once again the other categories remain available and unchanged.

toggle between categories 03

Clicking on the next category shows the same behaviour.

toggle between categories 04
Toggle between tabs

This is a special type of facet that is presented as a series of tabs. It is designed to allow a search to be separated into a series of tabs, with other facets then being used to further filter the result set. Because of this the tab facet isn’t included with the selected facets that are displayed above the search results.

Example: Searching for engineer jobs

A toggle between tabs facet, based on the state metadata defines the tabs displayed for this search.

filter on a single category 01

Selecting the WA tab filters the results to only include items from WA. Observe that the Location facet only shows WA, and that the selected facet block above the search results is not displayed to indicate that the results are filtered to WA. This is because the tabs are treated as completely separate UI element to the faceted navigation filter panel.

toggle between tabs 02

Further facets can be applied from the left hand faceted panel.

toggle between tabs 03

Switching tabs filters the result set to the new tab, removing any other facets that were applied within the previous tab.

toggle between tabs 04

Tutorial: Faceted navigation

  1. Run a search for french

  2. Examine the faceted navigation returned with the search results.

  3. Faceted navigation for type of meat and tags are returned alongside the results. These allow the search user to filter the result set based on these properties.

    exercise faceted navigation 01
  4. Categories default to being ordered by frequency (or the estimated number of results that are tagged with the word), with the top 8 categories shown initially. The number shown to the right of the facet is an estimate of the number of results that are tagged with the corresponding category. For the current search approximately 11 of the results are tagged as French.

    exercise faceted navigation 02
  5. Clicking more on the categories facet expands the list to show the top 16 categories. The sort and limits on number of categories returned can be adjusted by an administrator.

    exercise faceted navigation 03
  6. Clicking the dessert facet category applies a filter to the search query that filters the result set down to only return results that are tagged with a category of dessert. The facet display for the categories facet also updates to indicate that the dessert filter has been selected.

    exercise faceted navigation 04
  7. A selected categories widget also appears above the result indicating the currently selected filters:

    exercise faceted navigation 05
  8. Click on the clear all link, or click on the dessert facet to remove the filter and return the results to the previous list. Select the dessert facet category again and fish from the tags filter. The result set is now filtered to include only items where the tag is marked as either dessert or fish. Categories within a facet are combined with an OR by default (this can be changed in the configuration). Also observe the selected filters widget.

    exercise faceted navigation 06
  9. Select seafood from the type of meat facet. This shows that categories can be combined across facets. Selected categories across facets are combined with an AND - the results are now filtered to include items with type of meat = seafood AND tags = either dessert OR fish.

    exercise faceted navigation 07
  10. Now that we’ve applied one or more facets the search query input box has a checkbox adjacent to it labelled within selected categories only. This allows us to keep our facet categories selected when running a new search query.

    exercise faceted navigation 08
  11. Run a search for roasted salmon keeping the within selected categories box ticked and observe that the new search maintains the tags: fish and dessert, and types of meat: seafood facets. Also observe that a tier bar indicating that the search results are partially matching (reading results that match 1 of 2 words) is displayed above the search result.

Best bets

Best bets allows an administrator to configure a featured result item to be displayed when a user conducts a specific search.

A best bet is not a search result, but it can feature or promote a page / URL that is not part of a website that is being indexed.

For example, when a user searches for the term Foodista a best bet to featuring the Wikipedia page on Foodista wiki is displayed above the search results. This is presented in addition to the search results.

best bets 01

The style and appearance of the best bets in the search results is governed by the stylesheets that are applied to the search results template, and the position in the search results can be controlled by a search administrator with the ability to edit the templates. A best bet can include HTML formatting.

In the example above a HTML snippet including an image of the Foodista logo has been returned with the best bet.

Tutorial: Best bets

  1. Run a search for foodista, making sure the within selected categories only box is unticked. Observe a best bet that is returned above the search results, to the page in Wikipedia discussing the Foodista site:

    best bets 01

Contextual navigation

Contextual navigation suggests a list of related searches by analyzing the result summaries in the set of results returned for a search.

These suggestions are grouped into types and topics related to the original search query.

For example, if a user performed a search for pork the related searches suggested may include types of pork such as pork chops and pork belly as well as topics on pork.

Clicking on a suggestion runs a proximity search for the suggested words. A proximity search looks for matching results where the suggested words are close to each other in the text.

related searches contextual navigation 01
  1. Perform a search for the term milk.

  2. Scroll to the bottom of the page to view the related searches that the term milk has produced.

    exercise related searches contextual navigation 01
  3. From the types of milk section, click on the term condensed milk. The search results will then repopulate to reflect the term condensed milk.

    exercise related searches contextual navigation 02
Observe that the search keyword is surrounded in special quote characters (called backticks). This tells Funnelback to run a proximity search - this looks for the words condensed and milk but only when they are close together in the text.

Spelling suggestions

Spelling suggestions sourced from words and phrases found in the content are automatically returned by Funnelback, regardless of if the search returns any results for the query.

Spelling suggestions when clicked will always return search results because they are based on words in the content and not an external dictionary.

spelling suggestions 01

Tutorial: Spelling suggestions and no results

  1. Run a search for qinwa.

  2. Funnelback will return with zero results, but provide you with a possible spelling correction, so that you can then click on the correct spelling of the word to improve your search.

    spelling suggestions 01
  3. Beneath the spelling suggestion you will see a no results message. This message can be adjusted and configured by an administrator of Funnelback.

    no results message 01
  4. Click on the quinoa spelling suggestion (the did you mean quinoa link) to run a search for quinoa.

No results message

If a user enters a query that produces no results, a customizable no results message can be displayed. This message is customized via the design template of the search results page. The following is the default message:

no results message 01

Pagination

The search results pagination control allows a user to page through the search results.

The control can be displayed multiple times (e.g. above and below the search results) and can be customized by an administrator.

pagination 01

Tutorial: Pagination

  1. Run a search for chicken.

  2. Scroll down to the bottom search results page. There you will see pagination controls displayed below the search results.

    pagination 01
  3. Click on the second page of results: Observe that the prev button appears, and the page indicator change to show which page of results you are on.

    exercise pagination 02

Cached results

Every page that a search engine indexes is stored locally by Funnelback. This cached version of the page can be viewed from the search results, with the search query terms highlighted within the content. The cached version of a search result is accessed from the downward caret icon beside the URL of a search result.

cached results 01

Clicking this menu item opens a cached version of the page with the queried keywords highlighted within the content. The cached version is the content as it was when Funnelback crawled the URL. The example below shows the cached version of this result when the user searched for the keyword quinoa.

cached results 02

Tutorial: Cached results

  1. Run the search for the keyword term: quinoa.

  2. From the list of search results produced, click on the downward arrow beside the search result for Argula tomato quinoa salad to access the cached option.

    exercise cached results 01
  3. Click on cached from the menu displayed to display the cached version of the page: Observe that quinoa is highlighted wherever it appears in the page text.

    exercise cached results 02
  4. Return to the search results page by clicking on the back to results link.

Sessions and history

The search sessions and history feature in Funnelback requires cookies to be enabled in the user’s browser. And therefore feature is only supported for Funnelback endpoints over HTTPS protocol. Funnelback has the ability to track a user’s search terms and recently visited items as well as providing the user with the ability to create a shortlist of search result items.

Result shortlisting

Result shortlisting provides a user with the ability to flag certain result items to be added to a shortlist (or result cart) by pinning the item. The pin changes to a cross once the item has been shortlisted. This allows a user to remove the item from the shortlist by clicking on the cross.

result shortlisting 01

The shopping cart icon in the panel containing the search box shows how many items have been shortlisted. All the shortlisted items can also be viewed by clicking on the shopping cart icon.

result shortlisting 02

Tutorial: Search result shortlisting

In this exercise we’ll use Funnelback to help plan a menu using the search result shortlisting feature.

  1. Run a search for appetizers and choose one of the recipes and shortlist this by clicking on the pin that appears before the title. Observe that the pin changes to a cross when you click it.

    exercise search result shortlisting 01
  2. Observe that the shopping cart icon changes to show a 1 next to the cart icon.

    exercise search result shortlisting 02
  3. Search for grilled chicken and shortlist one of the recipes.

  4. Search for dessert and shortlist one of the recipes.

  5. View the shortlist by clicking on the shopping cart icon. Observe the three shortlisted results are returned as saved results. Return to the search results by clicking the back to results link or clicking again on the cart.

    Clicking on an item in the shortlist opens the recipe.

    exercise search result shortlisting 03
  6. Return to the search results page by clicking on the go back to search results link.

Search history

When search sessions are enabled a user’s search history is tracked as searches are submitted. A summary of the search history is displayed above the search results.

search history 01

Additional information is displayed if you hover over an item. If the item has a filter applied (as indicated by the funnel icon) then the applied filters will display on hover, otherwise a tooltip indicating the time is displayed.

search history 02

Clicking the more link opens the full history screen.

search history 03

Visited items history

Funnelback also keeps track of all the search results that a user visits. Once a result has been visited an additional field is displayed within the result summary indicating the time that the visit occurred.

visited items history 01

Clicking on the Last visited link also opens up the history screen which provides a full list of recently visited URLs.

visited items history 02

Tutorial: Search history

  1. Observe the search history that is displayed above your search results. It will look something similar to:

    exercise search history 01
  2. Access the search history by selecting history from the menu under the gear icon on the right-hand side of the search results interface, or by clicking on the more link within the recent searches panel.

    exercise search history 02
  3. The full list of recent searches is displayed along with the number of results for each search and the time at which the search was performed. Clear the history by clicking on the clear button. When prompted confirm that the history should be cleared.

    exercise search history 03

Result collapsing

Result collapsing groups similar search results together when displaying search results. The key or keys used for collapsing are configured by a search administrator and are based on either the documents content, or the document’s metadata.

For example, the result item below from a search of the complete works of Shakespeare shows that there are 21 very similar results to the Merchant of Venice: list of scenes result. This search has been configured to group or collapse all search results that are tagged with the same metadata value for a field recording the name of the play.

search result grouping result collapsing 01

Clicking on the 21 very similar results lists the 22 search results that form part of this group (the list of scenes result plus the 21 similar results). These can be sorted by providing a sort parameter in your More…​ link or by setting -sort in your query processor options. The sort options for this list of similar results are the same as for any normal search result sort.

search result grouping result collapsing 02

The initial search result can also be customized to show some additional information for the collapsed results.

The example result below, from another site groups sections of a publication together. The search result is configured to display the title of the first four collapsed items with a More…​ link which is the same as the 21 similar items link in the previous example.

search result grouping result collapsing 03

These (nested) collapsed results can be sorted using the -collapsed_docs_sort query processor option. This option will affect the set of collapsed results that are returned within the intial search response (controlled by the -collapsed_num_ranks option).

Quick links are direct links to important sub-pages of a home page and can be displayed as part of the result summary. Quick links are normally only used when providing a multi-site search such as a whole of university search. The search result can also include an optional search box that restricted to the specific site displayed for the search result.

For example a search for the University of Sydney might produce the following search result. Entering a search into the search box will run a search restricted to pages from the Sydney Conservatorium of Music website.

quick links 01

Advanced search results templating

Funnelback has an extremely flexible templating language. This means that search results can be adapted to any site design and can be used in innovative ways.

For example, search results could be returned in a table, or even plotted onto a map if geospatial coordinate metadata is available:

advanced search results templating 01

Search results can also be templated to any text-based format.

This means that results could be returned in CSV, JSON, XML or other custom text formats.