SEARCH 101 - Using search
Introduction
This course is aimed at communications and marketing professionals and takes you through the features available in the Funnelback search interface and the reporting and tools provided by the Funnelback insights dashboard.
Each section contains:
-
A summary: A brief overview of what you will accomplish and learn throughout the exercise.
-
Exercise requirements: A list of requirements, such as files, that are needed to complete the exercise.
The exercises can be read as tutorials or completed interactively. In order to interactively complete the exercises you will need to set up the training searches, or use a training VM. -
Detailed step-by-step instructions: detailed step-by-step instructions to guide you through completing the exercise.
-
Some extended exercises are also provided. These exercises can be attempted if the standard exercises are completed early, or as some review exercises that can be attempted in your own time.
What is search?
How does search work?
A search engine’s job is to compile a unified index of content which can come from many sources.
This index is similar to the index at the back of a book - it provides a way of quickly finding the information that is relevant to a series of keywords provided to it when a user runs a search.
How does the index get built?
In order to create the index a search engine like Funnelback has to first scan through all the content that needs to be indexed. The process of gathering this content will depend on the type of content being indexed - for web content a tool called a web crawler is used.
A web crawler gathers content by loading a start page (or set of start pages) and following all the links that it finds. Each link is downloaded and the process repeated until all the content is gathered. Rules about what links can be followed are set when configuring the search (a rule might be something like only gather pages from www.mysite.com).
This means that a page must be linked in order for it to be discovered by a search engine.
Other types might just supply a list of items to gather and index.
A copy of each item is stored as they are discovered. Once all the content is gathered the search engine analyses all the content to build the index. A lot of information about each item is recorded, from counts of each keyword in a document to the number of items that link to the document. This information all forms part of the index and is used to evaluate how relevant an item might be when a search is run.
What happens when a search is run?
Modern search engines have to be able to return results that a relevant to the end user very quickly, often with very little information from the user making the search.
The first thing a search engine does it to take the words entered by the user and simplify them - this includes removing punctuation and words such as and, or, or of.
It will then take this shorter list of words and find pages that match each of the words, compiling these into a list of search results.
The order of the search results is determined by many factors and these will vary from search engine to search engine, and Funnelback allows you to change things that are important in determining how relevant something is.
Some examples of things considered include:
-
How many times each word appears in the content of an item
-
How many times the item is linked to by other items
-
Do the word(s) appear in the item’s title?
-
When was the item last modified?
What is Funnelback?
Funnelback is a search platform that not only produces search results, but also includes a suite of tools that can be used to transform, audit and analyse site content.
As a search engine Funnelback’s role is similar to other search engines - to find relevant documents based on very basic keywords. These keywords are matched against the text inside all of the documents that it finds and retrieves.
Funnelback excels at searching unstructured content with simple keywords. This differs from a database SQL search that operates on a very structured (or fielded) repository and uses complex queries to provide very specific matches within the data.
The Funnelback platform also provides a number of other tools to assist you in understanding and auditing your content such as:
-
Content auditor
-
Accessibility auditor
-
SEO auditor
-
Search analytics
Using search
The search results page is the means by which a user refines and interacts with their search results. It provides direct access to the results but also provides summary information for each result and tools for refining the search. A search results page and how it looks is infinitely variable, and can be defined by an administrator.
Search box
Simple search form
The search box is the means by which most users interact with a search engine.
A question is asked by typing a few words into the search box and clicking the search button or pressing enter.
This question is then processed by the search engine and a list of results returned to the user.
Search boxes have become simpler over the years now mostly being a single input field and submit button. However, the level of functionality provided by the search box has increased despite the search box becoming simpler.
A lot of things happen behind the scenes when interacting with a modern search box - this can include things like expanding a user’s query with synonyms, or using information that is known about the user’s current location.
This has led to users having high expectations of the quality of what is returned by a search engine, even when minimal information is provided. Most users input single word queries, yet search engines are still able to deliver results that are relevant to the user.
Advanced search form
Advanced search forms were very common in the past - they provide users with lots of fields that allow for very specific searches to be defined by a user. However, they have disappeared from searches as simple search boxes have become smarter to the point where they are quite rare nowadays.
Funnelback has support for advanced search forms but the practical use for an advanced search form has diminished to the point where they are only suitable for very specific audiences such as librarians or legal professionals who have a long history of using advanced search.
It is a common misconception that advanced search will provide the user with better results. In fact studies have found that in most cases users of advanced search yield poorer results because the complexity of the search form means the user doesn’t really understand how to use the form, thus specifying queries that are suboptimal, or queries that don’t make sense.
The advanced search form can be fully customized.
The location of an advanced search feature using Funnelback will vary with each implementation, but will often look similar to the following:
Auto-completion
Auto-completion provides the user with real-time suggestions as search terms are typed into the search box. Funnelback supports several forms of auto-completion.
The different types of auto-completion can be combined so that suggestions are returned from multiple auto-completion sources
Simple auto-completion
As the user begins to type a search keyword or phrase, the search box presents the user a list of suggestions that predict what the user might want. For example:
The suggestions presented by simple auto-completion are based off words and phrases found within the content that is being searched.
Clicking on the suggested term runs a search for the suggestion.
Structured auto-completion
This provides advanced auto-completions based on structured data provided by the search administrator.
The suggestions can be grouped by an optional category value and provide rich content (such as thumbnail images and other data) to the users.
Clicking on a suggestion can do a number of things, depending on how the suggestion has been configured. It might:
-
Run a search (similar to simple auto-completion)
-
Take you directly to a page (like clicking on a search result)
-
Add some keywords to the search box
-
Display some results that you can’t click on (like information about a person in directory)
Tutorial: Auto-completion
-
Log in to the search dashboard where you are doing your training.
See: Training - search dashboard access information if you’re not sure how to access the training. Ignore this step if you’re treating this as a non-interactive tutorial. -
Inside the search box start to type the word chicken. As you begin to type, a list of suggestions will begin to display after you’ve typed a minimum of three characters. This list of suggestions updates as you enter each additional character:
-
Select chickpeas from the list of suggestions to generate the search results page for the search query chickpeas.
-
Start to type a search for the word baking. Observe that structured suggestions for cooking techniques are returned.
-
Select the baking cookies suggestion and observe that you are taken directly to a page about baking cookies. This contrasts to the simple suggestion that ran a search for the suggestion. What happens when you click on a structured suggestion depends on what is in the CSV file used to create the suggestions.
Note: the CSV can be configured to run a search as was the case for the simple auto-completion suggestion.
Using the search results page
Result set summaries
Result set summary
Once a search has been conducted via the search box, a results page for that search will be produced containing a number of search results.
A short summary of the set of results is returned along with the search results. This summary provides information about the number of results returned, which page of results is currently displayed and what was searched for.
Result item summary
Each search result includes information about the result. This information will vary depending on what data is available and how the search has been configured. By default, the following is returned by Funnelback:
-
Linked title of the result item
-
URL of the item
-
Access to a cached version of the result
-
The date recorded for the result
-
A short summary of the result. This summary is usually automatically generated and centred around occurrences of the search terms within the context.
-
Indexed metadata can be used to enhance your search results - in this example the thumbnail URL is indexed as metadata and used to display the thumbnail image in the result template.
The keywords used by the user will always be highlighted in both the title, URL and result.
Tutorial: Results summary
-
Run a search for chickpeas.
-
Observe the result set summary message:
-
Run a query for roasted chickpea and observe how the result summary message changes:
-
The results summary message informs us that the search has returned with 90 results. 3 of those results are fully matching documents or pages that contain the keywords roasted and chickpea. The remaining 87 results are partial matches. A partial match is when the result does not contain all the search terms - in this case either roasted or chickpea but not both. Fully matching results will always appear before partially matching results when sorting by relevance.
Faceted navigation
Faceted navigation provides search users with a set of filters that can be applied to narrow down a set of search results.
What is a facet?
A facet is best thought of as a way of filtering or segmenting the content within a website based on some property of the item. Each facet will include one or more facet categories which are the values of this item property.
For example a facet for color might be defined, with facet category values corresponding to individual colors (red, blue and so on).
Configuration of faceted navigation relies on each item being tagged with the appropriate values that indicate the categories that apply for the item.
| The number of the results for each category within a facet are estimates only. |
Types of facets
Funnelback supports five types of facets. Each type allows a user to filter the result set.
The types differ by proving choice in how the filtering is applied - allowing filtering on single or multiple categories and providing options on how the filters are combined and how they apply to the result set.
Filter on a single category
Allows a user to filter the result set using a single category value chosen from the facet. Upon selection the facet displays only the selected category value.
Example: Searching for engineer jobs
Selecting the permanent category value from the type facet results in the search results being filtered to only include permanent jobs.
Further category selection from the type facet is not possible, however additional filters can be applied from other facets (location and industry).
Filter on a single category, with drill down
This is sometimes referred to as a hierarchical facet.
Allows a user to filter the result set using a single category value chosen from the facet. After the filter is applied sub-categories may be displayed for the facet.
Example: Searching for engineer jobs
Selecting the VIC category value from the location facet results in the search results being filtered to only include jobs located in Victoria. This is the same behaviour as for a filter on single category facet.
However further sub-category selection from the type facet for cities is now possible.
Selecting Bendigo from the sub-categories on the location facet further restricts the results to now only include results from Bendigo. Further refinement is possible if additional sub-category metadata is defined (for this example the type facet has been refined as much as possible).
Further refinement on other facets is possible.
Filter on multiple categories
This is sometimes referred to multi-select or checkbox faceting.
Allows a user to filter the result set using multiple categories from the same facet. The categories can be combined using either AND or OR logic.
Example: Searching for engineer jobs
In this example the industry facet is defined as a multi-select facet using the default OR combine logic.
Note: the location facet has been collapsed in the examples so that the industry facet is visible in the screenshots.
Selecting the engineering category value from the industry facet results in the search results being filtered to only include jobs categorised as engineering. This is the same behaviour as for a filter on single category facet.
The display updates to show that the engineering category filter is applied, but allows additional categories to be added. Selecting the sales category adds this filter so that the result set is now filtered to return results that are categorised as engineering OR sales.
For most facets an OR combine logic will give the desired behaviour, however this can be chosen when defining the facet and changed so that the categories are combined with an AND.
Toggle between categories
Sometimes also referred to as a radio facet due to the facet operating like a radio button list.
Allows a user to filter the result set on a single category. The set of categories remain present after the filter is applied allowing a user to toggle through the different category values.
Example: Searching for engineer jobs
In this example the industry facet is defined as a toggle between categories facet.
Selecting advertising & media from the industry facet filters the result set to only results categorised as advertising & media. Observe that the other categories remain available and the counts are unchanged.
Selecting banking & financial services switches (or toggles) to filter the result set to only include results categorised as banking & financial services. Once again the other categories remain available and unchanged.
Clicking on the next category shows the same behaviour.
Toggle between tabs
This is a special type of facet that is presented as a series of tabs. It is designed to allow a search to be separated into a series of tabs, with other facets then being used to further filter the result set. Because of this the tab facet isn’t included with the selected facets that are displayed above the search results.
Example: Searching for engineer jobs
A toggle between tabs facet, based on the state metadata defines the tabs displayed for this search.
Selecting the WA tab filters the results to only include items from WA. Observe that the Location facet only shows WA, and that the selected facet block above the search results is not displayed to indicate that the results are filtered to WA. This is because the tabs are treated as completely separate UI element to the faceted navigation filter panel.
Further facets can be applied from the left hand faceted panel.
Switching tabs filters the result set to the new tab, removing any other facets that were applied within the previous tab.
Tutorial: Faceted navigation
-
Run a search for french
-
Examine the faceted navigation returned with the search results.
-
Faceted navigation for type of meat and tags are returned alongside the results. These allow the search user to filter the result set based on these properties.
-
Categories default to being ordered by frequency (or the estimated number of results that are tagged with the word), with the top 8 categories shown initially. The number shown to the right of the facet is an estimate of the number of results that are tagged with the corresponding category. For the current search approximately 11 of the results are tagged as French.
-
Clicking more on the categories facet expands the list to show the top 16 categories. The sort and limits on number of categories returned can be adjusted by an administrator.
-
Clicking the dessert facet category applies a filter to the search query that filters the result set down to only return results that are tagged with a category of dessert. The facet display for the categories facet also updates to indicate that the dessert filter has been selected.
-
A selected categories widget also appears above the result indicating the currently selected filters:
-
Click on the clear all link, or click on the dessert facet to remove the filter and return the results to the previous list. Select the dessert facet category again and fish from the tags filter. The result set is now filtered to include only items where the tag is marked as either dessert or fish. Categories within a facet are combined with an OR by default (this can be changed in the configuration). Also observe the selected filters widget.
-
Select seafood from the type of meat facet. This shows that categories can be combined across facets. Selected categories across facets are combined with an AND - the results are now filtered to include items with type of meat = seafood AND tags = either dessert OR fish.
-
Now that we’ve applied one or more facets the search query input box has a checkbox adjacent to it labelled within selected categories only. This allows us to keep our facet categories selected when running a new search query.
-
Run a search for roasted salmon keeping the within selected categories box ticked and observe that the new search maintains the tags: fish and dessert, and types of meat: seafood facets. Also observe that a tier bar indicating that the search results are partially matching (reading results that match 1 of 2 words) is displayed above the search result.
Best bets
Best bets allows an administrator to configure a featured result item to be displayed when a user conducts a specific search.
A best bet is not a search result, but it can feature or promote a page / URL that is not part of a website that is being indexed.
For example, when a user searches for the term Foodista a best bet to featuring the Wikipedia page on Foodista wiki is displayed above the search results. This is presented in addition to the search results.
The style and appearance of the best bets in the search results is governed by the stylesheets that are applied to the search results template, and the position in the search results can be controlled by a search administrator with the ability to edit the templates. A best bet can include HTML formatting.
In the example above a HTML snippet including an image of the Foodista logo has been returned with the best bet.
Tutorial: Best bets
-
Run a search for foodista, making sure the within selected categories only box is unticked. Observe a best bet that is returned above the search results, to the page in Wikipedia discussing the Foodista site:
Contextual navigation
Contextual navigation suggests a list of related searches by analyzing the result summaries in the set of results returned for a search.
These suggestions are grouped into types and topics related to the original search query.
For example, if a user performed a search for pork the related searches suggested may include types of pork such as pork chops and pork belly as well as topics on pork.
Clicking on a suggestion runs a proximity search for the suggested words. A proximity search looks for matching results where the suggested words are close to each other in the text.
Tutorial: Related searches (contextual navigation)
-
Perform a search for the term milk.
-
Scroll to the bottom of the page to view the related searches that the term milk has produced.
-
From the types of milk section, click on the term condensed milk. The search results will then repopulate to reflect the term
condensed milk.
| Observe that the search keyword is surrounded in special quote characters (called backticks). This tells Funnelback to run a proximity search - this looks for the words condensed and milk but only when they are close together in the text. |
Spelling suggestions
Spelling suggestions sourced from words and phrases found in the content are automatically returned by Funnelback, regardless of if the search returns any results for the query.
Spelling suggestions when clicked will always return search results because they are based on words in the content and not an external dictionary.
Tutorial: Spelling suggestions and no results
-
Run a search for qinwa.
-
Funnelback will return with zero results, but provide you with a possible spelling correction, so that you can then click on the correct spelling of the word to improve your search.
-
Beneath the spelling suggestion you will see a no results message. This message can be adjusted and configured by an administrator of Funnelback.
-
Click on the quinoa spelling suggestion (the did you mean quinoa link) to run a search for quinoa.
No results message
If a user enters a query that produces no results, a customizable no results message can be displayed. This message is customized via the design template of the search results page. The following is the default message:
Pagination
The search results pagination control allows a user to page through the search results.
The control can be displayed multiple times (e.g. above and below the search results) and can be customized by an administrator.
Tutorial: Pagination
-
Run a search for chicken.
-
Scroll down to the bottom search results page. There you will see pagination controls displayed below the search results.
-
Click on the second page of results: Observe that the prev button appears, and the page indicator change to show which page of results you are on.
Cached results
Every page that a search engine indexes is stored locally by Funnelback. This cached version of the page can be viewed from the search results, with the search query terms highlighted within the content. The cached version of a search result is accessed from the downward caret icon beside the URL of a search result.
Clicking this menu item opens a cached version of the page with the queried keywords highlighted within the content. The cached version is the content as it was when Funnelback crawled the URL. The example below shows the cached version of this result when the user searched for the keyword quinoa.
Tutorial: Cached results
-
Run the search for the keyword term: quinoa.
-
From the list of search results produced, click on the downward arrow beside the search result for Argula tomato quinoa salad to access the cached option.
-
Click on cached from the menu displayed to display the cached version of the page: Observe that quinoa is highlighted wherever it appears in the page text.
-
Return to the search results page by clicking on the back to results link.
Sessions and history
| The search sessions and history feature in Funnelback requires cookies to be enabled in the user’s browser. And therefore feature is only supported for Funnelback endpoints over HTTPS protocol. Funnelback has the ability to track a user’s search terms and recently visited items as well as providing the user with the ability to create a shortlist of search result items. |
Result shortlisting
Result shortlisting provides a user with the ability to flag certain result items to be added to a shortlist (or result cart) by pinning the item. The pin changes to a cross once the item has been shortlisted. This allows a user to remove the item from the shortlist by clicking on the cross.
The shopping cart icon in the panel containing the search box shows how many items have been shortlisted. All the shortlisted items can also be viewed by clicking on the shopping cart icon.
Tutorial: Search result shortlisting
In this exercise we’ll use Funnelback to help plan a menu using the search result shortlisting feature.
-
Run a search for appetizers and choose one of the recipes and shortlist this by clicking on the pin that appears before the title. Observe that the pin changes to a cross when you click it.
-
Observe that the shopping cart icon changes to show a 1 next to the cart icon.
-
Search for grilled chicken and shortlist one of the recipes.
-
Search for dessert and shortlist one of the recipes.
-
View the shortlist by clicking on the shopping cart icon. Observe the three shortlisted results are returned as saved results. Return to the search results by clicking the back to results link or clicking again on the cart.
Clicking on an item in the shortlist opens the recipe.
-
Return to the search results page by clicking on the go back to search results link.
Search history
When search sessions are enabled a user’s search history is tracked as searches are submitted. A summary of the search history is displayed above the search results.
Additional information is displayed if you hover over an item. If the item has a filter applied (as indicated by the funnel icon) then the applied filters will display on hover, otherwise a tooltip indicating the time is displayed.
Clicking the more link opens the full history screen.
Visited items history
Funnelback also keeps track of all the search results that a user visits. Once a result has been visited an additional field is displayed within the result summary indicating the time that the visit occurred.
Clicking on the Last visited link also opens up the history screen which provides a full list of recently visited URLs.
Tutorial: Search history
-
Observe the search history that is displayed above your search results. It will look something similar to:
-
Access the search history by selecting history from the menu under the gear icon on the right-hand side of the search results interface, or by clicking on the more link within the recent searches panel.
-
The full list of recent searches is displayed along with the number of results for each search and the time at which the search was performed. Clear the history by clicking on the clear button. When prompted confirm that the history should be cleared.
Result collapsing
Result collapsing groups similar search results together when displaying search results. The key or keys used for collapsing are configured by a search administrator and are based on either the documents content, or the document’s metadata.
For example, the result item below from a search of the complete works of Shakespeare shows that there are 21 very similar results to the Merchant of Venice: list of scenes result. This search has been configured to group or collapse all search results that are tagged with the same metadata value for a field recording the name of the play.
Clicking on the 21 very similar results lists the 22 search results that form part of this group (the list of scenes result plus the 21 similar results). These can be sorted by providing a sort parameter in your More… link or by setting -sort in your query processor options. The sort options for this list of similar results are the same as for any normal search result sort.
The initial search result can also be customized to show some additional information for the collapsed results.
The example result below, from another site groups sections of a publication together. The search result is configured to display the title of the first four collapsed items with a More… link which is the same as the 21 similar items link in the previous example.
These (nested) collapsed results can be sorted using the -collapsed_docs_sort query processor option. This option will affect the set of collapsed results that are returned within the intial search response (controlled by the -collapsed_num_ranks option).
Quick links
Quick links are direct links to important sub-pages of a home page and can be displayed as part of the result summary. Quick links are normally only used when providing a multi-site search such as a whole of university search. The search result can also include an optional search box that restricted to the specific site displayed for the search result.
For example a search for the University of Sydney might produce the following search result. Entering a search into the search box will run a search restricted to pages from the Sydney Conservatorium of Music website.
Advanced search results templating
Funnelback has an extremely flexible templating language. This means that search results can be adapted to any site design and can be used in innovative ways.
For example, search results could be returned in a table, or even plotted onto a map if geospatial coordinate metadata is available:
Search results can also be templated to any text-based format.
This means that results could be returned in CSV, JSON, XML or other custom text formats.