OpenCms documentation
OpenCms documentation

Full text search with Solr

Solr is a mighty tool to perform full text search with many of extra features, such as (various kinds of) facets, "Did you mean?" functionality and highlighting with suitable selection of text snippets for an extract.

Implementing a fully featured Solr search can be involved and time consuming. Up to OpenCms 9.5 a client side Solr search was integrated in the demo template shipped with OpenCms. Even partly configurable, it was hard to adjust for the use in other web sites. In particular, a lot of GWT code that handled the state of the search form and rendered the output had to be touched.

The complexity of implementing a client-side search function led to the decision to add support for a server side search. This will remove the complexity of writing client side code and might be sufficient for many purposes. With the tag <cms:search>, introduced in OpenCms 10, writing a full text search for a website is done within just a few minutes or hours. And this search already supports facets, highlighting, pagination, etc.

Adding a full text search to your webpage

You should use the way to build a website search describe here for testing. For a productive search, the configuration of the search via JSON instead of the the XML content might be more appropriate. The XML content should not be editable by users having no knowledge about searching with Solr. They are likely to misconfigure it. But, the content is great for testing.

OpenCms 10 provides the content type "Search Configuration" (jsp-search), that is available in the "System elements" content view. It comes with a default formatter that already can handle all combination of configuration options. Just create a page where your search should be located and place a "Search Configuration" on it.

Since OpenCms 11 the default formatter is not part of OpenCms anymore. You can download it here. Configure it for the type "jsp-search" to use it.

Open the content to see the available options:

Via the configuration content all configuration options for <cms:search> are available. The different tabs represent the different search features. To get an overview, just click through the tabs, have a look at the options and read the tooltips. They are the best documentation.

You do not need to configure any option you do not use. Even with no configuration at all, you have a, thought simple, full text search.

The option you may need in nearly all cases is "Additional Solr Query Parameters" in the tab "Query options". Here you should specify your site root as parent folder, and possibly the types of content you want to search. See the tooltip for the editor field on how to do this.

As long as you have not written an own formatter for jsp-search, the default formatter is used. It can handle all the options, but may not yield the output you want. Hence, you should write your own formatter.

Writing your own full text search

The simplest way to implement a full-text search is to write a formatter for the jsp-search content, shipped with the OpenCms core in the module org.opencms.jsp.search and get your configuration right. For the formatter, you can start with the demo formatter referenced in the above section.

Do not let normal editors edit your search content directly. Your formatter may not be capable of all configuration options and you typically do not want to let persons not skilled for it change the configuration.

When you got everything correct, you rewrite your formatter JSP to be used for a dynamic function or a content type providing configuration options specific to your needs. The rewrite is very easy, since you only replace the confguration handed over to the <cms:search> tag. You may even keep your configuration content, or what might be more valueable, replace it with a JSON configuration. There's a one-to-one correspondence between the options in the XML content and in the JSON  configuration.. Everything else remains the same.

The formatter basically:

  • hands over the jsp-search content (or later the JSON configuration) to the <cms:search> tag as configuration content (or string).
  • explores the result object exposed by <cms:search> to build up the search form and render the search results.

The result exposed by <cms:search> is of type org.opencms.jsp.search.result.I_CmsSearchResultWrapper. A good explanation of its structure is given in the presentation of the tag on the OpenCms Days 2015. Moreover, you can explore the formatters for the full text search of

  • the Mercury demo template shipped with OpenCms 10.0 (the formatter is located in /system/modules/alkacon.mercury.template/functions/search.jsp)
  • the search included in this documentation (the formatter is located in /system/modules/com.alkacon.opencms.documentation.searchform/formatters/searchform-formatter.jsp)

as examples for using it.

When your formatter is finished, ensure that you set the fl parameter for your search correctly to prevent loading too much fields that you do not use in the formatter.

Moreover, for the JSON configuration, an example JSON file with all options is found in the test cases for OpenCms here.

Besides the configuration via an XML content and the use of <cms:search> in the formatter of that content, the tag can be configured via a JSON configuration that can be provided either as file or as string. These configuration options are typically preferable not only for full-text serach, but also for implementing lists of content.

Using a JSON string is in particular valuable if you want to provide your own content type, or a dynamic function with element settings or parameters for rendering the search form. Then you can use the content elements, element settings and parameters to adjust the JSON string. The benefit is a fine-grained control on the configuration options that are exposed. Moreover, it becomes possible to change configuration depending on the seach state, e.g., you can make the search configurable by the end user.

The JSON configuration provides the same options as the XML configuration content. An example is found here. To get an overview on all the configuration options, please look at the (JavaDoc and source code of the) configuration parser org.opencms.jsp.search.config.parser.CmsJSONSearchConfigurationParser.

Note that since OpenCms version 11, you can also overwrite options from the configuration file by the configuration string.

Preparing your content for the search

When your content is searchable, you should take care on how it is indexed. In particular, you should