Home > Server > Data Store > Indexing and Query
Server

Indexing and Query

Updated: April 14, 2025

Hyland University
Watch the related course on Hyland University:
Video on Indexes from the Data Persistence course
university_indexes.png
university_indexes.png

Architecture

Since Nuxeo LTS 2025, we aim to support different search engines (Elasticsearch, OpenSearch, etc.). In order to be independent of the search engine's release cycle and end-of-life, this support is provided by installing a search client package on top of the Nuxeo Platform.

Data store and index

The Nuxeo Platform stores documents and their property values either in a database (VCS) or in a NoSQL database (DBS). This data can at the same time be indexed by a search engine. To query those documents, several approaches are offered by the platform depending on whether you are from a remote application or in some Java code executed server-side. In the end, all the methods lead to two possibilities:

  • Query the data store (VCS or DBS). Queries there are "transactional", which means that the result will reflect exactly the state of the database in the transaction where the query is executed
  • Query a search engine index. This is the most scalable and efficient way to perform a query. Benchmarks show querying the repository using a search engine such as Elasticsearch scales orders of magnitude better than the database.

A Query Language

The natural way of expressing a query in the Nuxeo Platform is with NXQL, the Nuxeo Query Language. 

SELECT * FROM Document WHERE
dc:contributors = ?                          -- simple match on a multi-valued field
AND ecm:mixinType != 'Folderish'             -- use facet to remove all folderish documents
AND ecm:mixinType != 'HiddenInNavigation'    -- use facet to remove all documents that should be hidden
AND ecm:isVersion = 0                        -- only get checked-out documents
AND ecm:isProxy = 0                          -- don't return proxies
AND ecm:isTrashed = 0                        -- don't return documents that are in the trash

As you may see, there is no security clause, because the repository will always only return documents that the current user can see. Security filtering is built-in, so you don't have to post-filter results returned by a search, even if you use complex custom security policies.

The Nuxeo Platform is also compatible with the CMISQL defined in the CMIS standard.

Page Providers: a Pagination Service

The framework also provides a paginated query system, the Page Providers. Page Providers are a way to expose a query defined in NXQL with additional services: pagination, parameters, maximum number of results, aggregates definition. Page providers are named and declared to the server via a contribution. More information can be found about the page provider object. Page providers are used in the platform in many places: Web application for browsing, for dashboards, and so on.

Resources Endpoint are also based on a page provider. By being declarative, page providers are very easy to override. That way, most of the document lists logic of the default application can be redefined just by overriding the corresponding page provider. You can also build your own application in the same way. Note that in the web application, page providers are associated to a higher concept, the Content View, that wraps all the UI aspects of executing and presenting a search result (see paragraph below).

How to Query the Repository

The following table and schema gives an overview of the different ways of querying the repository.

  1. Search Endpoint (Client side) A resource oriented REST API that allows to execute direct NXQL queries or to use a named page provider that has been declared server side. The API returns serialised JSON documents and offers all the mechanisms provided by Nuxeo Platform Rest API (Content Enricher, Specific headers…).

    Example:

    http://localhost:8080/nuxeo/site/api/v1/search/lang/NXQL/execute?query=select * from Document&pageSize=2&currentPageIndex=1

    Related topics:

  2. Command Operations (Client side) A set of Automation operations allow to query a page provider that has been declared on the server. Scope is pretty much the same as with the search endpoint.

    Example:

    curl -XPOST -u Administrator:Administrator -H"Content-Type: application/json+nxrequest; charset=UTF-8" http://localhost:8080/nuxeo/site/automation/Repository.PageProvider -d '{"params":{"providerName":"default_document_suggestion"}}'

    Related topics:

  3. CMIS (Client side & Server side) The Nuxeo Platform is compatible with the CMIS standard. CMIS covers query scope, using CMISQL. It is possible to query the Nuxeo Platform repository using CMISQL in Java server side, or via JSON-based or AtomPub bindings remotely.

    Example:

    ItemIterable<QueryResult> results = session.query("SELECT * FROM cmis:document" , false);
for (QueryResult hit: results) {
  for(PropertyData<?> property: hit.getProperties()) {
    String queryName = property.getQueryName();
    Object value = property.getFirstValue();
  }
}

    Related topics:

  4. PageProvider (Server side) Page Providers objects implement the PageProvider interface. It provides in Java all the primitives for getting each documents, pages and related information.

    Example

    PageProvider<DocumentModel> pp = (PageProvider<DocumentModel>) ppService.getPageProvider(
        "TREE_CHILDREN_PP", null, null, null, props,
        new Object[] { myDoc.getId() });
List<DocumentModel> documents = pp.getCurrentPage();

    Related topics:

  5. CoreSession.query (Server side) The CoreSession object is the main server side Java interface for accession among the repository. Among available methods is the query() that allows to perform directly an NXQL query and get a list of documentModels (the basic Java wrapping of a Nuxeo Document). In most of the situations it is better to rely on a page provider as it is easier to override, maintain, etc… but session.query() is still an option.
  6. CoreSession.queryAndFetch() (Server side) Like session.query(), CoreSession.queryAndFetch() provides a way to perform an NXQL query and get an iterable of Java Map instead of DocumentModel.
  7. CoreSession.queryProjection() (Server side) Methods queryProjection() allow to perform an NXQL query in order to get a page of projections as Java Map. Note: query() allows to get a page too, but you get a DocumentModelList as result.
  8. SearchService.search() (Server side) Since Nuxeo LTS 2025, there is a new SearchService which enables to perform an NXQL query on one or multiple repositories using available indexes. This new service provides a uniform way to search on the repository, OpenSearch 1.x, ElasticSearch 7.x or 8.x and future versions as soon as new search client packages are available. This supersedes the legacy ElasticSearchService.

SearchClient Configuration

Since Nuxeo LTS 2025, you have to explicitly choose and install a SearchClient package. By default the SearchService will be configured with a Repository SearchClient that is using the underlying repository data store (VCS or DBS) with limited capabilities regarding fulltext search, aggregation and scalability.

It's therefore recommended to rely on an external search engine like OpenSearch 1.x, ElasticSearch 7.x or 8.x cluster. For this purpose, you need to install thenuxeo-search-client-opensearch1 package.

It's also possible for testing purpose to run an embedded OpenSearch 1.x server in the same JVM as Nuxeo by installing the nuxeo-opensearch1-embed package.

This embedded mode is only for testing purpose and should not be used in production.

See the documentation to setup and configure an OpenSearch or Elasticsearch cluster.

Full-Text Capabilities

Both VCS/DBS implementations and OpenSearch SearchClient provide full-text search capabilities. Depending on the back end (Oracle, Postgres, SQL server, …) capabilities may be slightly different. The OpenSearch implementation performs bests in terms of relevancy, for configuring dictionaries, running the stemming, and so on. Thus it is advised to leverage a SearchService page provider when you want to do searches on full text index.

More documentation can be found about full-text search expressions.

You should also read carefully how you can tune the full-text index for maximizing the relevance depending on your context of use.

Facets and Other Aggregates Support

Aggregates are a way to compute additional information on a search result so as to group and count result items and project them against various axis. For instance, "in the search result, 5 of the documents have the value "Specifications" for the field dc:nature ".

The SearchService page provider implementation provides aggregates support. It is possible to define which aggregates can be requested to OpenSearch with each queries, and a mechanism is implemented so as to filter following queries with the aggregates system offered by OpenSearch.

It is possible to leverage aggregates both at the API level and in the user interface, where a set of dedicated aggregates widgets has been added. They can be used from Nuxeo Studio.

See the How-to about aggregates widgets.

  • Terms with nuxeo-checkbox-aggregation element:

    terms-vocabulary.png
    terms-vocabulary.png

  • Date Histograms and Date Ranges:

    date-histogram.png
    date-histogram.png

  • Terms with User & Groups with the nuxeo-dropdown-aggregation element:

    authors-suggestion.png
    authors-suggestion.png

  • Range:

    range.png
    range.png

Configuring Search Interfaces in the Nuxeo Platform

Searches have been conceptualized throughout the notion of “page providers” in the Nuxeo Platform. The Page Provider object holds all the necessary information for rendering a search: a NXQL query, pagination logics, quick filters, sorting capabilities etc. Page providers are used in a non-UI context: you need to generate the form layout and the corresponding result layouts to display the search to the users (in Nuxeo Web UI typically).

These elements are fully configurable via Nuxeo Studio which makes it a matter of a few minutes to configure new business specific search screens for your application users.

search-exemple.png
search-exemple.png

Indexing Logic

The section Search Indexing Logic provides more details on how documents are indexed in Elasticsearch.