Nuxeo Server

Elasticsearch Indexing Logic

Updated: November 13, 2017 Page Information Edit on GitHub

 
Extract from the course What's New in Nuxeo Platform LTS 2015? in Nuxeo University

Indexing

When manipulating a session to create, update or delete documents, a synchronous listener stacks the indexing commands to process. These commands are factorized and are processed either in an asynchronous job or at post commit time.

Post Commit Mode

If the commands are recorded on a UI thread (a thread used to render a JSF page for instance) the commands are treated in post commit. This means that after the transaction is committed the indexing command are sent to Elasticsearch and a refresh operation is also send to make the indexed documents available to the next query. This approach give a real time indexing appearance. A document that is created by an action is searchable on the next action.

Asynchronous Mode

The asynchronous mode will process the commands and not send any refresh operation so they are treated in near real time (~1s after the indexing command is send).

Recursive Commands

A command can be on a single document or applied to its children (recursive). So the number of command processed reported in the Admin tab doesn't have to match the number of document processed.

Recursive command that are triggered when moving a folder or changing an ACL are not treated in post commit listener. Only the first level is treated in post commit the recursive indexing is done asynchronously.

JSON Document

When indexing a document the Nuxeo Platform sends a JSON representation to be indexed. For now a creation or an update command submits the complete document. The JSON document can be viewed in the _source field of the Elasticsearch document. The _source contains all the fields.

Searching and Limitations

NXQL Queries

A NXQL query can be translated to Elasticsearch query with some limitations. See the page NXQL documentation.

Operators and Mapping

Some operators need an explicit mapping to work properly. This is the case for FULLTEXT and ILIKE operators (STARTSWITH for ecm:path has a special mapping setup by default). See the page Configuring the Elasticsearch Mapping for more information.

Security and ACLs

The security clause is automatically added to match the principal and its groups. Each document contains the list of the users or groups that have permission to browse the document.

Only the simplified ACL is supported with Elasticsearch (this is the default security mode since 6.0). Simplified ACL means we only handle DENY on Everyone (block all rights) and not DENY on principals.

 


5 days ago manonlumeau Added content-review-lts2017 label
a month ago manonlumeau NXDOC-1346-FT review screenshot
2 years ago Bertrand Chauvin 32
2 years ago Bertrand Chauvin 31 | Added video
2 years ago Guillaume Renard 30
2 years ago Bertrand Chauvin 29 | Fixed typo
3 years ago Benoit Delbosc 28
3 years ago Benoit Delbosc 27
3 years ago Benoit Delbosc 26
3 years ago Benoit Delbosc 25
3 years ago Benoit Delbosc 24
3 years ago Solen Guitter 23 | Add subtitiles
3 years ago Benoit Delbosc 22
3 years ago Benoit Delbosc 21
3 years ago Alain Escaffre 20
3 years ago Alain Escaffre 19
3 years ago Benoit Delbosc 18
3 years ago Solen Guitter 17
3 years ago Alain Escaffre 16 | Removing the Indexing strategy part which is incomplete.
3 years ago Guillaume Renard 15
3 years ago Solen Guitter 14
4 years ago Alain Escaffre 13
4 years ago Alain Escaffre 12
4 years ago Solen Guitter 11
4 years ago Solen Guitter 10
4 years ago Solen Guitter 9
4 years ago Solen Guitter 8
4 years ago Alain Escaffre 7
4 years ago Alain Escaffre 6
4 years ago Alain Escaffre 5
4 years ago Alain Escaffre 4
4 years ago Alain Escaffre 3
4 years ago Alain Escaffre 2
4 years ago Alain Escaffre 1
History: Created by Alain Escaffre