Nuxeo Server

Nuxeo and Redis

Updated: November 13, 2017 Page Information Edit on GitHub

Principles

Nuxeo provides a Redis integration via the nuxeo-core-redis bundle.

The idea is that, at least for now, Redis is not a hard requirement for running the Nuxeo Platform; We simply use Redis as a backend to provide alternate implementation of some services inside the platform. However, we do provide these implementations because we think they can be useful.

Nuxeo can use Redis to store both data to be persisted (e.g. jobs list) and transient data (e.g. cache data). After a normal cluster shutdown, you can flush (erase) the transient data in Redis. Note however that Nuxeo can not work with a Redis configured as an LRU cache; there should be no eviction under memory pressure.

Nuxeo Core Cache

Nuxeo Core Cache provides a service to declare and use caches. This cache system:

  • is used in Nuxeo Directories (caching directories entries)
  • is used in UserManager (caching principals)
  • is used in Nuxeo Drive (caching synchronization roots)
  • can be used in your custom code 

The Cache has a default "in memory" implementation, but nuxeo-core-redis provides another implementation that allows:

  • To have out of JVM memory cache storage. It opens the way to have big caches without hurting the JVM.
  • To share the same cache between several Nuxeo nodes. In cluster mode this can increase cache efficiency.
  • To manage cluster wide invalidations. Updating the user on one node will impact the central cache: all nodes see the exact same data.

You can configure the backend storage on a per cache basis: Directory A could use Redis while directory B could use in memory.

Work Manager

The WorkManager handles asynchronous jobs:

  • Schedule Jobs and store them in queues
  • Assign execution slots to queues 
  • Execute the jobs

In the default implementation, job queues are in the JVM memory. But this model has some limitations:

  • Stacking a lot of jobs will consume JVM Memory
  • In cluster mode each Nuxeo node maintains its own queue
  • When a Nuxeo server is restarted, all the queued jobs are lost

nuxeo-core-redis provides an alternate implementation of the queuing system based on Redis:

  • Jobs are then stored outside of JVM memory. That's why Work have to be serializable: they are serialized and stored inside Redis.
  • Jobs can be shared across cluster nodes. This allows to dedicate some nodes to some specific processing.
  • Jobs survive a Nuxeo restart. When Redis persistence is activated, the jobs even survive a Redis restart.

Lock Manager

By default Lock managed on documents are stored inside the repository backend.When the locking system is heavily used, it can create a lot of very small transactions against the database backend.

nuxeo-core-redis provides a Redis-based implementation of the locking system that is resilient as the database implementation, but easier to scale.

Clustering invalidation

VCS Row Cache Invalidation

Managing VCS row cache invalidations with Redis instead of using the database can improve performance on concurrent writes and provides synchronous invalidations.

DBS Cache Invalidation

Since Nuxeo 8.10 the DBS layer (used by MongoDB or Marklogic backends) has a cache its invalidation in cluster mode requires Redis.

Transient Store

The RedisTransientStore is a Redis-based implementation of the Transient Store.

It is the one used by the default Transient Store if Redis is enabled.

It allows the parameters associated to the stored blobs to be shared accross cluster nodes.

 


   
5 days ago manonlumeau Added content-review-lts2017 label
a month ago manonlumeau NXDOC-1346-FT review screenshot
a year ago Benoit Delbosc 15
a year ago Manon Lumeau 14
2 years ago Benoit Delbosc 13
2 years ago Antoine Taillefer 12
2 years ago Antoine Taillefer 11
2 years ago Manon Lumeau 10 | Formatting
2 years ago Benoit Delbosc 9
2 years ago Benoit Delbosc 8
2 years ago Benoit Delbosc 7 | Add a note about VCS row invalidation
3 years ago Manon Lumeau 6
3 years ago Solen Guitter 5
3 years ago Solen Guitter 4
3 years ago Solen Guitter 3
3 years ago Thierry Delprat 2
3 years ago Thierry Delprat 1
History: Created by Thierry Delprat