Home > Server > Installation > Nuxeo and Redis
Server

Nuxeo and Redis

Updated: November 15, 2024

Principles

Nuxeo provides a Redis integration via the nuxeo-core-redis bundle.

If you are targetting to store more than hundred millions of document, you should consider using Kafka instead of Redis.

Redis is not a hard requirement for running the Nuxeo Platform; it is used as a backend to provide alternate implementation of some services inside the platform. However, we do provide these implementations because we think they can be useful.

Nuxeo can use Redis to store both data to be persisted (e.g. jobs list) and transient data (e.g. cache data). After a normal cluster shutdown, you can flush (erase) the transient data in Redis. Note however that Nuxeo can not work with a Redis configured as an LRU cache; there should be no eviction under memory pressure.

Using Nuxeo in cluster mode does not necessarily need Redis, for more info check the related documentation below.

Nuxeo Core Cache

Nuxeo Core Cache provides a service to declare and use caches. This cache system is used for:

  • Nuxeo Directories (caching directories entries)
  • UserManager (caching principals)
  • Nuxeo Drive (caching synchronization roots)

It can also be used in your custom code.

The Cache has a default "in memory" implementation, but nuxeo-core-redis provides another implementation that allows to:

  • Have an out of JVM memory cache storage.
    It opens the way to have big caches without hurting the JVM.
  • Share the same cache between several Nuxeo nodes.
    In cluster mode this can increase cache efficiency.
  • Manage cluster wide invalidations.
    Updating the user on one node will impact the central cache: all nodes see the exact same data.

You can configure the backend storage on a per cache basis: Directory A could use Redis while directory B could use in memory.

Work Manager

The WorkManager handles asynchronous jobs:

  • Schedule Jobs and store them in queues
  • Assign execution slots to queues
  • Execute the jobs

In the default implementation, job queues are in the JVM memory. But this model has some limitations:

  • Stacking a lot of jobs will consume JVM Memory
  • In cluster mode each Nuxeo node maintains its own queue
  • When a Nuxeo server is restarted, all the queued jobs are lost

nuxeo-core-redis provides an alternate implementation of the queuing system based on Redis:

  • Jobs are then stored outside of JVM memory.
    That's why Work have to be serializable: they are serialized and stored in Redis.
  • Jobs can be shared across cluster nodes.
    This allows to dedicate some nodes to some specific processing.
  • Jobs survive a Nuxeo restart.
    When Redis persistence is activated, the jobs even survive a Redis restart.

Lock Manager

By default Lock managed on documents are stored in the repository backend. When the locking system is heavily used, it can create a lot of very small transactions against the database backend.

nuxeo-core-redis provides a Redis-based implementation of the locking system that is resilient as the database implementation, but easier to scale.

Clustering invalidation

Visual Content Store (VCS) Row Cache Invalidation

Managing VCS (Meaning RDBMS) row cache invalidations with Redis instead of using the database can improve performance on concurrent writes and provides synchronous invalidations.

Document Based Storage (DBS) Cache Invalidation

For Nuxeo 8.10 and 9.10 the DBS layer (used to connect with MongoDB or Marklogic) has a cache its invalidation in cluster mode requires Redis.

A Nuxeo No-Redis cluster configuration can be applied at certain conditions, see: No-Redis Nuxeo Cluster Configuration for more info.

Transient Store

The RedisTransientStore is a Redis-based implementation of the Transient Store.

It is not anymore the default Transient Store when Redis is enabled since LTS2019, more details in: NXP-26581.

The old Redis Transient Store can still be used with nuxeo.transientstore.provider=redis set in the nuxeo.conf, but it is not cluster safe. Binaries are temporarily stored in the Nuxeo node's file system. More details in: NXP-21871.

Clean-up

After a Nuxeo node crash (low-level error, machine reboot, etc.), Redis may still contain data that Nuxeo didn't have the opportunity to clean up.

In the following Lua scripts, the nuxeo: prefix is used for Redis keys. If a different value is set for nuxeo.redis.prefix in nuxeo.conf, you should update the code samples accordingly. More detail in Configuring Nuxeo for Redis.

Running workers

The following code can be used with the Redis client to delete old workers marked as running:

local keys = redis.call('KEYS', 'nuxeo:work:run:*')

for _,k in ipairs(keys) do
    redis.call('DEL', k)
end

Copy this code to a file named delete_running_works.lua:

redis-cli --eval /path/to/delete_running_works.lua

It finds all keys prefixed with nuxeo:work:run and deletes them.

Scheduled workers

The following code can be used with the Redis client to delete old workers marked as scheduled:

local keys = redis.call('KEYS', 'nuxeo:work:sched:*')

for _,k in ipairs(keys) do
  redis.call('DEL', k)
end

Copy this code to a file named delete_scheduled_works.lua:

redis-cli --eval /path/to/delete_scheduled_works.lua

It finds all keys prefixed with nuxeo:work:sched and deletes them.

Related Documentation