Transient Store

A Transient Store allows you to store temporary blobs and associated parameters (file name, MIME type, etc.) in a Nuxeo instance but outside of the document repository, thus the "transient" aspect.

It is typically used by:

• The Batch Upload API to temporarily store a batch of uploaded blobs until they are attached to a document.
• The ConversionService to store the BlobHolder resulting from an Asynchronous Conversion Work.

The TransientStore API is based on the following methods:

void putBlobs(String key, List blobs);
void putParameter(String key, String parameter, Serializable value);
void putParameters(String key, Map parameters);

List getBlobs(String key);
Serializable getParameter(String key, String parameter);
Map getParameters(String key);

Configuration

You can configure any number of transient stores with the following extension point:

<extension target="org.nuxeo.ecm.core.transientstore.TransientStorageComponent"
  point="store">
  <store name="myTransientStore" class="my.transientstore.implementation">
    <targetMaxSizeMB>-1</targetMaxSizeMB>
    <absoluteMaxSizeMB>-1</absoluteMaxSizeMB>
    <firstLevelTTL>240</firstLevelTTL>
    <secondLevelTTL>10</secondLevelTTL>
  </store>
</extension>

The store element supports two attributes:

• name: Used to identify the store.
• class: Optionally references an implementation of the TransientStore interface (the default is SimpleTransientStore, or RedisTransientStore if Redis is enabled).

The nested configuration elements are:

• targetMaxSizeMB: The target size that ideally should never be exceeded.
• absoluteMaxSizeMB: The size that must never be exceeded.
• firstLevelTTL: TTL in minutes of the first level cache.
• secondLevelTTL: TTL in minutes of the second level cache.

Have a look at the default transient store configuration, defined in a template:

<?xml version="1.0"?>
<component name="org.nuxeo.ecm.core.transient.store.config">
  <#if "${nuxeo.redis.enabled}" == "true">
    <#assign className = "org.nuxeo.ecm.core.redis.contribs.RedisTransientStore" />
  <#else>
    <#assign className = "org.nuxeo.ecm.core.transientstore.SimpleTransientStore" />
  </#if>
  <extension target="org.nuxeo.ecm.core.transientstore.TransientStorageComponent"
    point="store">
    <store name="default" class="${className}">
      <targetMaxSizeMB>-1</targetMaxSizeMB>
      <absoluteMaxSizeMB>-1</absoluteMaxSizeMB>
      <firstLevelTTL>240</firstLevelTTL>
      <secondLevelTTL>10</secondLevelTTL>
      <minimalRetention>10</minimalRetention>
    </store>
  </extension>
</component>

In this template the class is dynamically defined depending on whether Redis is enabled or not. If you need to define a custom transient store we strongly recommend you use such a template with this dynamic class definition mechanism so that:

• In development mode, where Redis is usually not enabled, you rely on the in-memory cache implementation.
• In cluster mode, where Redis needs to be enabled, the data stored in the transient store is shared between cluster nodes.

To retrieve a given transient store, just call TransientStoreService#getStore(String name). If the specified transient store hasn't been registered, the default one is used instead.

Implementation

As seen above there are two default implementations available in the default Nuxeo platform. They both handle blob storage in the same way but store the parameters differently.

A new implementation, KeyValueBlobTransientStore, is also available.

Blob Storage

The transient blob storage directory used by the two default implementations of the transient store is located in $DATA_DIR/transientstores/<transientstore_name>. Typically for the default transient store on a Nuxeo instance where the data directory is not externalized, it is: $NUXEO/nxserver/data/transientstores/default.

Parameter Storage

The SimpleTransientStore implementation relies on Nuxeo's in-memory cache implementation which uses Google's Guava cache. It is not distributed and not persistent so should not be used in a clustered setting.

The RedisTransientStore relies on Redis. It is distributed and persistent.

See NXP-18051 for details about the RedisTransientStore cluster-aware implementation.

KeyValueBlobTransientStore

This implementation, available since Nuxeo 9.3, allows you to abstract yourself from Redis and the filesystem, using a Key/Value store for the parameters and a Blob Provider for the blobs. It can be set up by defining an XML contribution like:

<extension target="org.nuxeo.ecm.core.transientstore.TransientStorageComponent" point="store">
  <store name="myname" class="org.nuxeo.ecm.core.transientstore.keyvalueblob.KeyValueBlobTransientStore">
    <targetMaxSizeMB>-1</targetMaxSizeMB>
    <absoluteMaxSizeMB>-1</absoluteMaxSizeMB>
    <firstLevelTTL>240</firstLevelTTL>
    <secondLevelTTL>10</secondLevelTTL>
  </store>
</extension>

It will read and write parameters in a Key/Value store of the same name, and the binaries in a Blob Provider of the same name (myname in this example). Both must be configured through their own extension points.

Time To Live and Garbage Collector

Entries have a Time To Live (TTL) defined in the transient store configuration.

Initially the TTL of an entry is set to firstLevelTTL. It can later be set to secondLevelTTL by calling TransientStore#release(String key).

The garbage collection:

• Is scheduled every 15 minutes to wipe the files associated to entries that don't exist anymore.
• Can be triggered manually on a given store with TransientStore#doGC or all stores with TransientStoreService#doGC.

Example Services Using a Transient Store

BatchManager

It relies on the BatchManagerCache transient store which in fact is not registered so it falls back on the default transient store, this is to allow overriding the configuration if needed.

See the "Batch Upload Example" section of NXP-18050 for details.

TransientStoreWork

A Work allowing to store a result in a transient store.

It relies on the transientStoreWorkCache transient store, also not registered with a fallback on the default transient store.

For example, the Asynchronous Conversions rely on such instances of TransientStoreWork via the ConversionWork class.

---