Class KeyValueBlobTransientStore
- java.lang.Object
-
- org.nuxeo.ecm.core.transientstore.keyvalueblob.KeyValueBlobTransientStore
-
- All Implemented Interfaces:
TransientStore
,TransientStoreProvider
public class KeyValueBlobTransientStore extends Object implements TransientStoreProvider
Transient Store storing properties in a Key/Value store, and storing blobs using a Blob Provider.This transient store is configured with the following properties:
- keyValueStore: the name of the key/value store to use. If not provided, it defaults to "transient_" + the transient store name.
- blobProvider: the name of the blob provider to use. If not provided, it defaults to "transient_" + the transient store name.
- defaultBlobProvider: if the configured or defaulted blob provider doesn't exist, a namespaced copy of this one will be used instead. The default is "default".
The storage format is the following:
__blobsize__: storage size; because entries may expire without us being notified due to their TTL, this may be higher than the actual storage size entryKey.completed: "true" if completed, "false" if not; presence of this key marks entry existence entryKey.paraminfo: ["foo", "bar"] entryKey.param.foo: value for param foo entryKey.param.foo__format: "java" for java serializable format, otherwise string entryKey.param.bar: value for param bar etc. entryKey.bloblock: "true" if there is a blob read/write in progress, null otherwise entryKey.blobinfo: {"count": number of blobs, "size": storage size of the blobs} entryKey.blob.0: {"key": key in blob provider for first blob, "mimetype": MIME Type, "encoding": encoding, "filename": filename, "digest": digest} entryKey.blob.1: {...} same for second blob etc.
- Since:
- 9.3
-
-
Field Summary
Fields Modifier and Type Field Description protected long
absoluteMaxSize
protected static int
BLOB_LOCK_TTL
protected String
blobProviderId
static String
CONFIG_BLOB_PROVIDER
static String
CONFIG_DEFAULT_BLOB_PROVIDER
static String
CONFIG_DEFAULT_BLOB_PROVIDER_DEFAULT
static String
CONFIG_KEY_VALUE_STORE
static String
COUNT
protected String
defaultBlobProviderId
static String
DIGEST
static String
DOT_BLOB_DOT
static String
DOT_BLOBINFO
static String
DOT_BLOBLOCK
static String
DOT_COMPLETED
static String
DOT_PARAM_DOT
static String
DOT_PARAMINFO
static String
ENCODING
static String
FILENAME
static String
FORMAT
static String
FORMAT_JAVA
static String
KEY
protected String
keyValueStoreName
static String
LENGTH
protected static com.fasterxml.jackson.core.type.TypeReference<List<String>>
LIST_STRING
protected static long
LOCK_ACQUIRE_TIME_NANOS
protected static long
LOCK_EXPONENTIAL_BACKOFF_AFTER_NANOS
protected static com.fasterxml.jackson.core.type.TypeReference<Map<String,String>>
MAP_STRING_STRING
protected com.fasterxml.jackson.databind.ObjectMapper
mapper
static String
MIMETYPE
protected String
name
protected int
releaseTTL
TTL used to keep objects around a bit longer if there's space for them, for caching.static String
SEP
static String
SIZE
static String
STORAGE_SIZE
protected long
targetMaxSize
protected int
ttl
Basic TTL for all entries.protected static long
WARN_DURATION_MS_THRESHOLD
-
Constructor Summary
Constructors Constructor Description KeyValueBlobTransientStore()
-
Method Summary
All Methods Instance Methods Concrete Methods Deprecated Methods Modifier and Type Method Description protected void
acquireBlobLockOrThrow(String key, KeyValueStore kvs)
protected boolean
acquireLock(BooleanSupplier tryAcquireOnce)
protected void
addStorageSize(long delta)
Deprecated.since 11.1protected void
addStorageSize(long delta, KeyValueStore kvs)
void
atomicUpdate(String key, Function<String,String> updateFunction, long ttl)
Deprecated.since 11.1protected void
atomicUpdate(String key, Function<String,String> updateFunction, long ttl, KeyValueStore kvs)
protected void
computeStorageSize()
Computes an exact value for the current storage size (sum of all blobs size).void
doGC()
Runs garbage collecting to delete the file system resources that are associated with entries that were removed.boolean
exists(String key)
Returnstrue
if an entry exists with the givenkey
.protected List<String>
getBlobKeys(String key)
BlobProvider
getBlobProvider()
List<Blob>
getBlobs(String key)
Gets the blobs associated with the entry with the givenkey
.protected KeyValueStore
getKeyValueStore()
Serializable
getParameter(String key, String parameter)
Gets the value ofparameter
in the entry with the givenkey
.Map<String,Serializable>
getParameters(String key)
Gets values of the parameters in the entry with the givenkey
.long
getSize(String key)
Returns the size of the blobs associated with the entry with the givenkey
or-1
if entry does not exist.long
getStorageSize()
Returns the size (in bytes) of the disk storage used for blobs.void
init(TransientStoreConfig config)
Initializes the store from the givenconfig
.boolean
isCompleted(String key)
Returnstrue
if the entry with the givenkey
is ready.protected List<String>
jsonToList(String json)
protected Map<String,String>
jsonToMap(String json)
Stream<String>
keyStream()
Returns aStream
of keys for all entries.protected void
markEntryExists(String key)
Deprecated.since 11.1protected void
markEntryExists(String key, KeyValueStore kvs)
void
putBlobs(String key, List<Blob> blobs)
Associates the givenblobs
with the entry with the givenkey
.void
putParameter(String key, String parameter, Serializable value)
Setsparameter
tovalue
in the entry with the givenkey
.void
putParameters(String key, Map<String,Serializable> parameters)
Putsparameters
in the entry with the givenkey
.void
release(String key)
Informs the store that the entry with the givenkey
can be released if TTL or GC parameters require to do some cleanup.protected void
releaseBlobLock(String key, KeyValueStore kvs)
void
remove(String key)
Removes entry with the givenkey
.void
removeAll()
Removes all entries from the store.protected void
removeBlobs(String key)
Deprecated.since 11.1protected void
removeBlobs(String key, KeyValueStore kvs)
protected void
removeCompleted(String key)
Deprecated.since 11.1protected void
removeCompleted(String key, KeyValueStore kvs)
protected void
removeParameters(String key)
Deprecated.since 11.1protected void
removeParameters(String key, KeyValueStore kvs)
void
setCompleted(String key, boolean completed)
Marks the entry with the givenkey
as ready.protected void
setReleaseTTL(String key)
void
shutdown()
Shuts down the store.protected String
toJson(Object object)
protected boolean
tryAcquireBlobLock(String key, KeyValueStore kvs)
protected boolean
tryAcquireOnceBlobLock(String key, KeyValueStore kvs)
-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
Methods inherited from interface org.nuxeo.ecm.core.transientstore.api.TransientStoreProvider
getStorageSizeMB, keySet
-
-
-
-
Field Detail
-
SEP
public static final String SEP
- See Also:
- Constant Field Values
-
STORAGE_SIZE
public static final String STORAGE_SIZE
- See Also:
- Constant Field Values
-
DOT_COMPLETED
public static final String DOT_COMPLETED
- See Also:
- Constant Field Values
-
DOT_PARAMINFO
public static final String DOT_PARAMINFO
- See Also:
- Constant Field Values
-
DOT_PARAM_DOT
public static final String DOT_PARAM_DOT
- See Also:
- Constant Field Values
-
FORMAT
public static final String FORMAT
- See Also:
- Constant Field Values
-
FORMAT_JAVA
public static final String FORMAT_JAVA
- See Also:
- Constant Field Values
-
DOT_BLOBLOCK
public static final String DOT_BLOBLOCK
- Since:
- 11.1
- See Also:
- Constant Field Values
-
DOT_BLOBINFO
public static final String DOT_BLOBINFO
- See Also:
- Constant Field Values
-
COUNT
public static final String COUNT
- See Also:
- Constant Field Values
-
SIZE
public static final String SIZE
- See Also:
- Constant Field Values
-
DOT_BLOB_DOT
public static final String DOT_BLOB_DOT
- See Also:
- Constant Field Values
-
KEY
public static final String KEY
- See Also:
- Constant Field Values
-
MIMETYPE
public static final String MIMETYPE
- See Also:
- Constant Field Values
-
ENCODING
public static final String ENCODING
- See Also:
- Constant Field Values
-
FILENAME
public static final String FILENAME
- See Also:
- Constant Field Values
-
LENGTH
public static final String LENGTH
- See Also:
- Constant Field Values
-
DIGEST
public static final String DIGEST
- See Also:
- Constant Field Values
-
CONFIG_KEY_VALUE_STORE
public static final String CONFIG_KEY_VALUE_STORE
- See Also:
- Constant Field Values
-
CONFIG_BLOB_PROVIDER
public static final String CONFIG_BLOB_PROVIDER
- See Also:
- Constant Field Values
-
CONFIG_DEFAULT_BLOB_PROVIDER
public static final String CONFIG_DEFAULT_BLOB_PROVIDER
- Since:
- 11.1
- See Also:
- Constant Field Values
-
CONFIG_DEFAULT_BLOB_PROVIDER_DEFAULT
public static final String CONFIG_DEFAULT_BLOB_PROVIDER_DEFAULT
- Since:
- 11.1
- See Also:
- Constant Field Values
-
BLOB_LOCK_TTL
protected static final int BLOB_LOCK_TTL
- Since:
- 11.1
- See Also:
- Constant Field Values
-
LOCK_ACQUIRE_TIME_NANOS
protected static final long LOCK_ACQUIRE_TIME_NANOS
- Since:
- 11.1
-
LOCK_EXPONENTIAL_BACKOFF_AFTER_NANOS
protected static final long LOCK_EXPONENTIAL_BACKOFF_AFTER_NANOS
- Since:
- 11.1
-
WARN_DURATION_MS_THRESHOLD
protected static final long WARN_DURATION_MS_THRESHOLD
- Since:
- 2021.17
- See Also:
- Constant Field Values
-
name
protected String name
-
keyValueStoreName
protected String keyValueStoreName
-
blobProviderId
protected String blobProviderId
-
defaultBlobProviderId
protected String defaultBlobProviderId
-
ttl
protected int ttl
Basic TTL for all entries.
-
releaseTTL
protected int releaseTTL
TTL used to keep objects around a bit longer if there's space for them, for caching.
-
targetMaxSize
protected long targetMaxSize
-
absoluteMaxSize
protected long absoluteMaxSize
-
mapper
protected com.fasterxml.jackson.databind.ObjectMapper mapper
-
LIST_STRING
protected static final com.fasterxml.jackson.core.type.TypeReference<List<String>> LIST_STRING
-
-
Method Detail
-
init
public void init(TransientStoreConfig config)
Description copied from interface:TransientStoreProvider
Initializes the store from the givenconfig
.- Specified by:
init
in interfaceTransientStoreProvider
-
getKeyValueStore
protected KeyValueStore getKeyValueStore()
-
getBlobProvider
public BlobProvider getBlobProvider()
-
shutdown
public void shutdown()
Description copied from interface:TransientStoreProvider
Shuts down the store.- Specified by:
shutdown
in interfaceTransientStoreProvider
-
keyStream
public Stream<String> keyStream()
Description copied from interface:TransientStoreProvider
Returns aStream
of keys for all entries.- Specified by:
keyStream
in interfaceTransientStoreProvider
-
getStorageSize
public long getStorageSize()
Description copied from interface:TransientStoreProvider
Returns the size (in bytes) of the disk storage used for blobs.- Specified by:
getStorageSize
in interfaceTransientStoreProvider
- Returns:
- the number of bytes used by stored blobs
-
addStorageSize
@Deprecated protected void addStorageSize(long delta)
Deprecated.since 11.1
-
addStorageSize
protected void addStorageSize(long delta, KeyValueStore kvs)
-
computeStorageSize
protected void computeStorageSize()
Computes an exact value for the current storage size (sum of all blobs size). THIS METHOD IS COSTLY.Does not take into account blob de-duplication that may be done by the blob provider.
Does not take into account blobs that still exist in the blob provider but are not referenced anymore (due to TTL expiration or GC not having been done).
-
doGC
public void doGC()
Description copied from interface:TransientStoreProvider
Runs garbage collecting to delete the file system resources that are associated with entries that were removed.- Specified by:
doGC
in interfaceTransientStoreProvider
-
removeAll
public void removeAll()
Description copied from interface:TransientStoreProvider
Removes all entries from the store.- Specified by:
removeAll
in interfaceTransientStoreProvider
-
atomicUpdate
@Deprecated public void atomicUpdate(String key, Function<String,String> updateFunction, long ttl)
Deprecated.since 11.1
-
atomicUpdate
protected void atomicUpdate(String key, Function<String,String> updateFunction, long ttl, KeyValueStore kvs)
-
exists
public boolean exists(String key)
Description copied from interface:TransientStore
Returnstrue
if an entry exists with the givenkey
.- Specified by:
exists
in interfaceTransientStore
-
markEntryExists
@Deprecated protected void markEntryExists(String key)
Deprecated.since 11.1
-
markEntryExists
protected void markEntryExists(String key, KeyValueStore kvs)
-
putParameter
public void putParameter(String key, String parameter, Serializable value)
Description copied from interface:TransientStore
Setsparameter
tovalue
in the entry with the givenkey
.If entry does not exist a new entry is created. If
parameter
already exists in the entry it is overwritten.- Specified by:
putParameter
in interfaceTransientStore
-
getParameter
public Serializable getParameter(String key, String parameter)
Description copied from interface:TransientStore
Gets the value ofparameter
in the entry with the givenkey
.Returns
null
if entry or parameter does not exist.- Specified by:
getParameter
in interfaceTransientStore
-
putParameters
public void putParameters(String key, Map<String,Serializable> parameters)
Description copied from interface:TransientStore
Putsparameters
in the entry with the givenkey
. Overwrites any existing parameter in the entry.If entry does not exist a new entry is created.
- Specified by:
putParameters
in interfaceTransientStore
-
getParameters
public Map<String,Serializable> getParameters(String key)
Description copied from interface:TransientStore
Gets values of the parameters in the entry with the givenkey
.Returns
null
if entry does not exist.- Specified by:
getParameters
in interfaceTransientStore
-
removeParameters
@Deprecated protected void removeParameters(String key)
Deprecated.since 11.1
-
removeParameters
protected void removeParameters(String key, KeyValueStore kvs)
-
putBlobs
public void putBlobs(String key, List<Blob> blobs)
Description copied from interface:TransientStore
Associates the givenblobs
with the entry with the givenkey
.If entry does not exist a new entry is created.
- Specified by:
putBlobs
in interfaceTransientStore
-
removeBlobs
@Deprecated protected void removeBlobs(String key)
Deprecated.since 11.1
-
removeBlobs
protected void removeBlobs(String key, KeyValueStore kvs)
-
getBlobs
public List<Blob> getBlobs(String key)
Description copied from interface:TransientStore
Gets the blobs associated with the entry with the givenkey
.Returns
null
if entry does not exist.- Specified by:
getBlobs
in interfaceTransientStore
-
acquireBlobLockOrThrow
protected void acquireBlobLockOrThrow(String key, KeyValueStore kvs)
-
tryAcquireBlobLock
protected boolean tryAcquireBlobLock(String key, KeyValueStore kvs)
-
tryAcquireOnceBlobLock
protected boolean tryAcquireOnceBlobLock(String key, KeyValueStore kvs)
-
releaseBlobLock
protected void releaseBlobLock(String key, KeyValueStore kvs)
-
acquireLock
protected boolean acquireLock(BooleanSupplier tryAcquireOnce)
-
getSize
public long getSize(String key)
Description copied from interface:TransientStore
Returns the size of the blobs associated with the entry with the givenkey
or-1
if entry does not exist.- Specified by:
getSize
in interfaceTransientStore
-
isCompleted
public boolean isCompleted(String key)
Description copied from interface:TransientStore
Returnstrue
if the entry with the givenkey
is ready.- Specified by:
isCompleted
in interfaceTransientStore
-
setCompleted
public void setCompleted(String key, boolean completed)
Description copied from interface:TransientStore
Marks the entry with the givenkey
as ready.If entry does not exist a new entry is created.
- Specified by:
setCompleted
in interfaceTransientStore
-
removeCompleted
@Deprecated protected void removeCompleted(String key)
Deprecated.since 11.1
-
removeCompleted
protected void removeCompleted(String key, KeyValueStore kvs)
-
release
public void release(String key)
Description copied from interface:TransientStore
Informs the store that the entry with the givenkey
can be released if TTL or GC parameters require to do some cleanup.Has no effect if entry does not exist.
- Specified by:
release
in interfaceTransientStore
-
setReleaseTTL
protected void setReleaseTTL(String key)
-
remove
public void remove(String key)
Description copied from interface:TransientStore
Removes entry with the givenkey
.Has no effect if entry does not exist.
- Specified by:
remove
in interfaceTransientStore
-
-