Package org.nuxeo.ecm.blob.s3

Class S3BlobStore

java.lang.Object
org.nuxeo.ecm.core.blob.AbstractBlobStore
org.nuxeo.ecm.blob.s3.S3BlobStore
All Implemented Interfaces:
BlobStore
public class S3BlobStore extends AbstractBlobStore
Blob storage in S3.
Since:
11.1

  • Field Details

    • USER_METADATA_USERNAME

      protected static final String USER_METADATA_USERNAME
      See Also:

    • config

      protected final S3BlobStoreConfiguration config

    • amazonS3

      protected final com.amazonaws.services.s3.AmazonS3 amazonS3

    • bucketName

      protected final String bucketName

    • bucketPrefix

      protected final String bucketPrefix

    • pathStrategy

      protected final PathStrategy pathStrategy

    • pathSeparatorIsBackslash

      protected final boolean pathSeparatorIsBackslash

    • allowByteRange

      protected final boolean allowByteRange

    • useVersion

      protected final boolean useVersion
      If true, include the object version in the key.

    • useAsyncDigest

      protected volatile Boolean useAsyncDigest

    • gc

      protected final BinaryGarbageCollector gc

  • Constructor Details

  • Method Details

    • getS3BinaryManager

      public S3BlobStore getS3BinaryManager()

    • isMissingKey

      protected static boolean isMissingKey(com.amazonaws.AmazonServiceException e)

    • isNotImplemented

      protected static boolean isNotImplemented(com.amazonaws.AmazonServiceException e)

    • isBucketVersioningEnabled

      protected boolean isBucketVersioningEnabled()

    • hasVersioning

      public boolean hasVersioning()
      Description copied from interface: BlobStore
      Whether this blob store has versioning.

      With versioning, two writes may use the same key. The returned keys will include a different version number to distinguish the writes.

      Specified by:
      hasVersioning in interface BlobStore
      Overrides:
      hasVersioning in class AbstractBlobStore

    • useAsyncDigest

      public boolean useAsyncDigest()
      Description copied from class: AbstractBlobStore
      Whether this blob store is configured for async digest computation.
      Overrides:
      useAsyncDigest in class AbstractBlobStore

    • supportsAsyncDigest

      protected boolean supportsAsyncDigest()
      Checks that all repositories support queries on blob keys.

    • supportsAsyncDigest

      protected boolean supportsAsyncDigest(Repository repository)

    • bucketKey

      protected String bucketKey(String key)

    • writeBlobGeneric

      protected String writeBlobGeneric(BlobWriteContext blobWriteContext) throws IOException
      Description copied from class: AbstractBlobStore
      Writes the blob without using any store-to-store optimization.
      Overrides:
      writeBlobGeneric in class AbstractBlobStore
      Throws:
      IOException

    • writeFile

      protected String writeFile(String key, Path file, BlobContext blobContext, String fileTraceSource) throws IOException
      Writes a file with the given key and returns its version id.
      Throws:
      IOException

    • setMetadata

      protected void setMetadata(com.amazonaws.services.s3.model.ObjectMetadata objectMetadata, BlobContext blobContext)

    • getFile

      public BlobStore.OptionalOrUnknown<Path> getFile(String key)
      Description copied from interface: BlobStore
      Gets an already-existing file containing the blob for the given key, if present.

      Note that this method is best-effort, it may return unknown even though the blob exists in the store, it's just that it's not handily available locally in a file.

      Parameters:
      key - the blob key
      Returns:
      the file containing the blob, or empty if the blob cannot be found, or unknown if no file is available locally

    • getStream

      public BlobStore.OptionalOrUnknown<InputStream> getStream(String key) throws IOException
      Description copied from interface: BlobStore
      Gets the stream of the blob for the given key, if present.

      Note that this method is best-effort, it may return unknown even though the blob exists in the store, it's just that it's not efficient to return it as a stream.

      Parameters:
      key - the blob key
      Returns:
      the blob stream, or empty if the blob cannot be found, or unknown if no stream is efficiently available
      Throws:
      IOException

    • exists

      public boolean exists(String key)
      Description copied from interface: BlobStore
      Does a blob with the given key exist in the blob store.

      The existence of the blob is checked in the store where the blob is effectively stored. Intermediate cache is ignored.

      Parameters:
      key - the blob key
      Returns:
      true if it exists in the blob store

    • hasDefaultStorageClass

      public boolean hasDefaultStorageClass(String key)
      Description copied from interface: BlobStore
      Does a blob with the given key exist in the blob store with the default storage class.

      The existence of the blob is checked in the store where the blob is effectively stored. Intermediate cache is ignored. A blob with a cold storage storage class is not in default storage class and this method will return false for such a blob.

      Parameters:
      key - the blob key
      Returns:
      true if it exists in the blob store and has the default storage class

    • bucketKeyHasDefaultStorageClass

      protected boolean bucketKeyHasDefaultStorageClass(String bucketKey)

    • bucketKeyExists

      protected boolean bucketKeyExists(String bucketKey)

    • lengthOfBlob

      protected long lengthOfBlob(String key)
      Returns:
      object length, or -1 if missing

    • clear

      public void clear()
      Description copied from interface: BlobStore
      INTERNAL (TESTS). Clears the blob store of all its data.

    • readBlob

      public boolean readBlob(String key, Path dest) throws IOException
      Description copied from interface: BlobStore
      Reads a blob based on its key into the given file.
      Parameters:
      key - the blob key
      dest - the file to use to store the fetched data
      Returns:
      true if the file was fetched, false if the file was not found
      Throws:
      IOException

    • copyBlobIsOptimized

      public boolean copyBlobIsOptimized(BlobStore sourceStore)
      Description copied from interface: BlobStore
      Checks if blob copy/move from another blob store to this one can be done efficiently.
      Specified by:
      copyBlobIsOptimized in interface BlobStore
      Overrides:
      copyBlobIsOptimized in class AbstractBlobStore
      Parameters:
      sourceStore - the source store
      Returns:
      true if the copy/move can be done efficiently

    • copyOrMoveBlob

      public String copyOrMoveBlob(String key, BlobStore sourceStore, String sourceKey, boolean atomicMove) throws IOException
      Description copied from interface: BlobStore
      Writes a file based on a key, as an optimized copy/move from a source in another compatible blob store.

      The target key may be null, which is a signal from the caller that it has determined that deduplication is enabled and async digest computation is enabled, but the needed digest hasn't been computed, so this method should either find the digest in an efficient way if it can, or otherwise trigger an async digest computation.

      If the copy/move is requested to be atomic, then the destination file is created atomically. In case of atomic move, in some stores the destination will be created atomically but the source will only be deleted afterwards.

      The returned key may be different than the passed one when versioning is used.

      Parameters:
      key - the key; or null if the store should choose it or trigger async digest computation
      sourceStore - the source store
      sourceKey - the source key
      atomicMove - true for an atomic move, false for a regular copy
      Returns:
      the key of the copied/moved file, or null if copy/move failed
      Throws:
      IOException

    • copyBlob

      @Deprecated protected boolean copyBlob(String key, S3BlobStore sourceBlobStore, String sourceKey, boolean move) throws com.amazonaws.AmazonServiceException
      Deprecated.
      since 11.5, use copyOrMoveBlob(String, S3BlobStore, String, boolean) instead
      Throws:
      com.amazonaws.AmazonServiceException

    • copyOrMoveBlob

      protected String copyOrMoveBlob(String key, S3BlobStore sourceBlobStore, String sourceKey, boolean move) throws com.amazonaws.AmazonServiceException
      Returns:
      null if generic copy is needed
      Throws:
      com.amazonaws.AmazonServiceException - if the source is missing

    • getMD5DigestFromETag

      protected String getMD5DigestFromETag(String bucketKey)
      Gets the MD5 of an object from its ETag, if possible.
      Since:
      11.5

    • copyBlob

      @Deprecated protected void copyBlob(S3BlobStoreConfiguration sourceConfig, String sourceKey, S3BlobStoreConfiguration destinationConfig, String destinationKey, boolean move)
      Deprecated.
      since 11.5, use copyOrMoveBlob(S3BlobStoreConfiguration, String, String, S3BlobStoreConfiguration, String, boolean) instead

    • copyOrMoveBlob

      protected String copyOrMoveBlob(S3BlobStoreConfiguration sourceConfig, String sourceKey, String sourceVersionId, S3BlobStoreConfiguration destinationConfig, String destinationKey, boolean move)
      Returns the version id, or null.

    • copyBlobGeneric

      @Deprecated protected boolean copyBlobGeneric(String key, BlobStore sourceStore, String sourceKey, boolean atomicMove) throws IOException
      Deprecated.
      since 11.5, use copyOrMoveBlobGeneric(String, BlobStore, String, boolean) instead
      Throws:
      IOException

    • copyOrMoveBlobGeneric

      protected String copyOrMoveBlobGeneric(String key, BlobStore sourceStore, String sourceKey, boolean atomicMove) throws IOException
      Throws:
      IOException

    • writeBlobProperties

      public void writeBlobProperties(BlobUpdateContext blobUpdateContext) throws IOException
      Description copied from interface: BlobStore
      Sets properties on a blob.
      Specified by:
      writeBlobProperties in interface BlobStore
      Overrides:
      writeBlobProperties in class AbstractBlobStore
      Parameters:
      blobUpdateContext - the blob update context
      Throws:
      IOException

    • deleteBlob

      public void deleteBlob(String key)
      Description copied from interface: BlobStore
      Deletes a blob based on a key. No error occurs if the blob does not exist.

      This method does not throw IOException, but may log an error message.

      Parameters:
      key - the blob key

    • getBinaryGarbageCollector

      public BinaryGarbageCollector getBinaryGarbageCollector()
      Description copied from interface: BlobStore
      Returns the binary garbage collector (GC).

      Several calls to this method will return the same GC, so that its status can be monitored using BinaryGarbageCollector.isInProgress().

      Returns:
      the binary GC