Amazon S3 Online Storage

The Amazon S3 Online Storage is a Nuxeo Binary Manager for S3. It stores Nuxeo's binaries (the attached documents) in an Amazon S3 bucket.

Before You Start

You should be familiar with Amazon S3 and be in possession of your credentials.

Installing the Package

This addon requires no specific installation steps. It can be installed like any other package from the Update Center or using the nuxeoctl command line.

Configuring the Package

In order to configure the package, you will need to change a few Nuxeo templates, and provide values for the configuration variables that define your S3 credentials, bucket and encryption choices

Specifying Your Amazon S3 Parameters

In nuxeo.conf, add the following lines:

nuxeo.s3storage.bucket=your_s3_bucket_name
nuxeo.s3storage.awsid=your_AWS_ACCESS_KEY_ID
nuxeo.s3storage.awssecret=your_AWS_SECRET_ACCESS_KEY

If you installed the bundle JAR manually instead of using the marketplace package you will also need:

nuxeo.core.binarymanager=org.nuxeo.ecm.core.storage.sql.S3BinaryManager

The bucket name is unique across all of Amazon, you should find something original and specific.

The file nuxeo.conf now contains S3 secret access keys, you should protect it from prying eyes.

You can also add the following optional parameters:

nuxeo.s3storage.region=us-west-1
nuxeo.s3storage.cachesize=100MB
nuxeo.s3storage.bucket.prefix=myfolder/

The region code can be:

for us-east-1 (the default), don't specify this parameter
for us-west-1 (Northern California), use us-west-1
for eu-west-1 (Ireland), use EU
for ap-southeast-1 (Singapore), use ap-southeast-1

Since 5.6, you can also use:

for us-west-2 (Oregon), us-west-2
for ap-southeast-2 (Tokyo), use ap-southeast-2
for sa-east-1 (Sao Paulo), use sa-east-1

Since 5.9.4, you can also use a bucket prefix to localize your binaries within specific S3 folder.

Crypto Options

With S3 you have the option of storing your data encrypted (note that the local cache will not be encrypted).

The S3 Binary Manager can use a keystore containing a keypair, but there are a few caveats to be aware of:

The Sun/Oracle JDK doesn't always allow the AES256 cipher which the AWS SDK uses internally. Depending on the US export restrictions for your country, you may be able to modify your JDK to use AES256 by installing the "Java Cryptography Extension Unlimited Strength Jurisdiction Policy Files". See the following link to download the files and installation instructions: http://www.oracle.com/technetwork/java/javase/downloads/index.html
Don't forget to specify the key algorithm if you create your keypair with the keytool command, as this won't work with the default (DSA). The S3 Binary Manager has been tested with a keystore generated with this command:
```
keytool -genkeypair -keystore </path/to/keystore/file> -alias <key alias> -storepass <keystore password> -keypass <key password> -dname <key distinguished name> -keyalg RSA
```
If you get keytool error: java.io.IOException: Incorrect AVA format, then ensure that the distinguished name parameter has a form such as: -dname <span style="color: rgb(34,34,34);">"CN=AWS S3 Key</span> <span style="color: rgb(34,34,34);">, O=example, DC=com".</span>

Don't forget to make backups of the /path/to/keystore/file file along with the store password, key alias and key password . If you lose them (for instance if the EC2 machine hosting the Nuxeo instance with the original keystore is lost) you will lose the ability to recover any encrypted blob from the S3 backet.

With all that above in mind, here are the crypto options that you can add to nuxeo.conf (they are all mandatory once you specify a keystore):

nuxeo.s3storage.crypt.keystore.file=/absolute/path/to/the/keystore/file
nuxeo.s3storage.crypt.keystore.password=the_keystore_password
nuxeo.s3storage.crypt.key.alias=the_key_alias
nuxeo.s3storage.crypt.key.password=the_key_password

The Nuxeo S3BinaryManager class is using S3 Client-Side Encryption instead of S3 Server-Side Encrytption. CSE is safer than SSE. With CSE an attacker need both access to the AWS credentials and the key to be able to access the unencrypted data while SSE will only require the potential attacker to provide the AWS credentials.

Connection Pool Options

Since Nuxeo 5.9.2 (and Nuxeo 5.8.0-HF06 and Nuxeo 5.6.0-HF30) you can configure the internal S3 connection pool. This pool has a size of 50 by default, so if you've configured Nuxeo to use more sessions than this and all the sessions are accessing S3, you may run out of connections.

The following parameters can be used to change some connection pool parameters (the defaults are shown):

nuxeo.s3storage.connection.max=50
nuxeo.s3storage.connection.retry=3
nuxeo.s3storage.connection.timeout=50000
nuxeo.s3storage.socket.timeout=50000

The timeouts are expressed in milliseconds.

You can read more about these parameters on the AWS ClientConfiguration documentation page.

Checking Your Configuration

To check that installation went well, you can check your startup logs and look for a line like:

INFO  [S3BinaryManager] Repository 'default' using S3BinaryManager

Don't forget to enable the INFO level for the group org.nuxeo in $NUXEO_HOME/lib/log4j.xml to see INFO level messages from Nuxeo classes.

If your configuration is incorrect, this line will be followed by some error messages describing the problems encountered.