Addons

Technical Overview

Updated: May 4, 2020

Dedicated Low-Level Fields

To provide the required data to Amazon S3 (expiration date and legal hold) and to ensure that records can’t be deleted when not eligible, some new low-level fields have been added at the storage level:

  • ecm:isRecord (boolean): true if the document is declared as a record.
  • ecm:retainUntil (datetime): Expiration date (end of retention period).
  • ecm:hasLegalHold (boolean): true if there is a legal hold applied to the record.
  • ecm:isUnderRetentionOrLegalHold (boolean): true if the record is under retention and/or legal hold.

If a legal hold is applied to a document or if a retention rule is applied to a document, then:

  • Modification of the main blob (file:content) is prevented: this includes modification through restoring an older version.
  • Deletion (including recursive delete) is prevented: this is enforced at low-level and this information is made available per-document through a security policy.
  • ecm:isUnderRetentionOrLegalHold is set to true.

This applies in all situations, no exception, including for Administrators.

In case of event-based retention, the record is put on a temporary hold waiting for the event to occur (see Retention flow section). In this case, ecm:retainUntil is "indeterminate" (by setting the field to 9999-01-01T00:00:00.000+00:00), so deletion is prevented as if it was a date far in the future (this is turned into a legal hold at the S3 storage level).

A copy of a document resets its record, retention and legal hold information. This includes creating a new version (that’s why versioning must be disabled to be compliant). These fields don't apply to proxies, as proxies are just a convenience to access content stored elsewhere, and are not themselves records.

Retention Flow

When a granted user applies a retention rule to a document, the document is declared as a record and copied into the S3 compliance mode bucket.

Then, depending on the retention rule type (immediate, metadata, or event-based retention), the expiration date is calculated and applied to both Nuxeo and S3 object.

A background process checks when the event occurs and then starts the retention period (event-based retention).

Another background process checks the document expiration to change the status and make the record deletable. This process is run once a day (there may be a short delay for the record to become deletable).

retention-sequence-application
retention-sequence-application

When a granted user applies a legal hold to a record, the legal hold is applied at Nuxeo level and also at S3 level.

There are some specific use cases:

  • A legal hold is applied to a document which is not a record, the document is first declared as a record and moved to the compliance mode S3 bucket before applying the legal hold,

  • There is an event-based retention rule applied to the document and the event hasn't occurred yet, a temporary hold is applied to the record. So, the legal hold is only applied at Nuxeo level (as it is already done on S3 level),

  • The document is already on legal hold, there is no action done except to log the legal hold action on the audit log.

retention-sequencing-legal-hold
retention-sequencing-legal-hold

Deletion Flow

Standard Deletion Flow

The standard Nuxeo deletion flow is based on the following principles:

  • Immediate deletion of the file content referenced in the Nuxeo database when a granted user requests deletion action,

  • As Nuxeo handles deduplication of the file content (when 2 Nuxeo documents contain the exact same file, the file is stored only once), the deletion of the files in the storage media is done by the Garbage Collector process:

    • The Garbage Collector regularly compares the list of files stored on the storage media and the list of files referenced in Nuxeo Database as part of the document content,

    • If and only if a file is not referenced in any Nuxeo document (meaning all documents included the file have been deleted), the Garbage Collector process deletes the file in the storage media,

retention-standard-deletion
retention-standard-deletion

Record Deletion Flow

For the Retention Management addon, a specific deletion flow has been added for the following reasons:

  • The deduplication feature is turned off for records content as it is not compliant with retention features (ex: it's not possible to handle a case where 2 records referenced the same file content with 2 different retention period and the legal constraint to delete it right after the retention period),

  • For compliance purpose, we need to make sure the deletion is done when a granted user requests the deletion of a record and when Nuxeo logs the deletion event, which is complex to implement with an asynchronous deletion process,

That's why the record deletion flow is done as follow:

  • The Garbage Collector process is disabled for the S3 Compliance mode buckets.
  • The deletion process is done synchronously.

retention-sequencing-deletion
retention-sequencing-deletion

We'd love to hear your thoughts!

All fields required