Blob Upload for Batch Processing

Motivations

The default way Automation deals with Blobs is to use the standard HTTP MultiPart Encoding.

This strategy can not fit when:

• Your client does not natively support multipart encoding; Ex: JavaScript (without using a Form), Android SDK 2.x.
• You have several files to send, but prefer to send them as separated chunk; Ex: You have an HTTP proxy that will limit POST size.
• You want to upload files as soon as possible and then run the operation when everything has been uploaded on the server; Ex: You upload pictures you select from a mobile device.

Uploading Files

The principle is that you can upload the file to the server using the URL:

POST /site/automation/batch/upload

You can do a simple POST with the payload containing your file.

However, you will need to set some custom HTTP headers:

Optionally depending on the HTTP client you are using you might need to add the Content-Length header to specify the size of the file in bytes.

The batch identifier should be common to all the files you want to upload and attach to the same batch. This identifier should be client side generated:

• GUID,
• Timestamp + random number,
• whatever that can be reasonably considered as unique.

The X-File-Idx is here in case you later want to reference the file by its index and also to keep track of the client side ordering: because the order the server receives the files may not be the same.

The files attached to the batch are stored on a temporary disk storage (inside java.io.tmp) until the batch is executed or dropped.

To drop a batch you must use:

GET /site/automation/batch/drop/{batchId}

Technically, a batch is automatically "started" when the first upload is received.

Executing a batch will automatically remove it.

Using Files From a Batch

Batch Execute

You can execute an automation chain or an automation operation using the blobs associated to a batch as input.

To place the blobs as input, call a specific batch operation, passing the operationId and batchId as parameter:

POST /site/automation/batch/execute
Accept: application/json+nxentity, */*
Content-Type: application/json+nxrequest; charset=UTF-8
X-NXDocumentProperties: *

{"params":{"operationId":"Chain.FileManager.ImportInSeam","batchId":"batch-1370282334917-258", ...},"context":{...}}

Optionally you can pass the fileIdx parameter to specify the index of the file inside the batch that you want to use as input of the chain or operation to execute.

This way of calling automation operation is actually used in the default UI to manage Drag&Drop:

1. Files are progressively uploaded to the server:

   • You can drop several sets of files,
   • There is a maximum number of concurrent uploads.
2. When upload is finished you can select the operation or chain to execute.

More info about drag and drop configuration.

Referencing a Blob From a Batch

An other option is to reference the file within the batch to create input parameters of an operation.

For that you can add a parameter of type properties that will automatically be resolved to the correct blob if the provided properties are the correct ones:

type = blob
length = 657656
mime-type = application/pdf
name = myfile.pdf
upload-batch = 989676879865765
upload-fileId = myfile.pdf

When using Java automation client, this would look like:

PropertyMap blobProp = new PropertyMap();
blobProp.set("type", "blob");
blobProp.set("length", new Long(blobUploading.getLength()));
blobProp.set("mime-type", blobUploading.getMimeType());
blobProp.set("name", blobToUpload.getFileName());
// set information for server side Blob mapping
blobProp.set("upload-batch", batchId);
blobProp.set("upload-fileId", blobUploading.getFileName());

Referencing a Blob from a JSON Document Resource

You can use the batchId property for blob in the JSON document you're sending to the REST API.

{
    "entity-type": "document",
    "repository": "default",
    "uid": "531d9636-46c2-497d-996b-1ae7a8f43e89",
    "path": "/default-domain",
    "type": "Domain",
    "state": "project",
    "versionLabel": "",
    "title": "Default domain",
    "lastModified": "2013-09-06T08:53:10.00Z",
    "properties": {
        "file:content": {
             "upload-batch":"batchId-20358",
             "upload-fileId":"0" // referencing the first file of the batch
         }
     },
    "facets": [
        "SuperSpace",
        "Folderish"
    ],
    "changeToken": "1378457590000",
    "contextParameters": {}
}

Java API