Install in Kubernetes With Nuxeo Helm Chart

Introduction

The Nuxeo Helm Chart allows to install Nuxeo Platform on a Kubernetes cluster using the Helm package manager.

Currently, there is a single version of this chart for all maintained versions of Nuxeo, e.g.: 2023, 2021.

TL;DR

helm repo add nuxeo https://packages.nuxeo.com/repository/helm-releases-public/
helm install my-release nuxeo/nuxeo

Scope

By default, this chart deploys the strict minimum to have Nuxeo running with:

• A single Nuxeo node.
• emptyDir volume for binaries.
• H2 database.
• Opensearch embedded.
• Memory-only WorkManager and Nuxeo Streams.

To make a production-ready deployment of Nuxeo in Kubernetes, please read the sections below:

• Architecture
• Full Deployment
• Parameters

Architecture

Nuxeo can be deployed as a single node or API-Worker architecture, depending on the architecture parameter value.

Single Node

When setting architecture=singleNode, which is the default, the Nuxeo pod(s) handle the whole workload without any differentiation: HTTP requests and asynchronous processing (Nuxeo workers and stream processing). This is implemented by setting these nuxeo.conf properties in the chart's ConfigMap:

nuxeo.stream.processing.enabled=true
nuxeo.work.processing.enabled=true

Note that Clustering can be enabled by setting a number of replicas greater than 1, using the replicaCount parameter (default: 1). Enabling clustering in a single node architecture only allows to have mulitple Nuxeo pods, yet they are all "versatile", handling both the HTTP requests and asynchronous processing. To differentiate the Nuxeo pods according to the type of workload, please us the API-Worker architecture.

API-Worker

When setting architecture=api-worker, the Nuxeo pod(s) are differentiated according to the type of workload: the API pods exclusively handle HTTP requests, whereas the Worker pods exclusively handle asynchronous processing. This is implemented by setting different nuxeo.conf properties for each type of pod in the chart's ConfigMap:

API pods:

nuxeo.stream.processing.enabled=false
nuxeo.work.processing.enabled=false

Worker pods:

nuxeo.stream.processing.enabled=true
nuxeo.work.processing.enabled=true

The number of pods is set by the following parameters:

• Number of API pods: replicaCount (default 1)
• Number of Worker pods: workerCount (unset by default)

In an API-Worker architecture, Clustering is necessarily enabled.

Clustering

Clustering is enabled for an API-Worker architecture or a Single Node architecture with multiple replicas (replicaCount > 1).

When clustering is enabled, the following nuxeo.conf properties are set in the chart's ConfigMap:

nuxeo.cluster.enabled=true
nuxeo.cluster.nodeid=POD_UID

When installing the chart, an error occurs if all three conditions below are not met:

1. A cloud provider is enabled for binary storage, by setting either googleCloudStorage.enabled=true or amazonS3.enabled=true.
2. A database is enabled for metadata storage, by setting either mongodb.enabled=true or postgresql.enabled=true.
3. Kafka is enabled for the WorkManager, PubSub Service and Nuxeo Streams, by setting kafka.enabled=true.

Also, if Ingress is enabled with ingress.enabled=true, the following warning will guide you:

Nuxeo needs sticky session for cookie to work correctly. Unfortunately, this can not be achieved by the Nuxeo Helm Chart as it depends on your Ingress controller. See below some examples:
* NGINX - add the annotation below to the ingress:
    ingress:
      annotations:
        nginx.ingress.kubernetes.io/affinity: "cookie"
* Traefik - add the annotation below to the service:
    service:
      annotations:
        traefik.ingress.kubernetes.io/service.sticky.cookie: "true"

Installing the Chart

The nuxeo chart can be installed either:

• Remotely, from the Nuxeo Helm chart repository.
• Locally, from a checkout of the nuxeo-helm-chart GitHub repository.

Remote Installation

To install the chart from the Nuxeo Helm chart respository, add this repository to the Helm configuration:

helm repo add nuxeo https://packages.nuxeo.com/repository/helm-releases-public/

Then, use nuxeo/nuxeo as a replacement of NUXEO_CHART in the commands below.

Local Installation

To install the chart from a checkout of the nuxeo-helm-chart GitHub repository, use nuxeo as a replacement of NUXEO_CHART in the commands below.

Installation Command Samples

To install the chart:

helm install RELEASE_NAME NUXEO_CHART

You can override any value of the base values.yaml file by creating your own myvalues.yaml file and pass it to the helm install command:

helm install RELEASE_NAME NUXEO_CHART --values=myvalues.yaml

You can also pass values directly to the helm install command:

helm install RELEASE_NAME NUXEO_CHART --set nuxeo.image.tag=x.y.z

For example, to install some packages using a Nuxeo CLID:

helm install RELEASE_NAME NUXEO_CHART --set nuxeo.packages=nuxeo-web-ui,nuxeo-drive --set nuxeo.clid=NUXEO_CLID

To see the templates of the installed release:

helm get manifest RELEASE_NAME

Installation Errors

Besides possible errors when Clustering is enabled, the following errors might occur when installing the chart:

• Database mutual exclusion:

  If mongodb.enabled=true and postgresql.enabled=true, an error indicates that MongoDB and PostgreSQL databases cannot be enabled at the same time.

• Binary storage mutual exclusion:

  If two parameters are set to true out of these three: googleCloudStorage.enabled, amazonS3.enabled, persistentVolumeStorage.enabled, an error indicates that only one type of binary storage can be enabled among Google Cloud Storage, Amazon S3 and PersistentVolume.

Installation Warnings

Besides possible warnings when Clustering is enabled, the following warnings might occur when installing the chart:

• Binary storage recommendations:

  If no cloud provider is enabled for binary storage, by setting either googleCloudStorage.enabled=true or amazonS3.enabled=true, a warning indicates that using a cloud provider for binary storage is preferred for production. In this case, if persistentVolumeStorage.enabled=false, an extra warning indicates that by not enabling a persistent volume for binary storage, binaries will be stored in an emptyDir volume, thus not surviving a pod restart.

• Binary storage persistent volume access mode:

  If persistentVolumeStorage.enabled=true and persistentVolumeStorage.accessModes=["ReadWriteMany"], a warning indicates that enabling a persistent volume for binary storage with the ReadWriteMany access mode is not supported by the Nuxeo Helm Chart, and additional configuration might need to be added, looking at the File Storage documentation.

  If persistentVolumeStorage.enabled=true and persistentVolumeStorage.accessModes=["ReadWriteOnce"], a warning indicates that by enabling a persistent volume for binary storage mounted with the ReadWriteOnce access mode, the deployment won't be able to scale up in Kubernetes.

• Log storage persistent volume access mode:

  If logs.persistence.enabled=true, a warning indicates that this setup is not recommended for production, a log collector being preferred.

  If logs.persistence.enabled=true and logs.persistence.accessModes=["ReadWriteMany"], an extra warning indicates that enabling a persistent volume for log storage with the ReadWriteMany access mode is not supported by the Nuxeo Helm chart, and additional configuration might need to be added, otherwise every Nuxeo pod will write to the same server.log file.

  If logs.persistence.enabled=true and logs.persistence.accessModes=["ReadWriteOnce"], an extra warning indicates that by enabling a persistent volume for log storage mounted with the ReadWriteOnce access mode, the deployment won't be able to scale up in Kubernetes.

Upgrading an Existing Deployment

For example, to pull another Nuxeo image:

helm upgrade RELEASE_NAME NUXEO_CHART --set image.tag=2023.x

Uninstalling the Chart

helm uninstall RELEASE_NAME

Full Deployment

To deploy Nuxeo along with MinIO, MongoDB, Opensearch and Kafka, one way is to use Helmfile to deploy multiple charts. You can have a look at the helmfile used to test and validate a Nuxeo cluster deployment with the cluster environment.

To run it, you need to:

• Install the helmfile command line.
• Clone the nuxeo-helm-chart GitHub repository or download the helm directory.
• In the ci/helm directory, run:

NAMESPACE=NAMESPACE helmfile --namespace=NAMESPACE --environment=cluster sync

NAMESPACE is the target Kubernetes namespace for the various Helm releases of the Helmfile.

The parameters of the nuxeo chart, such as the Nuxeo image, can be configured in the values-nuxeo.yaml.gotmpl and cluster.yaml.gotmpl files.

The parameters of the minio, mongodb, opensearch and kafka charts can be configured in the values-CHART.yaml files.

Note: These values are just sample values. The related charts need a fine-grained configuration to be suitable for production, see the available values for each one of them:

• nuxeo
• minio
• mongodb
• opensearch
• kafka

Parameters

The following tables lists the configurable parameters of this chart and their default values. Also see the values.yaml file.