Distributed Architecture with Kafka

Updated: July 9, 2024

Architecture Description

Compared to the Compact Architecture with Kafka, this architecture includes a Kafka cluster and a Zookeeper cluster so that SPOF (single point of failure) no longer exists, and consequently provides a full high availability and fault tolerant Nuxeo architecture.

In this architecture:

  • A load balancer with sticky sessions is used.
  • A total of two machines are prepared for the application cluster. Each machine holds a Nuxeo server node and a reverse proxy. More machines can be added later for scalability purpose.
  • An Elasticsearch cluster with at least 3 nodes
  • A database cluster with at least 2 nodes
  • A Kafka cluster, with at least 2 nodes, as the default implementation for Nuxeo Stream
  • A Zookeeper cluster, with at least 3 nodes, to orchestrate the Kafka cluster nodes, Kafka topics, partitions etc.



At this stage, this architecture is fault tolerant and in high availability, and the architecture should be tuned depending on the kind of requests and loads.

→ Check the scalability options to:

  • Add Nuxeo nodes for specific loads (API node, worker node and import nodes)
  • Set up data partitioning with database sharding and a multi-repository approach